Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You and Your Team.

Learn More →

Online Ratings of Neurosurgeons: An Examination of Web Data and its Implications

Online Ratings of Neurosurgeons: An Examination of Web Data and its Implications Abstract BACKGROUND Patient satisfaction ratings are increasingly used for hospital rankings, referral base and physician reimbursement. As such, online physician rating websites (PRWs) are quickly becoming a topic of interest. OBJECTIVE To analyze the distribution of neurosurgeons’ ratings on the 3 most widely used PRWs, and examine factors associated with positive and negative ratings. METHODS We used a key term search to identify board-certified neurosurgeons on 3 widely used PRWs: RateMD.com, Healthgrades.com, and Vitals.com. Data were collected on average rating and number of ratings. Demographic, training-related and practice-related data, as well as location of practice, and place of training were also collected. RESULTS Data was non-normally distributed (P < .001 for all 3). Having fewer reviews was associated with higher variance in ratings between PRWs for a given surgeon (odds ratio 0.99, P = .001). All surgeons below the 25th percentile with respect to the number of reviews that had been written about them were eliminated. Of the remaining surgeons (n = 3054), the median composite score was 4.11 out of 5, interquartile range (3.69, 4.44). Surgeons had higher median modified composite scores if they were fellowship-trained (P = .0001) or graduated from a top 25 medical school (P = .0117), but not if they graduated from a top 25 residency (P = .1056). Surgeons located in major cities had higher median composite scores (P = .0025). CONCLUSION Online ratings for neurosurgeons must be evaluated in context. Median ratings are generally high, but variable between websites. Median scores also vary among regions and practice settings. Higher scores were associated with ranking of medical school, recent graduation, and fellowship training completion. Physician ratings, Neurosurgeon ratings, Patient satisfaction, Online ratings, Physician rating websites ABBREVIATIONS ABBREVIATIONS IQR interquartile range OR odds ratio PRW physician rating website Patient satisfaction is an increasingly important metric for hospital rankings, referral bases, and even physician reimbursement.1-3 As such, online physician rating websites (PRWs) are quickly becoming a topic of interest for physicians, patients, and policy makers alike.1-3 First arising in 2004, PRWs have grown dramatically, with many new websites arising each year.1-5 Recent studies have suggested that up to 35% of patients may avoid seeing a physician with poor online ratings.1,6 However, without data to contextualize it, a surgeon's online rating is not informative. Moreover, while the online rating industry is both growing and influential, little is known about the validity and the integrity of such rating sites.4,6-15 PRWs are therefore important but poorly understood, warranting further investigation. The goal of this study is to better assess trends and correlations in PRWs amongst board-certified neurosurgeons. To the best of our knowledge, this is the first study to consider trends in this specific subset of physicians. METHODS PRW Data A key term search was done for all providers containing the key terms “Neurosurgery” and “Neurological Surgery” for the 3 most widely used provider rating websites; RateMD.com, Healthgrades.com, and Vitals.com.16 Search results were further narrowed to include only providers who were listed as American Board of Neurological Surgery board certified by their profile data. Data were collected for each of the remaining providers by means of third party freeware application “Webscraper.” The following information was collected for each provider: Average Rating, Number of Ratings, Age, Sex, Address, Medical School Attended, Graduation Date, Residency Attended, Completion Date, Fellowship Attended, Languages Spoken, Awards Earned, Conditions Treated, and Procedures Performed. Providers were manually matched between website data as categorized by full name and address. Data acquisition was completed on 7 April 2017. Data were subsequently cleaned and duplicate values were deleted. Medical School Rank was noted as determined by US News & World Report, Top Medical Schools 2017–Research.17 Residency Rank was determined through Doximity.com's Top Neurological Surgery Residency Programs–Reputation Rank.18 Providers who graduated from US medical schools ranked in the top 25 or from US residency programs ranked in the top 20 were identified. Each surgeon's practice location was categorized by state into one of the 9 US Census Bureau Divisions, as defined by most recent US Census (2010). US Census Bureau division are defined as follows—Division 1: New England (Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, and Vermont), Division 2: Mid-Atlantic (New Jersey, New York, and Pennsylvania), Division 3: East North Central (Illinois, Indiana, Michigan, Ohio, and Wisconsin), Division 4: West North Central (Iowa, Kansas, Minnesota, Missouri, Nebraska, North Dakota, and South Dakota), Division 5: South Atlantic (Delaware, Florida, Georgia, Maryland, North Carolina, South Carolina, Virginia, District of Columbia, and West Virginia), Division 6: East South Central (Alabama, Kentucky, Mississippi, and Tennessee), Division 7: West South Central (Arkansas, Louisiana, Oklahoma, and Texas), Division 8: Mountain (Arizona, Colorado, Idaho, Montana, Nevada, New Mexico, Utah, and Wyoming), Division 9: Pacific (Alaska, California, Hawaii, Oregon, and Washington). The city where each surgeon's practice was located was compared to 2010 US Census data to identify whether the practice was located in a major city, here defined as one of the 50 most populous cities in the USA. The distribution of the data from each PRW was assessed, and we identified providers with scores in the top and bottom quartile on each PRW. As each PRW used a 5-point rating system, and each surgeon's average rating and number of ratings were available, we calculated a composite score as a weighted average of all 3 PRWs. Stepwise, multivariable logistic regression was used to identify factors that independently predicted being rated in the top or bottom quartile for each PRW. We calculated the variance of scores between PRWs for each provider. We identified providers whose variance in score between PRWs was greater than the 75th percentile. Since having a lower number of ratings was found to be associated with increased variance between PRWs (see results), and was associated with having more extreme ratings (independently associated with being in the top or bottom quartile), we eliminated surgeons with too few ratings, defined as surgeons who were in the bottom quartile with respect to the number of reviews that had been written about them. We then performed a stepwise, multivariable logistic regression to identify factors that independently predicted being in the top or bottom quartile based on our composite score from the 3 PRWs, after censoring providers with too few ratings. Single-variable comparisons were also performed. Statistical Methods Microsoft Excel 2011 (Microsoft, Redmond, Washington) was used to manage data. Statistical analysis was performed using and Stata 12.0 (StataCorp, College Station, Texas). Skewness and kurtosis tests for normality were applied to all ratings data. Parametric data were given as mean ± standard deviation and compared using a t-test. Nonparametric data were compared using Wilcoxon rank-sum test (Mann–Whitney U-test), Chi-square test, or Fisher's exact test, as appropriate. Regression analysis was performed using stepwise, multivariable logistic regression, with an inclusion threshold for the multivariable model of P < .10 for candidate variables on single-variable logistic regression. A value of P < .05 was considered statistically significant. Figures were generated using Prism 6.0b (GraphPad Software Inc, La Jolla, California). RESULTS Provider Characteristics Data from all board-certified neurosurgeons with ratings in at least one of the most used PRWs were obtained (n = 3959). Out of these providers, 518 attended top 25 medical schools (13.1%), 311 were from top 25 residencies (7.86%), and 511 were fellowship-trained (12.9%; Table 1). The median number of years out of medical school for all providers was 24 with an interquartile range (IQR) of [16, 34], and the mean was 25.7 ± 11.2. Geographic area data as defined by the US Census Bureau was obtained for 2337 providers and unavailable for the other 1622 providers. Of the 2337 providers, 100 practice in New England (4.28%), 356 practice in the Mid-Atlantic (15.2%), 323 practice in the East North Central (13.8%), 148 practice in the West North Central (6.33%), 479 practice in the South Atlantic (20.5%), 152 practice in the East South Central (6.50%), 301 practice in the West South Central (12.9%), 138 practice in the Mountain (5.91%), and 337 practice in the Pacific (14.4%). TABLE 1. Provider Characteristics   All providers  Training (n, %)     Top 25 Medical School  518 (13.1%)   Top 25 Residency  311 (7.86%)   Fellowship-Trained  511 (12.9%)  Years Since MD (mean ± SD)  25.7 ± 11.2   Median  24   IQR  [16, 34]  Geographic Area (n, %)     New England  100 (4.28%)   Mid-Atlantic  356 (15.2%)   East North Central  323 (13.8%)   West North Central  148 (6.33%)   South Atlantic  479 (20.5%)   East South Central  152 (6.50%)   West South Central  301 (12.9%)   Mountain  138 (5.91%)   Pacific  337 (14.4%)    All providers  Training (n, %)     Top 25 Medical School  518 (13.1%)   Top 25 Residency  311 (7.86%)   Fellowship-Trained  511 (12.9%)  Years Since MD (mean ± SD)  25.7 ± 11.2   Median  24   IQR  [16, 34]  Geographic Area (n, %)     New England  100 (4.28%)   Mid-Atlantic  356 (15.2%)   East North Central  323 (13.8%)   West North Central  148 (6.33%)   South Atlantic  479 (20.5%)   East South Central  152 (6.50%)   West South Central  301 (12.9%)   Mountain  138 (5.91%)   Pacific  337 (14.4%)  View Large TABLE 1. Provider Characteristics   All providers  Training (n, %)     Top 25 Medical School  518 (13.1%)   Top 25 Residency  311 (7.86%)   Fellowship-Trained  511 (12.9%)  Years Since MD (mean ± SD)  25.7 ± 11.2   Median  24   IQR  [16, 34]  Geographic Area (n, %)     New England  100 (4.28%)   Mid-Atlantic  356 (15.2%)   East North Central  323 (13.8%)   West North Central  148 (6.33%)   South Atlantic  479 (20.5%)   East South Central  152 (6.50%)   West South Central  301 (12.9%)   Mountain  138 (5.91%)   Pacific  337 (14.4%)    All providers  Training (n, %)     Top 25 Medical School  518 (13.1%)   Top 25 Residency  311 (7.86%)   Fellowship-Trained  511 (12.9%)  Years Since MD (mean ± SD)  25.7 ± 11.2   Median  24   IQR  [16, 34]  Geographic Area (n, %)     New England  100 (4.28%)   Mid-Atlantic  356 (15.2%)   East North Central  323 (13.8%)   West North Central  148 (6.33%)   South Atlantic  479 (20.5%)   East South Central  152 (6.50%)   West South Central  301 (12.9%)   Mountain  138 (5.91%)   Pacific  337 (14.4%)  View Large Distribution of the Data Surgeons’ scores on all PRWs analyzed were non-normally distributed (Table 2; Figure 1). The median scores for all surgeons on Healthgrades.com, RateMDs.com, and Vitals.com were 4.2, with an IQR of [3.6, 4.7], 4.03 IQR [3.24, 4.83], and 4.2, IQR [3.7, 4.5], respectively. FIGURE 1. View largeDownload slide Distribution of patients’ ratings of neurosurgeons as seen on A, Healthgrades.com; B, RateMDs.com; C, Vitals.com; and D, using a modified composite score calculated as a weighted average of all 3 scores, censoring providers with too few ratings. FIGURE 1. View largeDownload slide Distribution of patients’ ratings of neurosurgeons as seen on A, Healthgrades.com; B, RateMDs.com; C, Vitals.com; and D, using a modified composite score calculated as a weighted average of all 3 scores, censoring providers with too few ratings. TABLE 2. Distribution of the Data   Median Score  IQR  Skewness  Skewness P-value  Kurtosis  Kurtosis P-value  Healthgrades.com  4.2  [3.6, 4.7]  −1.13  P < .0001  4.37  P < .0001  RateMDs.com  4.03  [3.24, 4.83]  −0.86  P < .0001  3.06  P = .537  Vitals.com  4.2  [3.7, 4.5]  −1.27  P < .0001  6.04  P < .0001    Median Score  IQR  Skewness  Skewness P-value  Kurtosis  Kurtosis P-value  Healthgrades.com  4.2  [3.6, 4.7]  −1.13  P < .0001  4.37  P < .0001  RateMDs.com  4.03  [3.24, 4.83]  −0.86  P < .0001  3.06  P = .537  Vitals.com  4.2  [3.7, 4.5]  −1.27  P < .0001  6.04  P < .0001  View Large TABLE 2. Distribution of the Data   Median Score  IQR  Skewness  Skewness P-value  Kurtosis  Kurtosis P-value  Healthgrades.com  4.2  [3.6, 4.7]  −1.13  P < .0001  4.37  P < .0001  RateMDs.com  4.03  [3.24, 4.83]  −0.86  P < .0001  3.06  P = .537  Vitals.com  4.2  [3.7, 4.5]  −1.27  P < .0001  6.04  P < .0001    Median Score  IQR  Skewness  Skewness P-value  Kurtosis  Kurtosis P-value  Healthgrades.com  4.2  [3.6, 4.7]  −1.13  P < .0001  4.37  P < .0001  RateMDs.com  4.03  [3.24, 4.83]  −0.86  P < .0001  3.06  P = .537  Vitals.com  4.2  [3.7, 4.5]  −1.27  P < .0001  6.04  P < .0001  View Large The median number of reviews per provider on Healthgrades.com was 11, IQR [5, 21]. The median number of reviews for providers ranked in the top or bottom 25th percentile on Healthgrades.com was 7, IQR [3, 17], as compared to 15, IQR [9, 23] for providers who whose ratings placed them between the 25th and 75th percentiles. This difference was statistically significant (P < .0001). The median number of reviews per provider on RateMDs.com was 4, IQR [2, 8]. The median number of reviews for providers ranked in the top or bottom 25th percentile on RateMDs.com was lower than for all other providers (2 [1, 5] vs 5 [3, 10], P < .0001). The median number of reviews per provider on Vitals.com was 19, IQR [10, 34]. The median number of reviews for providers ranked in the top or bottom 25th percentile on Vitals.com was lower than for all other providers (15 [6, 28] vs 23 [12, 36], P < .0001). Single-variable Analysis Analysis by Gender There was no difference between providers of different genders on Healthgrades.com (P = .633) or RateMDs.com (P = .461). There was a trend toward a difference favoring women on Vitals.com (P = .051). Women were no more likely to be rated in the top 25th percentile on Healthgrades.com (25.6% vs 24.4%, P = .6976), RateMDs.com (26.3% vs 24.4%, 0.6256), or Vitals.com (26.7% vs 20.5%, P = .1370). Analysis by Practice Setting Median ratings varied significantly among different US Census Bureau divisions (P < .0001, Figure 2). Providers whose practices are located in a major city had higher median ratings on Healthgrades.com (P < .0001) and RateMDs.com (P < .0001), but not on Vitals.com (P = .671). FIGURE 2. View largeDownload slide Neurosurgeons’ mean modified composite score ratings and their 95% confidence intervals (bars), stratified by location as defined by US Census Bureau divisions. US Census Bureau division are defined as follows: Division 1: New England (Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, and Vermont), Division 2: Mid-Atlantic (New Jersey, New York, and Pennsylvania), Division 3: East North Central (Illinois, Indiana, Michigan, Ohio, and Wisconsin), Division 4: West North Central (Iowa, Kansas, Minnesota, Missouri, Nebraska, North Dakota, and South Dakota), Division 5: South Atlantic (Delaware, Florida, Georgia, Maryland, North Carolina, South Carolina, Virginia, District of Columbia, and West Virginia), Division 6: East South Central (Alabama, Kentucky, Mississippi, and Tennessee), Division 7: West South Central (Arkansas, Louisiana, Oklahoma, and Texas), Division 8: Mountain (Arizona, Colorado, Idaho, Montana, Nevada, New Mexico, Utah, and Wyoming), Division 9: Pacific (Alaska, California, Hawaii, Oregon, and Washington). FIGURE 2. View largeDownload slide Neurosurgeons’ mean modified composite score ratings and their 95% confidence intervals (bars), stratified by location as defined by US Census Bureau divisions. US Census Bureau division are defined as follows: Division 1: New England (Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, and Vermont), Division 2: Mid-Atlantic (New Jersey, New York, and Pennsylvania), Division 3: East North Central (Illinois, Indiana, Michigan, Ohio, and Wisconsin), Division 4: West North Central (Iowa, Kansas, Minnesota, Missouri, Nebraska, North Dakota, and South Dakota), Division 5: South Atlantic (Delaware, Florida, Georgia, Maryland, North Carolina, South Carolina, Virginia, District of Columbia, and West Virginia), Division 6: East South Central (Alabama, Kentucky, Mississippi, and Tennessee), Division 7: West South Central (Arkansas, Louisiana, Oklahoma, and Texas), Division 8: Mountain (Arizona, Colorado, Idaho, Montana, Nevada, New Mexico, Utah, and Wyoming), Division 9: Pacific (Alaska, California, Hawaii, Oregon, and Washington). Analysis by Training-related Factors Providers who were fellowship-trained had better median ratings on Healthgrades.com (P = .047) and Vitals.com (P = .001), but not on RateMDs.com (P = .176; Figure 3). Providers who graduated from top 25 medical schools had higher median ratings on Healthgrades.com (P = .013), but not on Vitals.com (P = .421) or RateMDs.com (P = .194; Figure 4). Ratings for providers who trained at top 25 residencies were no different on any of the 3 websites (Healthgrades.comP = .652, Vitals.comP = .461, RateMDs.comP = .950). FIGURE 3. View largeDownload slide A comparison of neurosurgeons’ median ratings on 3 different only rating websites, stratified by whether the providers completed a postresidency fellowship (filled circles) vs not (open circles). For Healthgrades.com and Vitals.com, P < .05; for RateMDs.com, P > .05. FIGURE 3. View largeDownload slide A comparison of neurosurgeons’ median ratings on 3 different only rating websites, stratified by whether the providers completed a postresidency fellowship (filled circles) vs not (open circles). For Healthgrades.com and Vitals.com, P < .05; for RateMDs.com, P > .05. FIGURE 4. View largeDownload slide A comparison of neurosurgeons’ median ratings on 3 different only rating websites, stratified by whether the providers graduated from a medical school rank among the top 20 according to US News and World Report (filled circles) vs all other medical schools (open circles). FIGURE 4. View largeDownload slide A comparison of neurosurgeons’ median ratings on 3 different only rating websites, stratified by whether the providers graduated from a medical school rank among the top 20 according to US News and World Report (filled circles) vs all other medical schools (open circles). Factors Associated with Excellent Physician Ratings on Multivariable Analysis Healthgrades.com Data On multivariable analysis, having a Healthgrades.com rating in the top 25th percentile was positively associated with being located in a major city (odds ratio [OR] 2.00 [1.58, 2.54], P < .001), and having a practice located in New England (OR 2.03 [1.24, 3.31], P = .005), or the Mid-Atlantic (OR 1.71 [1.27, 2.31], P < .001), and was negatively associated with the number of reviews that were written (OR 0.94 [0.93, 0.95], P < .001). Having a Healthgrades.com rating in the bottom 25th percentile was positively associated with East South Central (OR 1.64 [1.12, 2.41], P = .011), and negatively associated with Mid-Atlantic (OR 0.63 [0.45, 0.89], P = .008), and being located in a major city (OR 0.69 [0.54, 0.89], P = .003). RateMDs.com Data Having a RateMDs.com rating in the top 25th percentile was positively associated with a practice located in the Pacific region (OR 1.54 [1.00, 2.37], P = .047), and was negatively associated with the number of reviews that were written (OR 0.86 [0.82, 0.90], P < .001), and the number of years since medical school graduation (OR 0.99 [0.97, 1.00], P = .029). Having a RateMDs.com rating in the bottom 25th percentile was negatively associated with the number of reviews that were written (OR 0.97 [0.95, 1.00], P = .019), and positively associated with years since graduation from medical school (OR 1.02 [1.01, 1.04], P = .001) and having a practice located in the East North Central region (OR 1.51 [1.04, 2.19], P = .030). Vitals.com Data Having a Vitals.com rating in the top 25th percentile was positively associated with having been fellowship-trained (OR 1.78 [1.29, 2.46], P < .001), graduating from a top 25 medical school (OR 1.65 [1.12, 2.45], P = .012), and graduating from a top 25 residency (OR 1.67 [1.10, 2.53], P = .015), and was negatively associated with the number of years since graduation from medical school (OR 0.96 [0.95, 0.98], P < .001) and the number of reviews that were written (OR 0.99 [0.98, 1.00], P = .007). No variable was independently associated with having a Vitals.com rating in the bottom 25th percentile, but fellowship training showed a trend toward a negative association (OR 0.74 [0.53, 1.04], P = .079) Analysis of the Number of Reviews and Variance of Reviews As all 3 online review websites used a numerical rating system from 1 to 5, we combined data across all 3 sites by taking a weighted average of their scores from each website, creating a composite score. The median number of reviews per provider on using our composite score was 21, IQR [8, 41]. The median number of reviews for providers ranked in the top or bottom 25th percentile on the composite score was lower than for all other providers (15 [5, 34] vs 27 [13, 47], P < .0001). We calculated the variance between providers’ scores between the 3 different websites investigated. The median variance between providers’ scores on the 3 different rating websites was 0.20 (0.07, 0.50). On multivariable regression, having high variance (>75th percentile) was negatively associated with the total number of reviews the provider received (OR 0.99 [0.99, 1.00], P = .001), and was positively associated with having fewer than 5 reviews specifically (OR 2.43 [1.00, 5.90], P = .05). As above, on multivariable regression, for all 3 online rating systems, the number of reviews written for a given provider was negatively associated with that provider being in the top 25th percentile or bottom 25th percentile (Healthgrades.com OR 0.94, P < .001; RateMD.com OR 0.86, P < .001; Vitals.com OR 0.99, P = .007). This negative association was also seen for the composite score we created from the providers’ other ratings (OR 0.99 [0.99, 0.99], P < .001). We therefore eliminated all providers who were below the 25th percentile with respect to the number of reviews that had been written about them (<8), resulting in a modified composite score distribution (Skewness –0.85, P < 0.0001; Kurtosis 3.98, P < .0001; Figure 1). Among the remaining providers, the median composite score was 4.11, with an IQR (3.69, 4.44). Single Variable and Multivariable Analysis of the Modified Composite Score Single Variable Analysis Providers had higher median-modified composite scores if they were fellowship-trained (P = .0001) or graduated from a top 25 medical school (P = .012), but not if they graduated from a top 25 residency (P = .106). Providers located in major cities had significantly higher median composite scores (P = .003, Table 3). TABLE 3. Single-Variable and Multivariable Analysis of the Modified Composite Score   P  OR  95% CI  Single-variable analysis     Top 25 Medical School  0.012         Top 25 Residency  0.106         Fellowship-Trained  0.0001         Practice in Major City  0.003        Multivariable Analysis     Top Quartile      Practice in Major City  0.001  1.63  1.23  2.17    Practice in New England  0.023  1.9  1.09  3.29    Practice in Mid-Atlantic  0.044  1.44  1.01  2.06    Practice in West North Central  0.082  0.55  0.27  1.08    Fellowship Training  <0.001  1.93  1.47  2.54    Years Since MD Graduation  <0.001  0.97  0.96  0.98   Bottom Quartile      Practice in Major City  0.062  0.78  0.59  1.01    Practice in East South Central  0.011  1.69  1.13  2.54    Practice in Mountain  0.007  1.79  1.17  2.74    P  OR  95% CI  Single-variable analysis     Top 25 Medical School  0.012         Top 25 Residency  0.106         Fellowship-Trained  0.0001         Practice in Major City  0.003        Multivariable Analysis     Top Quartile      Practice in Major City  0.001  1.63  1.23  2.17    Practice in New England  0.023  1.9  1.09  3.29    Practice in Mid-Atlantic  0.044  1.44  1.01  2.06    Practice in West North Central  0.082  0.55  0.27  1.08    Fellowship Training  <0.001  1.93  1.47  2.54    Years Since MD Graduation  <0.001  0.97  0.96  0.98   Bottom Quartile      Practice in Major City  0.062  0.78  0.59  1.01    Practice in East South Central  0.011  1.69  1.13  2.54    Practice in Mountain  0.007  1.79  1.17  2.74  View Large TABLE 3. Single-Variable and Multivariable Analysis of the Modified Composite Score   P  OR  95% CI  Single-variable analysis     Top 25 Medical School  0.012         Top 25 Residency  0.106         Fellowship-Trained  0.0001         Practice in Major City  0.003        Multivariable Analysis     Top Quartile      Practice in Major City  0.001  1.63  1.23  2.17    Practice in New England  0.023  1.9  1.09  3.29    Practice in Mid-Atlantic  0.044  1.44  1.01  2.06    Practice in West North Central  0.082  0.55  0.27  1.08    Fellowship Training  <0.001  1.93  1.47  2.54    Years Since MD Graduation  <0.001  0.97  0.96  0.98   Bottom Quartile      Practice in Major City  0.062  0.78  0.59  1.01    Practice in East South Central  0.011  1.69  1.13  2.54    Practice in Mountain  0.007  1.79  1.17  2.74    P  OR  95% CI  Single-variable analysis     Top 25 Medical School  0.012         Top 25 Residency  0.106         Fellowship-Trained  0.0001         Practice in Major City  0.003        Multivariable Analysis     Top Quartile      Practice in Major City  0.001  1.63  1.23  2.17    Practice in New England  0.023  1.9  1.09  3.29    Practice in Mid-Atlantic  0.044  1.44  1.01  2.06    Practice in West North Central  0.082  0.55  0.27  1.08    Fellowship Training  <0.001  1.93  1.47  2.54    Years Since MD Graduation  <0.001  0.97  0.96  0.98   Bottom Quartile      Practice in Major City  0.062  0.78  0.59  1.01    Practice in East South Central  0.011  1.69  1.13  2.54    Practice in Mountain  0.007  1.79  1.17  2.74  View Large Multivariable Regression Having a modified composite score in the top 25th percentile was negatively associated with increasing time since graduation from medical school (OR 0.97 [0.96, 0.98], P < .001), and positively associated with fellowship training (OR 1.93 [1.47, 2.54], P < .001), practice located in a major city (OR 1.63 [1.23, 2.17], P = .001), New England (OR 1.90 [1.09, 3.29], P = .023), or the Mid-Atlantic (OR 1.44 [1.01, 2.06], P = .044), and there was a trend toward significance for practices located in the West North Central region (OR 0.55 [0.27, 1.08], P = .082). Having a composite score in the bottom 25th percentile was positively associated with having a practice located in the East South Central region (OR 1.69 [1.13, 2.54], P = .011), Mountain (OR 1.79 [1.17, 2.74], P = .007), and there was a trend toward a negative association with being in a major city (OR 0.78 [0.59, 1.01], P = .062; Table 4) TABLE 4. Factors Associated With Excellent Physician Ratings on Multivariable Analysis   P  OR  95% CI  Healthgrades.com     Top Quartile      Practice in Major City  <.001  2  1.58  2.54    Practice in New England  .005  2.03  1.24  3.31    Practice in Mid-Atlantic  <.001  1.71  1.27  2.31    Number of Reviews Written  <.001  0.94  0.93  0.95   Bottom Quartile      Practice in East South Central  .011  1.64  1.12  2.41    Practice in Mid-Atlantic  .008  0.63  0.45  0.89    Practice in Major City  .003  0.69  0.54  0.89  RateMDs.com     Top Quartile      Practice in Pacific  .047  1.54  1  2.37    Number of Reviews Written  <.001  0.86  0.82  0.9    Years Since MD Graduation  .029  0.98  0.97  1   Bottom Quartile      Practice in East North Central  .03  1.51  1.04  2.19    Years Since MD Graduation  .001  1.02  1.01  1.04    Number of Reviews Written  .019  0.97  0.95  1  Vitals.com     Top Quartile      Top 25 Medical School  .012  1.65  1.12  2.45    Top 25 Residency  .015  1.67  1.1  2.53    Fellowship-Trained  <.001  1.78  1.29  2.46    Number of Reviews Written  .007  0.99  0.98  1    Years Since MD Graduation  <.001  0.96  0.95  0.98   Bottom Quartile      Fellowship-Trained  .079  0.74  0.53  1.04    P  OR  95% CI  Healthgrades.com     Top Quartile      Practice in Major City  <.001  2  1.58  2.54    Practice in New England  .005  2.03  1.24  3.31    Practice in Mid-Atlantic  <.001  1.71  1.27  2.31    Number of Reviews Written  <.001  0.94  0.93  0.95   Bottom Quartile      Practice in East South Central  .011  1.64  1.12  2.41    Practice in Mid-Atlantic  .008  0.63  0.45  0.89    Practice in Major City  .003  0.69  0.54  0.89  RateMDs.com     Top Quartile      Practice in Pacific  .047  1.54  1  2.37    Number of Reviews Written  <.001  0.86  0.82  0.9    Years Since MD Graduation  .029  0.98  0.97  1   Bottom Quartile      Practice in East North Central  .03  1.51  1.04  2.19    Years Since MD Graduation  .001  1.02  1.01  1.04    Number of Reviews Written  .019  0.97  0.95  1  Vitals.com     Top Quartile      Top 25 Medical School  .012  1.65  1.12  2.45    Top 25 Residency  .015  1.67  1.1  2.53    Fellowship-Trained  <.001  1.78  1.29  2.46    Number of Reviews Written  .007  0.99  0.98  1    Years Since MD Graduation  <.001  0.96  0.95  0.98   Bottom Quartile      Fellowship-Trained  .079  0.74  0.53  1.04  View Large TABLE 4. Factors Associated With Excellent Physician Ratings on Multivariable Analysis   P  OR  95% CI  Healthgrades.com     Top Quartile      Practice in Major City  <.001  2  1.58  2.54    Practice in New England  .005  2.03  1.24  3.31    Practice in Mid-Atlantic  <.001  1.71  1.27  2.31    Number of Reviews Written  <.001  0.94  0.93  0.95   Bottom Quartile      Practice in East South Central  .011  1.64  1.12  2.41    Practice in Mid-Atlantic  .008  0.63  0.45  0.89    Practice in Major City  .003  0.69  0.54  0.89  RateMDs.com     Top Quartile      Practice in Pacific  .047  1.54  1  2.37    Number of Reviews Written  <.001  0.86  0.82  0.9    Years Since MD Graduation  .029  0.98  0.97  1   Bottom Quartile      Practice in East North Central  .03  1.51  1.04  2.19    Years Since MD Graduation  .001  1.02  1.01  1.04    Number of Reviews Written  .019  0.97  0.95  1  Vitals.com     Top Quartile      Top 25 Medical School  .012  1.65  1.12  2.45    Top 25 Residency  .015  1.67  1.1  2.53    Fellowship-Trained  <.001  1.78  1.29  2.46    Number of Reviews Written  .007  0.99  0.98  1    Years Since MD Graduation  <.001  0.96  0.95  0.98   Bottom Quartile      Fellowship-Trained  .079  0.74  0.53  1.04    P  OR  95% CI  Healthgrades.com     Top Quartile      Practice in Major City  <.001  2  1.58  2.54    Practice in New England  .005  2.03  1.24  3.31    Practice in Mid-Atlantic  <.001  1.71  1.27  2.31    Number of Reviews Written  <.001  0.94  0.93  0.95   Bottom Quartile      Practice in East South Central  .011  1.64  1.12  2.41    Practice in Mid-Atlantic  .008  0.63  0.45  0.89    Practice in Major City  .003  0.69  0.54  0.89  RateMDs.com     Top Quartile      Practice in Pacific  .047  1.54  1  2.37    Number of Reviews Written  <.001  0.86  0.82  0.9    Years Since MD Graduation  .029  0.98  0.97  1   Bottom Quartile      Practice in East North Central  .03  1.51  1.04  2.19    Years Since MD Graduation  .001  1.02  1.01  1.04    Number of Reviews Written  .019  0.97  0.95  1  Vitals.com     Top Quartile      Top 25 Medical School  .012  1.65  1.12  2.45    Top 25 Residency  .015  1.67  1.1  2.53    Fellowship-Trained  <.001  1.78  1.29  2.46    Number of Reviews Written  .007  0.99  0.98  1    Years Since MD Graduation  <.001  0.96  0.95  0.98   Bottom Quartile      Fellowship-Trained  .079  0.74  0.53  1.04  View Large DISCUSSION PRWs are quickly becoming an important resource for patients, physicians, and policy makers alike. Studies have suggested that about a third of patients avoid seeing a physician with poor online ratings, yet little is understood about how to interpret surgeons’ PRW scores.1,6 Moreover, controversy exist about whether scores correlate with traditional clinical outcome metrics.3,8,19,20 It is therefore increasingly relevant to understand and contextualize the data on PRWs, in order to better understand the information that they provide. Demographic Effects Gender had no effect on PRW ratings in our analysis. This contrast with some existing literature shows that women are more likely to be rated positive reviews, though there is controversy about whether this association remains true universally.21-23 We found time since medical school graduation to be negatively associated with receiving ratings in the top 25th percentile on multivariate analysis for Vitals.com and RateMDs.com (OR 0.96, P < .001; OR 0.98, P = .029 respectively). This finding further remained significant on composite score multivariate analysis (OR 0.97, P < .001). These findings are consistent with previous literature documenting that increasing age leads to worse physician ratings on PRWs.9 Why older physicians receive lower ratings is not entirely clear. This could suggest that technological savviness and online presence plays a large role in PRW ratings, as older physicians are generally less apt to embrace new technological paradigm shifts. Of note, literature has reported worsening clinical outcomes correlating to increasing physician age, which could further explain the observed relationship.24 However, as no outcomes data were available, this hypothesis could not be explored. Data Distribution Each of the 3 most used PRWs had largely a positive skew with median ratings higher than 4 on a 5-point scale, which should be considered when interpreting surgeons’ ratings. While a rating of 4.0 may seem to suggest an above average performance on a 5-point scale, it is a comparatively average or slightly below average rating. These findings are consistent with previous literature citing that a majority of patients tend to give physicians positive ratings on online PRWs.20,25 Indeed, studies have noted that anywhere from 66% to 88% of reviews on PRWs are positive, perhaps contributing to the strong positive skew evidenced in the dataset.20,25 Our findings show that ratings with less reviews are less likely to be reliable and more likely to be at either a negative or positive extreme. Specifically, we found that surgeons with ratings in the top or bottom quartile had significantly fewer reviews than surgeons in the middle 50th percentile. Similarly, the number of reviews written for a provider was negatively correlated with positive ratings on multivariate analysis for all 3 PRWs. This may reflect the aforementioned propensity for patients to rate providers highly, likely combined with regression to the mean as more reviews are written over time. Moreover, on multivariable regression, having a larger variance of a surgeons’ ratings between PRWs was associated with having fewer reviews. Having 5 or fewer ratings on a single site was also independently associated with high interwebsite variance. As such, PRW profiles for surgeons with a small number of ratings may not accurately reflect true patient satisfaction and may require more data in order to come to more precise conclusions. To our knowledge, this is the first analysis to observe this effect on PRWs, and must be considered when interpreting a surgeon's PRW scores. Geographic Variation Our study further found that geographic location correlated with physician ratings. Providers located in major cities were found to have significantly higher composite scores on single-variable (P = .0025) and multivariable analysis (P = .001). As increasing trends in medicine favor “centers for excellence” for patient care, it has been hypothesized that many patients are opting to seek care in more urban settings, as many of such centers are often located in larger cities.26,27 There is controversy about whether such centers provide better care,28,29 but they are publicly perceived to do so.26-29 The higher PRW scores of surgeons at urban centers may reflect better care, the perception of better care, or a combination of both. Furthermore, our study found significant regional variation in ratings, as practices based out of New England, Mid-Atlantic, and West North Central regions were more likely to have ratings in the top 25 percentile, and practices in the East South Central and Mountain regions were more likely to have ratings in the bottom 25 percentile. Regional variation in PRWs has been described in the past, although to our knowledge, this is the first analysis to observe this effect on PRWs among neurosurgeons. Effects of Training On multivariable analysis, having ratings in the top quartile was positively associated with fellowship training and graduating from a top 25 medical school (Vitals.com OR 1.78, P < .001; OR 1.65, P = .012 respectively; OR 1.93, P < .001 for fellowship training on analysis of composite score). This was also reflected on single-variable analysis. It is reasonable for fellowship training to lead to better care or to the perception that a surgeon is providing better care, as has been described in other surgical disciplines.30,31 Of note, graduating from a top 20 residency program only showed significant association with higher scores on multivariable regression for one of 3 PRWs and not for composite score (Vitals.com, OR 1.67, P = .015). Surgical skills are best learned during residency and continue to be honed in practice. It is therefore unclear why graduating from a top 25 medical school may be associated with higher ratings, whereas graduating from a top 20 residency is not. This may be an example of the so-called halo effect, a cognitive bias wherein a good impression caused by 1 attribute leads to a good impression of other attributes that may or may not be related.32 Such effects have been described elsewhere in the surgical literature, but not for individual surgeons to our knowledge.33-35 The brand recognition of an excellent university is likely greater than the brand recognition of an excellent residency program, and the general public probably has more familiarity with university rankings than of residency rankings. As such, the halo effect may translate into better ratings for providers who attended highly ranked medical schools if patients perceive them to be better surgeons a priori. Limitations Our study is not without limitations. Due to the fluidity of online ratings, ratings one day may differ from ratings then next, making our analysis only relevant for a snapshot in time. Providers often change practice settings, which may confound our geographic analyses. We only analyzed data from the top 3 PRWs were included in the analysis, which may not be reflective of other PRWs. There is no verification process on such websites for board certification, which was one of our inclusion criteria for the providers we analyzed. There is no method in place on many of these websites to verify whether physicians listed are currently in practice, so providers who have subsequently retired may skew our data. We could not readily differentiate providers based on the type of surgeries they offered, eg, cranial vs spine, which may lead to comparisons between dissimilar surgical practices. We have no clinical outcomes data to analyze from these providers, and cannot therefore determine what outcomes driver surgeons’ ratings, if any. Moreover, we cannot determine whether factors outside of a surgeon's control, such as nursing staff or hospital facilities, impact a patient's impression of their care and their subsequent rating. Lastly, it should be noted that current medical school and residency rating systems are not perfect and similar to PRWs are only a snapshot in time, likely not reflecting the dynamic state of training over years past. Despite these limitations, our study is the first in the neurosurgical literature to examine PRWs, and to provide a context with which to interpret surgeons’ scores on PRWs, which is a topic of growing interest as patient satisfaction metrics continue to become more important. CONCLUSION Online ratings for neurosurgeons must be evaluated in context. Median ratings are approximately 4 on a 5-point scale, but vary between websites, and scores are not normally distributed. Ratings are more varied and more extreme when providers have fewer total ratings. Median scores also vary among regions and practice settings. Surgeons who graduated from highly ranked medical schools, who graduated from medical school more recently, and who are fellowship-trained tend to have higher ratings. Disclosure The authors have no personal, financial, or institutional interest in any of the drugs, materials, or devices described in this article. Notes Portions of this work were submitted as an abstract to the American Association of Neurosurgeons’ 2018 Annual Scientific Meeting in New Orleans, Louisiana, USA. REFERENCES 1. Hanauer DA, Zheng K, Singer DC, Gebremariam A, Davis MM. Public awareness, perception, and use of online physician rating sites. JAMA . 2014; 311( 7): 734- 735. Google Scholar CrossRef Search ADS PubMed  2. Huff DJ. Online physician rating websites. J Med Assoc Ga . 2013; 102( 2): 30, 34. 3. Jack RA 2nd, Burn MB, McCulloch PC, Liberman SR, Varner KE, Harris JD. Does experience matter? A meta-analysis of physician rating websites of orthopaedic surgeons. Musculoskelet Surg . 2017. 4. Hawkes N. Plethora of websites rating healthcare is unhelpful to public, says think tank. BMJ . 2015; 351: h5462. Google Scholar CrossRef Search ADS PubMed  5. Ricciardi BF, Waddell BS, Nodzo SR et al.  . Provider-initiated patient satisfaction reporting yields improved physician ratings relative to online rating websites. Orthopedics . 2017; 40( 5): 304- 310. Google Scholar CrossRef Search ADS PubMed  6. Galizzi MM, Miraldo M, Stavropoulou C et al.   Who is more likely to use doctor-rating websites, and why? A cross-sectional study in London. BMJ Open . 2012; 2( 6). 7. Adelhardt T, Emmert M, Sander U, Wambach V, Lindenthal J. Can patients rely on results of physician rating websites when selecting a physician? - A cross-sectional study assessing the association between online ratings and structural and quality of care measures from two German physician rating websites. Value Health . 2015; 18( 7): A545. Google Scholar CrossRef Search ADS PubMed  8. Emmert M, Adelhardt T, Sander U, Wambach V, Lindenthal J. A cross-sectional study assessing the association between online ratings and structural and quality of care measures: results from two German physician rating websites. BMC Health Serv Res . 2015; 15( 1): 414. Google Scholar CrossRef Search ADS PubMed  9. Emmert M, Meier F. An analysis of online evaluations on a physician rating website: evidence from a German public reporting instrument. J Med Internet Res . 2013; 15( 8): e157. Google Scholar CrossRef Search ADS PubMed  10. Ma L, Kaye AD, Bean M, Vo N, Ruan X. A five-star doctor? Online rating of physicians by patients in an internet driven world. Pain Physician . 2015; 18( 1): E15- E18. Google Scholar PubMed  11. Kirkpatrick W, Abboudi J, Kim N et al.  . An assessment of online reviews of hand surgeons. Arch Bone Jt Surg . 2017; 5( 3): 139- 144. Google Scholar PubMed  12. Lagu T, Metayer K, Moran M et al.   Website characteristics and physician reviews on commercial physician-rating websites. JAMA . 2017; 317( 7): 766- 768. Google Scholar CrossRef Search ADS PubMed  13. AlRuthia YS, Hong SH, Graff C, Kocak M, Solomon D, Nolly R. Exploring the factors that influence medication rating Web sites value to older adults: a cross-sectional study. Geriatric Nursing . 2016; 37( 1): 36- 43. Google Scholar CrossRef Search ADS PubMed  14. Burkle CM, Keegan MT. Popularity of internet physician rating sites and their apparent influence on patients’ choices of physicians. BMC Health Serv Res . 2015; 15( 1): 416. Google Scholar CrossRef Search ADS PubMed  15. Ramkumar PN, Navarro SM, Chughtai M, La T Jr., Fisch E, Mont MA. The patient experience: an analysis of orthopedic surgeon quality on physician-rating sites. J Arthroplasty . 2017; 32( 9): 2905- 2910. Google Scholar CrossRef Search ADS PubMed  16. Bakhsh W, Mesfin A. Online ratings of orthopedic surgeons: analysis of 2185 reviews. Am J Orthop . 2014; 43( 8): 359- 363. Google Scholar PubMed  17. Report UNW. Top Medical Schools- Research . 2017. Available at: https://www.usnews.com/best-graduate-schools/top-medical-schools/research-rankings. Accessed September 22, 2017. 18. Doximity. Neurological Surgery- Reputation . 2017. Available at: https://residency.doximity.com/programs?residency_specialty_id=46&sort_by=reputation. Accessed September 22, 2017. 19. Okike K, Peter-Bibb TK, Xie KC, Okike ON. Association between physician online rating and quality of care. J Med Internet Res . 2016; 18( 12): e324. Google Scholar CrossRef Search ADS PubMed  20. Lagu T, Hannon NS, Rothberg MB, Lindenauer PK. Patients’ evaluations of health care providers in the era of social networking: an analysis of physician-rating websites. J Gen Intern Med . 2010; 25( 9): 942- 946. Google Scholar CrossRef Search ADS PubMed  21. Ellimoottil C, Hart A, Greco K, Quek ML, Farooq A. Online reviews of 500 urologists. J Urol . 2013; 189( 6): 2269- 2273. Google Scholar CrossRef Search ADS PubMed  22. Kadry B, Santoro E, Zhang Q, Emmert M, Meier F. An analysis of online evaluations on a physician rating website: evidence from a german public reporting instrument. J Med Internet Res . 2013; 15( 8): e157. Google Scholar CrossRef Search ADS PubMed  23. Gao G, Greaves F, Emmert M, Sander U, Pisch F. Eight questions about physician-rating websites: a systematic review. J Med Internet Res . 2013; 15( 2): e24. Google Scholar CrossRef Search ADS PubMed  24. Tsugawa Y, Newhouse JP, Zaslavsky AM, Blumenthal DM, Jena AB. Physician age and outcomes in elderly patients in hospital in the US: observational study. BMJ . 2017; 357: j1797. Google Scholar CrossRef Search ADS PubMed  25. Kadry B, Chu LF, Kadry B, Gammas D, Macario A. Analysis of 4999 online physician ratings indicates that most patients give physicians a favorable rating. J Med Internet Res . 2011; 13( 4): e95. Google Scholar CrossRef Search ADS PubMed  26. Elrod JK, Fortenberry JL Jr. Centers of excellence in healthcare institutions: what they are and how to assemble them. BMC Health Serv Res . 2017; 17( S1): 425. Google Scholar CrossRef Search ADS PubMed  27. Sugerman DT. Centers of excellence. JAMA . 2013; 310( 9): 994. Google Scholar CrossRef Search ADS PubMed  28. Mehrotra A, Sloss EM, Hussey PS, Adams JL, Lovejoy S, SooHoo NF. Evaluation of a center of excellence program for spine surgery. Med Care . 2013; 51( 8): 748- 757. Google Scholar CrossRef Search ADS PubMed  29. Livingston EH. Bariatric surgery outcomes at designated centers of excellence vs nondesignated programs. Arch Surg . 2009; 144( 4): 319- 325; discussion 325. Google Scholar CrossRef Search ADS PubMed  30. Oliak D, Owens M, Schmidt HJ. Impact of fellowship training on the learning curve for laparoscopic gastric bypass. Obes Surg . 2004; 14( 2): 197- 200; 0960-8923. Google Scholar CrossRef Search ADS PubMed  31. Agrawal S. Impact of bariatric fellowship training on perioperative outcomes for laparoscopic Roux-en-Y gastric bypass in the first year as consultant surgeon. Obes Surg . 2011; 21( 12): 1817- 1821; 0960-8923. Google Scholar CrossRef Search ADS PubMed  32. Nisbett RE, Wilson TD. The halo effect: Evidence for unconscious alteration of judgments. J Person Social Psychol . 1977; 35( 4): 250- 256; 1939-1315. Google Scholar CrossRef Search ADS   33. Utter GH, Maier RV, Rivara FP, Nathens AB. Outcomes after ruptured abdominal aortic aneurysms: the “halo effect” of trauma center designation. J Am Coll Surg . 2006; 203( 4): 498- 505; 1072-7515. Google Scholar CrossRef Search ADS PubMed  34. Nagarajan N, Selvarajah S, Gani F et al.   “Halo effect” in trauma centers: does it extend to emergent colectomy? J Surg Res . 2016; 203( 1): 231- 237; 0022-4804. Google Scholar CrossRef Search ADS PubMed  35. Brown EG, Anderson JE, Burgess D, Bold RJ. Examining the “Halo Effect” of surgical care within health systems. JAMA Surg . 2016; 151( 10): 983- 984; 2168-6254. Google Scholar CrossRef Search ADS PubMed  COMMENTS This study shines the light of statistical analysis into the murky world of online physician rankings. This study is a first for neurosurgery, and the findings are consistent with other surgical specialties. While the results are not surprising, the demonstration of skew and bias in online rating platforms deserves notice. Based on this study, readers should consider the sources for perceptions of quality in neurosurgery. Hopefully this will direct further research into establishing concrete measures of quality and outcome within the specialty. Thomas Mattingly Richmond, Virginia Although this report is very limited in shedding any light on “physician rating websites”, it is an attempt to understand the trends in these now commonly used online instruments. There are no real surprises in the results in that surgeons who are younger, fellowship trained, and from better medical schools have better ratings. Urban practicing surgeons also have higher ratings, which fits the perception, especially in a highly specialized field such as neurosurgery, that surgeons in large medical centers are better. I think publishing this will spur others to look at this data in in a critical manner and may uncover inherent weaknesses or value in these systems. William F. Chandler Ann Arbor, Michigan Physician ranking is a currently unregulated and non-scientific method used by a biased population, patients. Current shifts in reimbursement are using patient satisfaction scores without understanding what, if any, usable objective data is present. This manuscript underscores this point and provides a framework to understand the bias involved in patient satisfaction scores. Jeffrey Steven Raskin Indianapolis, Indiana Research into various dimensions of quality in medicine has been undertaken for several decades.1 In 1998 the Institute of Medicine formed the Committee on Quality of Health Care in America. It set out to develop strategies that would improve the quality of health care by 2008.2, 3 Patient centered care rose from merely a high priority as measured by patient satisfaction surveys to a central part of health care quality assessment. Results of the patient satisfaction Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey comprise 30% of the total performance score of Hospital Value Based Purchasing, is tied to 2% of Medicare reimbursement.4 Not surprisingly, patient satisfaction with individual physicians and in the outpatient setting also came to command attention and substantial work has been done on the relationship between quality of outcomes and patient satisfaction.5, 6 The quality of physician-patient and nurse-patient interactions also clearly matter.7 With that said, patient satisfaction ratings (PSRs) are complicated instruments.8 The web-based systems described in this article are no less complicated and possibly more vulnerable to error because of the way their ratings are bestowed and disseminated. They are also a feature of the movement towards consumer-facing health care and media-based open information exchange. Like many rating systems, PSRs express subjective perceptions of quality whose vernacular authority derives from the ways they are used. Whether they matter because they exist or they exist because they matter cannot be determined. Irrespective of extent of validation against objective quality measures or any other rating standards, web-based PSRs will increasingly and unavoidably exert market influence. The question is how to respond. The practice of surgery and medicine is classified as a service. Services are characterized not only by the fact that they deal with the delivery of intangibles, unlike product-focused operations or industries that provide things that are tangible, but by the fact that the quality of the service cannot be fully assessed until it has been experienced. Insofar as all experiences are ultimately judged personally and subjectively, service industries rely heavily on client feedback for measures of quality and success. Reviews reflect the experiences of others. Review-based ratings, including the PSR, survey and aggregate the experiences of others so that potential clients might have a reasonable sense of what to expect, and a peer-anchored basis on which to decide whether to avail themselves of the services offered. The validity, usefulness, and relevance of any survey is limited by design specifications sampling considerations (including bias, sample size, and homogeneity) and by the range of questions asked and answered. In PSRs, for example, the patients most likely to respond are the most satisfied and the least satisfied, and this may constitute and important bias. Web-based PSRs are unlikely to undergo the testing and validation required for the HCAHPS (the Hospital Consumer Assessment of Healthcare Providers and Systems), for example, and it may be difficult to know how accurate and how representative they are. Nevertheless, for purposes of the casual guidance sought by most patients, they are deemed relevant enough because of inferred confidence in social media surveys. Relevance, validity, and meaning are attributed to consistency of the reported experience even though the questions that are asked may be biased and the questionnaire itself may be flawed. Surgeons have long responded to patient satisfaction surveys by saying that they are focused on saving lives and not on popularity contests. Their concerns follow patient outcomes and other traditional professional standards validated by the profession, even though they may not be fully appreciated either by the patient or by other relevant clients including payers. They accept the existence of disparities of value. How should surgeons respond in the era PSR and similar instruments of satisfaction? When McDonald's (a mixed model of service and food product) first opened in Moscow's Red Square, it had no competition. Service was poor and the lines were unimaginably long. Nevertheless, there were few complaints because it was Western. McDonald's was perceived to different from, and better than the alternatives, such as they may have been. And it probably was. McDonald's did not adhere to the same standard of service in Moscow as it did in St. Louis because it didn’t have to. As the level of consumer sophistication in Moscow rose, standards of service changed. Indeed, as a general rule, interest in and sensitivity to client feedback is heavily influenced by the competitive landscape and access to information about the competition. Health care is subject to the same rules. While health care has been historically construed as local phenomenon, geographic barriers are no longer as relevant as they once were. At one time, the characteristics of surgical services in Seattle were unlikely to be relevant in Key West, but in the era of consumer-facing health care, patients are increasingly encouraged to look around broadly before deciding where to have their care. They are led to integrate objective measures including price with subjective, client-based measures of satisfaction. For this and similar reasons, it is becoming increasing important to rethink standards of quality in neurosurgical practice. Patient outcomes are a given, but patient satisfaction must be targeted no less carefully. There is only 1 way to be sure that neurosurgical services successfully address patient needs, and that is to inquire, formally, thoughtfully, and consistently, and to respond appropriately. That is not necessarily the job of the surgeon, but in most cases, surgeons should oversee the process. There are good guidelines for team-based practices in this regard.9, 10, 11, 12 Whether web-based PSRs or other forms of patient satisfaction survey, current survey tools assess convenience, communication, promptness, access, and other traditional service-based quality measures. Patients are now encouraged explicitly to expect both good outcomes and a good experience. These are all factors that influence surgeon ratings. While consumer facing tools to assess quality and rate surgeons may be blunt, they cannot be avoided, and should not be ignored. It is important to understand them both in order to optimize patient care and to enhance patient satisfaction. T. Forcht Dagi Newton Centre, Massachusetts 1. Sitzia J, Wood N. Patient satisfaction: A review of issues and concepts. Social Science & Medicine . 1997: 45( i): 1829- 1843. Google Scholar CrossRef Search ADS   2. Kennedy GD, Tevis SE, Kent CK. Is There a Relationship Between Patient Satisfaction and Favorable Outcomes? Ann Surg . 2014; 260( 4): 592- 600. Google Scholar CrossRef Search ADS PubMed  3. Institute of Medicine (U.S.). Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century . 2001. Washington, D.C.: National Academy Press. 4. Petrullo K, Lamar S, Nwankwo-Otti O, Alexander-Mills K, Viola D. The Patient Satisfaction Survey: What does it mean to your bottom line? J Hosp Adm . 2012; 2: 1- 8. 5. Isaac T, Zaslavsky AM, Cleary PD et al. The relationship between patients' perception of care and measures of hospital quality and safety. Health Serv Res . 2010; 45: 1024- 1040. Google Scholar CrossRef Search ADS PubMed  6. Manary MP, Boulding W, Staelin R, Glickman SW. The Patient Experience and Health Outcomes. NEJM 2012 N Engl J Med  2013; 368: 201- 203. Google Scholar PubMed  7. Iannuzzi JC, Kahn SA, Zhang L, Gestring ML, Noyes K, Monson JR. Getting satisfaction: drivers of surgical Hospital Consumer Assessment of Health care Providers and Systems survey scores. J Surg Res . 2015 Jul; 197( 1): 155- 61. Epub 2015 Mar 24. 8. Bachman JW. The Problem with patient satisfaction scores. Fam Pract Manag . 2016; 23( 1): 23- 27. Google Scholar PubMed  9. Ogrinc GS, Headrick LA, Moore SM, Barton AJ, Dolansky MA, Madigosky WS. Fundamentals of Health Care Improvement: A Guide to Improving Your Patients' Care . 2nd ed. 2012. Oakbrook, IL: Joint Commission Resources and Institute for Healthcare Improvement. 10. Leon Guerrero CR, Anderson T, Zazulia AR. Education Research: Physician identification and patient satisfaction on an academic neurology inpatient service. Neurology . 2018. pii: 10.1212/WNL.0000000000004961. doi: 10.1212/WNL.0000000000004961. [Epub ahead of print]. 11. Fregene T, Wintle S, Venkat Raman V, Edmond H, Rizvi S. Making the experience of elective surgery better. BMJ Open Qual . 2017 Aug 9; 6( 2): e000079. doi: 10.1136/bmjoq-2017-000079. eCollection 2017. Google Scholar CrossRef Search ADS   12. de Vos MS, Hamming JF, Marang-van de Mheen PJ. The problem with using patient complaints for improvement. BMJ Qual Saf . 2018. pii: bmjqs-2017-0 07463. doi: 10.1136/bmjqs-2017-007463. [Epub ahead of print]. Copyright © 2018 by the Congress of Neurological Surgeons http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Neurosurgery Oxford University Press

Online Ratings of Neurosurgeons: An Examination of Web Data and its Implications

Loading next page...
 
/lp/oxford-university-press/online-ratings-of-neurosurgeons-an-examination-of-web-data-and-its-EHMh0BwkUh
Publisher
Oxford University Press
Copyright
Copyright © 2018 by the Congress of Neurological Surgeons
ISSN
0148-396X
eISSN
1524-4040
DOI
10.1093/neuros/nyy064
Publisher site
See Article on Publisher Site

Abstract

Abstract BACKGROUND Patient satisfaction ratings are increasingly used for hospital rankings, referral base and physician reimbursement. As such, online physician rating websites (PRWs) are quickly becoming a topic of interest. OBJECTIVE To analyze the distribution of neurosurgeons’ ratings on the 3 most widely used PRWs, and examine factors associated with positive and negative ratings. METHODS We used a key term search to identify board-certified neurosurgeons on 3 widely used PRWs: RateMD.com, Healthgrades.com, and Vitals.com. Data were collected on average rating and number of ratings. Demographic, training-related and practice-related data, as well as location of practice, and place of training were also collected. RESULTS Data was non-normally distributed (P < .001 for all 3). Having fewer reviews was associated with higher variance in ratings between PRWs for a given surgeon (odds ratio 0.99, P = .001). All surgeons below the 25th percentile with respect to the number of reviews that had been written about them were eliminated. Of the remaining surgeons (n = 3054), the median composite score was 4.11 out of 5, interquartile range (3.69, 4.44). Surgeons had higher median modified composite scores if they were fellowship-trained (P = .0001) or graduated from a top 25 medical school (P = .0117), but not if they graduated from a top 25 residency (P = .1056). Surgeons located in major cities had higher median composite scores (P = .0025). CONCLUSION Online ratings for neurosurgeons must be evaluated in context. Median ratings are generally high, but variable between websites. Median scores also vary among regions and practice settings. Higher scores were associated with ranking of medical school, recent graduation, and fellowship training completion. Physician ratings, Neurosurgeon ratings, Patient satisfaction, Online ratings, Physician rating websites ABBREVIATIONS ABBREVIATIONS IQR interquartile range OR odds ratio PRW physician rating website Patient satisfaction is an increasingly important metric for hospital rankings, referral bases, and even physician reimbursement.1-3 As such, online physician rating websites (PRWs) are quickly becoming a topic of interest for physicians, patients, and policy makers alike.1-3 First arising in 2004, PRWs have grown dramatically, with many new websites arising each year.1-5 Recent studies have suggested that up to 35% of patients may avoid seeing a physician with poor online ratings.1,6 However, without data to contextualize it, a surgeon's online rating is not informative. Moreover, while the online rating industry is both growing and influential, little is known about the validity and the integrity of such rating sites.4,6-15 PRWs are therefore important but poorly understood, warranting further investigation. The goal of this study is to better assess trends and correlations in PRWs amongst board-certified neurosurgeons. To the best of our knowledge, this is the first study to consider trends in this specific subset of physicians. METHODS PRW Data A key term search was done for all providers containing the key terms “Neurosurgery” and “Neurological Surgery” for the 3 most widely used provider rating websites; RateMD.com, Healthgrades.com, and Vitals.com.16 Search results were further narrowed to include only providers who were listed as American Board of Neurological Surgery board certified by their profile data. Data were collected for each of the remaining providers by means of third party freeware application “Webscraper.” The following information was collected for each provider: Average Rating, Number of Ratings, Age, Sex, Address, Medical School Attended, Graduation Date, Residency Attended, Completion Date, Fellowship Attended, Languages Spoken, Awards Earned, Conditions Treated, and Procedures Performed. Providers were manually matched between website data as categorized by full name and address. Data acquisition was completed on 7 April 2017. Data were subsequently cleaned and duplicate values were deleted. Medical School Rank was noted as determined by US News & World Report, Top Medical Schools 2017–Research.17 Residency Rank was determined through Doximity.com's Top Neurological Surgery Residency Programs–Reputation Rank.18 Providers who graduated from US medical schools ranked in the top 25 or from US residency programs ranked in the top 20 were identified. Each surgeon's practice location was categorized by state into one of the 9 US Census Bureau Divisions, as defined by most recent US Census (2010). US Census Bureau division are defined as follows—Division 1: New England (Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, and Vermont), Division 2: Mid-Atlantic (New Jersey, New York, and Pennsylvania), Division 3: East North Central (Illinois, Indiana, Michigan, Ohio, and Wisconsin), Division 4: West North Central (Iowa, Kansas, Minnesota, Missouri, Nebraska, North Dakota, and South Dakota), Division 5: South Atlantic (Delaware, Florida, Georgia, Maryland, North Carolina, South Carolina, Virginia, District of Columbia, and West Virginia), Division 6: East South Central (Alabama, Kentucky, Mississippi, and Tennessee), Division 7: West South Central (Arkansas, Louisiana, Oklahoma, and Texas), Division 8: Mountain (Arizona, Colorado, Idaho, Montana, Nevada, New Mexico, Utah, and Wyoming), Division 9: Pacific (Alaska, California, Hawaii, Oregon, and Washington). The city where each surgeon's practice was located was compared to 2010 US Census data to identify whether the practice was located in a major city, here defined as one of the 50 most populous cities in the USA. The distribution of the data from each PRW was assessed, and we identified providers with scores in the top and bottom quartile on each PRW. As each PRW used a 5-point rating system, and each surgeon's average rating and number of ratings were available, we calculated a composite score as a weighted average of all 3 PRWs. Stepwise, multivariable logistic regression was used to identify factors that independently predicted being rated in the top or bottom quartile for each PRW. We calculated the variance of scores between PRWs for each provider. We identified providers whose variance in score between PRWs was greater than the 75th percentile. Since having a lower number of ratings was found to be associated with increased variance between PRWs (see results), and was associated with having more extreme ratings (independently associated with being in the top or bottom quartile), we eliminated surgeons with too few ratings, defined as surgeons who were in the bottom quartile with respect to the number of reviews that had been written about them. We then performed a stepwise, multivariable logistic regression to identify factors that independently predicted being in the top or bottom quartile based on our composite score from the 3 PRWs, after censoring providers with too few ratings. Single-variable comparisons were also performed. Statistical Methods Microsoft Excel 2011 (Microsoft, Redmond, Washington) was used to manage data. Statistical analysis was performed using and Stata 12.0 (StataCorp, College Station, Texas). Skewness and kurtosis tests for normality were applied to all ratings data. Parametric data were given as mean ± standard deviation and compared using a t-test. Nonparametric data were compared using Wilcoxon rank-sum test (Mann–Whitney U-test), Chi-square test, or Fisher's exact test, as appropriate. Regression analysis was performed using stepwise, multivariable logistic regression, with an inclusion threshold for the multivariable model of P < .10 for candidate variables on single-variable logistic regression. A value of P < .05 was considered statistically significant. Figures were generated using Prism 6.0b (GraphPad Software Inc, La Jolla, California). RESULTS Provider Characteristics Data from all board-certified neurosurgeons with ratings in at least one of the most used PRWs were obtained (n = 3959). Out of these providers, 518 attended top 25 medical schools (13.1%), 311 were from top 25 residencies (7.86%), and 511 were fellowship-trained (12.9%; Table 1). The median number of years out of medical school for all providers was 24 with an interquartile range (IQR) of [16, 34], and the mean was 25.7 ± 11.2. Geographic area data as defined by the US Census Bureau was obtained for 2337 providers and unavailable for the other 1622 providers. Of the 2337 providers, 100 practice in New England (4.28%), 356 practice in the Mid-Atlantic (15.2%), 323 practice in the East North Central (13.8%), 148 practice in the West North Central (6.33%), 479 practice in the South Atlantic (20.5%), 152 practice in the East South Central (6.50%), 301 practice in the West South Central (12.9%), 138 practice in the Mountain (5.91%), and 337 practice in the Pacific (14.4%). TABLE 1. Provider Characteristics   All providers  Training (n, %)     Top 25 Medical School  518 (13.1%)   Top 25 Residency  311 (7.86%)   Fellowship-Trained  511 (12.9%)  Years Since MD (mean ± SD)  25.7 ± 11.2   Median  24   IQR  [16, 34]  Geographic Area (n, %)     New England  100 (4.28%)   Mid-Atlantic  356 (15.2%)   East North Central  323 (13.8%)   West North Central  148 (6.33%)   South Atlantic  479 (20.5%)   East South Central  152 (6.50%)   West South Central  301 (12.9%)   Mountain  138 (5.91%)   Pacific  337 (14.4%)    All providers  Training (n, %)     Top 25 Medical School  518 (13.1%)   Top 25 Residency  311 (7.86%)   Fellowship-Trained  511 (12.9%)  Years Since MD (mean ± SD)  25.7 ± 11.2   Median  24   IQR  [16, 34]  Geographic Area (n, %)     New England  100 (4.28%)   Mid-Atlantic  356 (15.2%)   East North Central  323 (13.8%)   West North Central  148 (6.33%)   South Atlantic  479 (20.5%)   East South Central  152 (6.50%)   West South Central  301 (12.9%)   Mountain  138 (5.91%)   Pacific  337 (14.4%)  View Large TABLE 1. Provider Characteristics   All providers  Training (n, %)     Top 25 Medical School  518 (13.1%)   Top 25 Residency  311 (7.86%)   Fellowship-Trained  511 (12.9%)  Years Since MD (mean ± SD)  25.7 ± 11.2   Median  24   IQR  [16, 34]  Geographic Area (n, %)     New England  100 (4.28%)   Mid-Atlantic  356 (15.2%)   East North Central  323 (13.8%)   West North Central  148 (6.33%)   South Atlantic  479 (20.5%)   East South Central  152 (6.50%)   West South Central  301 (12.9%)   Mountain  138 (5.91%)   Pacific  337 (14.4%)    All providers  Training (n, %)     Top 25 Medical School  518 (13.1%)   Top 25 Residency  311 (7.86%)   Fellowship-Trained  511 (12.9%)  Years Since MD (mean ± SD)  25.7 ± 11.2   Median  24   IQR  [16, 34]  Geographic Area (n, %)     New England  100 (4.28%)   Mid-Atlantic  356 (15.2%)   East North Central  323 (13.8%)   West North Central  148 (6.33%)   South Atlantic  479 (20.5%)   East South Central  152 (6.50%)   West South Central  301 (12.9%)   Mountain  138 (5.91%)   Pacific  337 (14.4%)  View Large Distribution of the Data Surgeons’ scores on all PRWs analyzed were non-normally distributed (Table 2; Figure 1). The median scores for all surgeons on Healthgrades.com, RateMDs.com, and Vitals.com were 4.2, with an IQR of [3.6, 4.7], 4.03 IQR [3.24, 4.83], and 4.2, IQR [3.7, 4.5], respectively. FIGURE 1. View largeDownload slide Distribution of patients’ ratings of neurosurgeons as seen on A, Healthgrades.com; B, RateMDs.com; C, Vitals.com; and D, using a modified composite score calculated as a weighted average of all 3 scores, censoring providers with too few ratings. FIGURE 1. View largeDownload slide Distribution of patients’ ratings of neurosurgeons as seen on A, Healthgrades.com; B, RateMDs.com; C, Vitals.com; and D, using a modified composite score calculated as a weighted average of all 3 scores, censoring providers with too few ratings. TABLE 2. Distribution of the Data   Median Score  IQR  Skewness  Skewness P-value  Kurtosis  Kurtosis P-value  Healthgrades.com  4.2  [3.6, 4.7]  −1.13  P < .0001  4.37  P < .0001  RateMDs.com  4.03  [3.24, 4.83]  −0.86  P < .0001  3.06  P = .537  Vitals.com  4.2  [3.7, 4.5]  −1.27  P < .0001  6.04  P < .0001    Median Score  IQR  Skewness  Skewness P-value  Kurtosis  Kurtosis P-value  Healthgrades.com  4.2  [3.6, 4.7]  −1.13  P < .0001  4.37  P < .0001  RateMDs.com  4.03  [3.24, 4.83]  −0.86  P < .0001  3.06  P = .537  Vitals.com  4.2  [3.7, 4.5]  −1.27  P < .0001  6.04  P < .0001  View Large TABLE 2. Distribution of the Data   Median Score  IQR  Skewness  Skewness P-value  Kurtosis  Kurtosis P-value  Healthgrades.com  4.2  [3.6, 4.7]  −1.13  P < .0001  4.37  P < .0001  RateMDs.com  4.03  [3.24, 4.83]  −0.86  P < .0001  3.06  P = .537  Vitals.com  4.2  [3.7, 4.5]  −1.27  P < .0001  6.04  P < .0001    Median Score  IQR  Skewness  Skewness P-value  Kurtosis  Kurtosis P-value  Healthgrades.com  4.2  [3.6, 4.7]  −1.13  P < .0001  4.37  P < .0001  RateMDs.com  4.03  [3.24, 4.83]  −0.86  P < .0001  3.06  P = .537  Vitals.com  4.2  [3.7, 4.5]  −1.27  P < .0001  6.04  P < .0001  View Large The median number of reviews per provider on Healthgrades.com was 11, IQR [5, 21]. The median number of reviews for providers ranked in the top or bottom 25th percentile on Healthgrades.com was 7, IQR [3, 17], as compared to 15, IQR [9, 23] for providers who whose ratings placed them between the 25th and 75th percentiles. This difference was statistically significant (P < .0001). The median number of reviews per provider on RateMDs.com was 4, IQR [2, 8]. The median number of reviews for providers ranked in the top or bottom 25th percentile on RateMDs.com was lower than for all other providers (2 [1, 5] vs 5 [3, 10], P < .0001). The median number of reviews per provider on Vitals.com was 19, IQR [10, 34]. The median number of reviews for providers ranked in the top or bottom 25th percentile on Vitals.com was lower than for all other providers (15 [6, 28] vs 23 [12, 36], P < .0001). Single-variable Analysis Analysis by Gender There was no difference between providers of different genders on Healthgrades.com (P = .633) or RateMDs.com (P = .461). There was a trend toward a difference favoring women on Vitals.com (P = .051). Women were no more likely to be rated in the top 25th percentile on Healthgrades.com (25.6% vs 24.4%, P = .6976), RateMDs.com (26.3% vs 24.4%, 0.6256), or Vitals.com (26.7% vs 20.5%, P = .1370). Analysis by Practice Setting Median ratings varied significantly among different US Census Bureau divisions (P < .0001, Figure 2). Providers whose practices are located in a major city had higher median ratings on Healthgrades.com (P < .0001) and RateMDs.com (P < .0001), but not on Vitals.com (P = .671). FIGURE 2. View largeDownload slide Neurosurgeons’ mean modified composite score ratings and their 95% confidence intervals (bars), stratified by location as defined by US Census Bureau divisions. US Census Bureau division are defined as follows: Division 1: New England (Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, and Vermont), Division 2: Mid-Atlantic (New Jersey, New York, and Pennsylvania), Division 3: East North Central (Illinois, Indiana, Michigan, Ohio, and Wisconsin), Division 4: West North Central (Iowa, Kansas, Minnesota, Missouri, Nebraska, North Dakota, and South Dakota), Division 5: South Atlantic (Delaware, Florida, Georgia, Maryland, North Carolina, South Carolina, Virginia, District of Columbia, and West Virginia), Division 6: East South Central (Alabama, Kentucky, Mississippi, and Tennessee), Division 7: West South Central (Arkansas, Louisiana, Oklahoma, and Texas), Division 8: Mountain (Arizona, Colorado, Idaho, Montana, Nevada, New Mexico, Utah, and Wyoming), Division 9: Pacific (Alaska, California, Hawaii, Oregon, and Washington). FIGURE 2. View largeDownload slide Neurosurgeons’ mean modified composite score ratings and their 95% confidence intervals (bars), stratified by location as defined by US Census Bureau divisions. US Census Bureau division are defined as follows: Division 1: New England (Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, and Vermont), Division 2: Mid-Atlantic (New Jersey, New York, and Pennsylvania), Division 3: East North Central (Illinois, Indiana, Michigan, Ohio, and Wisconsin), Division 4: West North Central (Iowa, Kansas, Minnesota, Missouri, Nebraska, North Dakota, and South Dakota), Division 5: South Atlantic (Delaware, Florida, Georgia, Maryland, North Carolina, South Carolina, Virginia, District of Columbia, and West Virginia), Division 6: East South Central (Alabama, Kentucky, Mississippi, and Tennessee), Division 7: West South Central (Arkansas, Louisiana, Oklahoma, and Texas), Division 8: Mountain (Arizona, Colorado, Idaho, Montana, Nevada, New Mexico, Utah, and Wyoming), Division 9: Pacific (Alaska, California, Hawaii, Oregon, and Washington). Analysis by Training-related Factors Providers who were fellowship-trained had better median ratings on Healthgrades.com (P = .047) and Vitals.com (P = .001), but not on RateMDs.com (P = .176; Figure 3). Providers who graduated from top 25 medical schools had higher median ratings on Healthgrades.com (P = .013), but not on Vitals.com (P = .421) or RateMDs.com (P = .194; Figure 4). Ratings for providers who trained at top 25 residencies were no different on any of the 3 websites (Healthgrades.comP = .652, Vitals.comP = .461, RateMDs.comP = .950). FIGURE 3. View largeDownload slide A comparison of neurosurgeons’ median ratings on 3 different only rating websites, stratified by whether the providers completed a postresidency fellowship (filled circles) vs not (open circles). For Healthgrades.com and Vitals.com, P < .05; for RateMDs.com, P > .05. FIGURE 3. View largeDownload slide A comparison of neurosurgeons’ median ratings on 3 different only rating websites, stratified by whether the providers completed a postresidency fellowship (filled circles) vs not (open circles). For Healthgrades.com and Vitals.com, P < .05; for RateMDs.com, P > .05. FIGURE 4. View largeDownload slide A comparison of neurosurgeons’ median ratings on 3 different only rating websites, stratified by whether the providers graduated from a medical school rank among the top 20 according to US News and World Report (filled circles) vs all other medical schools (open circles). FIGURE 4. View largeDownload slide A comparison of neurosurgeons’ median ratings on 3 different only rating websites, stratified by whether the providers graduated from a medical school rank among the top 20 according to US News and World Report (filled circles) vs all other medical schools (open circles). Factors Associated with Excellent Physician Ratings on Multivariable Analysis Healthgrades.com Data On multivariable analysis, having a Healthgrades.com rating in the top 25th percentile was positively associated with being located in a major city (odds ratio [OR] 2.00 [1.58, 2.54], P < .001), and having a practice located in New England (OR 2.03 [1.24, 3.31], P = .005), or the Mid-Atlantic (OR 1.71 [1.27, 2.31], P < .001), and was negatively associated with the number of reviews that were written (OR 0.94 [0.93, 0.95], P < .001). Having a Healthgrades.com rating in the bottom 25th percentile was positively associated with East South Central (OR 1.64 [1.12, 2.41], P = .011), and negatively associated with Mid-Atlantic (OR 0.63 [0.45, 0.89], P = .008), and being located in a major city (OR 0.69 [0.54, 0.89], P = .003). RateMDs.com Data Having a RateMDs.com rating in the top 25th percentile was positively associated with a practice located in the Pacific region (OR 1.54 [1.00, 2.37], P = .047), and was negatively associated with the number of reviews that were written (OR 0.86 [0.82, 0.90], P < .001), and the number of years since medical school graduation (OR 0.99 [0.97, 1.00], P = .029). Having a RateMDs.com rating in the bottom 25th percentile was negatively associated with the number of reviews that were written (OR 0.97 [0.95, 1.00], P = .019), and positively associated with years since graduation from medical school (OR 1.02 [1.01, 1.04], P = .001) and having a practice located in the East North Central region (OR 1.51 [1.04, 2.19], P = .030). Vitals.com Data Having a Vitals.com rating in the top 25th percentile was positively associated with having been fellowship-trained (OR 1.78 [1.29, 2.46], P < .001), graduating from a top 25 medical school (OR 1.65 [1.12, 2.45], P = .012), and graduating from a top 25 residency (OR 1.67 [1.10, 2.53], P = .015), and was negatively associated with the number of years since graduation from medical school (OR 0.96 [0.95, 0.98], P < .001) and the number of reviews that were written (OR 0.99 [0.98, 1.00], P = .007). No variable was independently associated with having a Vitals.com rating in the bottom 25th percentile, but fellowship training showed a trend toward a negative association (OR 0.74 [0.53, 1.04], P = .079) Analysis of the Number of Reviews and Variance of Reviews As all 3 online review websites used a numerical rating system from 1 to 5, we combined data across all 3 sites by taking a weighted average of their scores from each website, creating a composite score. The median number of reviews per provider on using our composite score was 21, IQR [8, 41]. The median number of reviews for providers ranked in the top or bottom 25th percentile on the composite score was lower than for all other providers (15 [5, 34] vs 27 [13, 47], P < .0001). We calculated the variance between providers’ scores between the 3 different websites investigated. The median variance between providers’ scores on the 3 different rating websites was 0.20 (0.07, 0.50). On multivariable regression, having high variance (>75th percentile) was negatively associated with the total number of reviews the provider received (OR 0.99 [0.99, 1.00], P = .001), and was positively associated with having fewer than 5 reviews specifically (OR 2.43 [1.00, 5.90], P = .05). As above, on multivariable regression, for all 3 online rating systems, the number of reviews written for a given provider was negatively associated with that provider being in the top 25th percentile or bottom 25th percentile (Healthgrades.com OR 0.94, P < .001; RateMD.com OR 0.86, P < .001; Vitals.com OR 0.99, P = .007). This negative association was also seen for the composite score we created from the providers’ other ratings (OR 0.99 [0.99, 0.99], P < .001). We therefore eliminated all providers who were below the 25th percentile with respect to the number of reviews that had been written about them (<8), resulting in a modified composite score distribution (Skewness –0.85, P < 0.0001; Kurtosis 3.98, P < .0001; Figure 1). Among the remaining providers, the median composite score was 4.11, with an IQR (3.69, 4.44). Single Variable and Multivariable Analysis of the Modified Composite Score Single Variable Analysis Providers had higher median-modified composite scores if they were fellowship-trained (P = .0001) or graduated from a top 25 medical school (P = .012), but not if they graduated from a top 25 residency (P = .106). Providers located in major cities had significantly higher median composite scores (P = .003, Table 3). TABLE 3. Single-Variable and Multivariable Analysis of the Modified Composite Score   P  OR  95% CI  Single-variable analysis     Top 25 Medical School  0.012         Top 25 Residency  0.106         Fellowship-Trained  0.0001         Practice in Major City  0.003        Multivariable Analysis     Top Quartile      Practice in Major City  0.001  1.63  1.23  2.17    Practice in New England  0.023  1.9  1.09  3.29    Practice in Mid-Atlantic  0.044  1.44  1.01  2.06    Practice in West North Central  0.082  0.55  0.27  1.08    Fellowship Training  <0.001  1.93  1.47  2.54    Years Since MD Graduation  <0.001  0.97  0.96  0.98   Bottom Quartile      Practice in Major City  0.062  0.78  0.59  1.01    Practice in East South Central  0.011  1.69  1.13  2.54    Practice in Mountain  0.007  1.79  1.17  2.74    P  OR  95% CI  Single-variable analysis     Top 25 Medical School  0.012         Top 25 Residency  0.106         Fellowship-Trained  0.0001         Practice in Major City  0.003        Multivariable Analysis     Top Quartile      Practice in Major City  0.001  1.63  1.23  2.17    Practice in New England  0.023  1.9  1.09  3.29    Practice in Mid-Atlantic  0.044  1.44  1.01  2.06    Practice in West North Central  0.082  0.55  0.27  1.08    Fellowship Training  <0.001  1.93  1.47  2.54    Years Since MD Graduation  <0.001  0.97  0.96  0.98   Bottom Quartile      Practice in Major City  0.062  0.78  0.59  1.01    Practice in East South Central  0.011  1.69  1.13  2.54    Practice in Mountain  0.007  1.79  1.17  2.74  View Large TABLE 3. Single-Variable and Multivariable Analysis of the Modified Composite Score   P  OR  95% CI  Single-variable analysis     Top 25 Medical School  0.012         Top 25 Residency  0.106         Fellowship-Trained  0.0001         Practice in Major City  0.003        Multivariable Analysis     Top Quartile      Practice in Major City  0.001  1.63  1.23  2.17    Practice in New England  0.023  1.9  1.09  3.29    Practice in Mid-Atlantic  0.044  1.44  1.01  2.06    Practice in West North Central  0.082  0.55  0.27  1.08    Fellowship Training  <0.001  1.93  1.47  2.54    Years Since MD Graduation  <0.001  0.97  0.96  0.98   Bottom Quartile      Practice in Major City  0.062  0.78  0.59  1.01    Practice in East South Central  0.011  1.69  1.13  2.54    Practice in Mountain  0.007  1.79  1.17  2.74    P  OR  95% CI  Single-variable analysis     Top 25 Medical School  0.012         Top 25 Residency  0.106         Fellowship-Trained  0.0001         Practice in Major City  0.003        Multivariable Analysis     Top Quartile      Practice in Major City  0.001  1.63  1.23  2.17    Practice in New England  0.023  1.9  1.09  3.29    Practice in Mid-Atlantic  0.044  1.44  1.01  2.06    Practice in West North Central  0.082  0.55  0.27  1.08    Fellowship Training  <0.001  1.93  1.47  2.54    Years Since MD Graduation  <0.001  0.97  0.96  0.98   Bottom Quartile      Practice in Major City  0.062  0.78  0.59  1.01    Practice in East South Central  0.011  1.69  1.13  2.54    Practice in Mountain  0.007  1.79  1.17  2.74  View Large Multivariable Regression Having a modified composite score in the top 25th percentile was negatively associated with increasing time since graduation from medical school (OR 0.97 [0.96, 0.98], P < .001), and positively associated with fellowship training (OR 1.93 [1.47, 2.54], P < .001), practice located in a major city (OR 1.63 [1.23, 2.17], P = .001), New England (OR 1.90 [1.09, 3.29], P = .023), or the Mid-Atlantic (OR 1.44 [1.01, 2.06], P = .044), and there was a trend toward significance for practices located in the West North Central region (OR 0.55 [0.27, 1.08], P = .082). Having a composite score in the bottom 25th percentile was positively associated with having a practice located in the East South Central region (OR 1.69 [1.13, 2.54], P = .011), Mountain (OR 1.79 [1.17, 2.74], P = .007), and there was a trend toward a negative association with being in a major city (OR 0.78 [0.59, 1.01], P = .062; Table 4) TABLE 4. Factors Associated With Excellent Physician Ratings on Multivariable Analysis   P  OR  95% CI  Healthgrades.com     Top Quartile      Practice in Major City  <.001  2  1.58  2.54    Practice in New England  .005  2.03  1.24  3.31    Practice in Mid-Atlantic  <.001  1.71  1.27  2.31    Number of Reviews Written  <.001  0.94  0.93  0.95   Bottom Quartile      Practice in East South Central  .011  1.64  1.12  2.41    Practice in Mid-Atlantic  .008  0.63  0.45  0.89    Practice in Major City  .003  0.69  0.54  0.89  RateMDs.com     Top Quartile      Practice in Pacific  .047  1.54  1  2.37    Number of Reviews Written  <.001  0.86  0.82  0.9    Years Since MD Graduation  .029  0.98  0.97  1   Bottom Quartile      Practice in East North Central  .03  1.51  1.04  2.19    Years Since MD Graduation  .001  1.02  1.01  1.04    Number of Reviews Written  .019  0.97  0.95  1  Vitals.com     Top Quartile      Top 25 Medical School  .012  1.65  1.12  2.45    Top 25 Residency  .015  1.67  1.1  2.53    Fellowship-Trained  <.001  1.78  1.29  2.46    Number of Reviews Written  .007  0.99  0.98  1    Years Since MD Graduation  <.001  0.96  0.95  0.98   Bottom Quartile      Fellowship-Trained  .079  0.74  0.53  1.04    P  OR  95% CI  Healthgrades.com     Top Quartile      Practice in Major City  <.001  2  1.58  2.54    Practice in New England  .005  2.03  1.24  3.31    Practice in Mid-Atlantic  <.001  1.71  1.27  2.31    Number of Reviews Written  <.001  0.94  0.93  0.95   Bottom Quartile      Practice in East South Central  .011  1.64  1.12  2.41    Practice in Mid-Atlantic  .008  0.63  0.45  0.89    Practice in Major City  .003  0.69  0.54  0.89  RateMDs.com     Top Quartile      Practice in Pacific  .047  1.54  1  2.37    Number of Reviews Written  <.001  0.86  0.82  0.9    Years Since MD Graduation  .029  0.98  0.97  1   Bottom Quartile      Practice in East North Central  .03  1.51  1.04  2.19    Years Since MD Graduation  .001  1.02  1.01  1.04    Number of Reviews Written  .019  0.97  0.95  1  Vitals.com     Top Quartile      Top 25 Medical School  .012  1.65  1.12  2.45    Top 25 Residency  .015  1.67  1.1  2.53    Fellowship-Trained  <.001  1.78  1.29  2.46    Number of Reviews Written  .007  0.99  0.98  1    Years Since MD Graduation  <.001  0.96  0.95  0.98   Bottom Quartile      Fellowship-Trained  .079  0.74  0.53  1.04  View Large TABLE 4. Factors Associated With Excellent Physician Ratings on Multivariable Analysis   P  OR  95% CI  Healthgrades.com     Top Quartile      Practice in Major City  <.001  2  1.58  2.54    Practice in New England  .005  2.03  1.24  3.31    Practice in Mid-Atlantic  <.001  1.71  1.27  2.31    Number of Reviews Written  <.001  0.94  0.93  0.95   Bottom Quartile      Practice in East South Central  .011  1.64  1.12  2.41    Practice in Mid-Atlantic  .008  0.63  0.45  0.89    Practice in Major City  .003  0.69  0.54  0.89  RateMDs.com     Top Quartile      Practice in Pacific  .047  1.54  1  2.37    Number of Reviews Written  <.001  0.86  0.82  0.9    Years Since MD Graduation  .029  0.98  0.97  1   Bottom Quartile      Practice in East North Central  .03  1.51  1.04  2.19    Years Since MD Graduation  .001  1.02  1.01  1.04    Number of Reviews Written  .019  0.97  0.95  1  Vitals.com     Top Quartile      Top 25 Medical School  .012  1.65  1.12  2.45    Top 25 Residency  .015  1.67  1.1  2.53    Fellowship-Trained  <.001  1.78  1.29  2.46    Number of Reviews Written  .007  0.99  0.98  1    Years Since MD Graduation  <.001  0.96  0.95  0.98   Bottom Quartile      Fellowship-Trained  .079  0.74  0.53  1.04    P  OR  95% CI  Healthgrades.com     Top Quartile      Practice in Major City  <.001  2  1.58  2.54    Practice in New England  .005  2.03  1.24  3.31    Practice in Mid-Atlantic  <.001  1.71  1.27  2.31    Number of Reviews Written  <.001  0.94  0.93  0.95   Bottom Quartile      Practice in East South Central  .011  1.64  1.12  2.41    Practice in Mid-Atlantic  .008  0.63  0.45  0.89    Practice in Major City  .003  0.69  0.54  0.89  RateMDs.com     Top Quartile      Practice in Pacific  .047  1.54  1  2.37    Number of Reviews Written  <.001  0.86  0.82  0.9    Years Since MD Graduation  .029  0.98  0.97  1   Bottom Quartile      Practice in East North Central  .03  1.51  1.04  2.19    Years Since MD Graduation  .001  1.02  1.01  1.04    Number of Reviews Written  .019  0.97  0.95  1  Vitals.com     Top Quartile      Top 25 Medical School  .012  1.65  1.12  2.45    Top 25 Residency  .015  1.67  1.1  2.53    Fellowship-Trained  <.001  1.78  1.29  2.46    Number of Reviews Written  .007  0.99  0.98  1    Years Since MD Graduation  <.001  0.96  0.95  0.98   Bottom Quartile      Fellowship-Trained  .079  0.74  0.53  1.04  View Large DISCUSSION PRWs are quickly becoming an important resource for patients, physicians, and policy makers alike. Studies have suggested that about a third of patients avoid seeing a physician with poor online ratings, yet little is understood about how to interpret surgeons’ PRW scores.1,6 Moreover, controversy exist about whether scores correlate with traditional clinical outcome metrics.3,8,19,20 It is therefore increasingly relevant to understand and contextualize the data on PRWs, in order to better understand the information that they provide. Demographic Effects Gender had no effect on PRW ratings in our analysis. This contrast with some existing literature shows that women are more likely to be rated positive reviews, though there is controversy about whether this association remains true universally.21-23 We found time since medical school graduation to be negatively associated with receiving ratings in the top 25th percentile on multivariate analysis for Vitals.com and RateMDs.com (OR 0.96, P < .001; OR 0.98, P = .029 respectively). This finding further remained significant on composite score multivariate analysis (OR 0.97, P < .001). These findings are consistent with previous literature documenting that increasing age leads to worse physician ratings on PRWs.9 Why older physicians receive lower ratings is not entirely clear. This could suggest that technological savviness and online presence plays a large role in PRW ratings, as older physicians are generally less apt to embrace new technological paradigm shifts. Of note, literature has reported worsening clinical outcomes correlating to increasing physician age, which could further explain the observed relationship.24 However, as no outcomes data were available, this hypothesis could not be explored. Data Distribution Each of the 3 most used PRWs had largely a positive skew with median ratings higher than 4 on a 5-point scale, which should be considered when interpreting surgeons’ ratings. While a rating of 4.0 may seem to suggest an above average performance on a 5-point scale, it is a comparatively average or slightly below average rating. These findings are consistent with previous literature citing that a majority of patients tend to give physicians positive ratings on online PRWs.20,25 Indeed, studies have noted that anywhere from 66% to 88% of reviews on PRWs are positive, perhaps contributing to the strong positive skew evidenced in the dataset.20,25 Our findings show that ratings with less reviews are less likely to be reliable and more likely to be at either a negative or positive extreme. Specifically, we found that surgeons with ratings in the top or bottom quartile had significantly fewer reviews than surgeons in the middle 50th percentile. Similarly, the number of reviews written for a provider was negatively correlated with positive ratings on multivariate analysis for all 3 PRWs. This may reflect the aforementioned propensity for patients to rate providers highly, likely combined with regression to the mean as more reviews are written over time. Moreover, on multivariable regression, having a larger variance of a surgeons’ ratings between PRWs was associated with having fewer reviews. Having 5 or fewer ratings on a single site was also independently associated with high interwebsite variance. As such, PRW profiles for surgeons with a small number of ratings may not accurately reflect true patient satisfaction and may require more data in order to come to more precise conclusions. To our knowledge, this is the first analysis to observe this effect on PRWs, and must be considered when interpreting a surgeon's PRW scores. Geographic Variation Our study further found that geographic location correlated with physician ratings. Providers located in major cities were found to have significantly higher composite scores on single-variable (P = .0025) and multivariable analysis (P = .001). As increasing trends in medicine favor “centers for excellence” for patient care, it has been hypothesized that many patients are opting to seek care in more urban settings, as many of such centers are often located in larger cities.26,27 There is controversy about whether such centers provide better care,28,29 but they are publicly perceived to do so.26-29 The higher PRW scores of surgeons at urban centers may reflect better care, the perception of better care, or a combination of both. Furthermore, our study found significant regional variation in ratings, as practices based out of New England, Mid-Atlantic, and West North Central regions were more likely to have ratings in the top 25 percentile, and practices in the East South Central and Mountain regions were more likely to have ratings in the bottom 25 percentile. Regional variation in PRWs has been described in the past, although to our knowledge, this is the first analysis to observe this effect on PRWs among neurosurgeons. Effects of Training On multivariable analysis, having ratings in the top quartile was positively associated with fellowship training and graduating from a top 25 medical school (Vitals.com OR 1.78, P < .001; OR 1.65, P = .012 respectively; OR 1.93, P < .001 for fellowship training on analysis of composite score). This was also reflected on single-variable analysis. It is reasonable for fellowship training to lead to better care or to the perception that a surgeon is providing better care, as has been described in other surgical disciplines.30,31 Of note, graduating from a top 20 residency program only showed significant association with higher scores on multivariable regression for one of 3 PRWs and not for composite score (Vitals.com, OR 1.67, P = .015). Surgical skills are best learned during residency and continue to be honed in practice. It is therefore unclear why graduating from a top 25 medical school may be associated with higher ratings, whereas graduating from a top 20 residency is not. This may be an example of the so-called halo effect, a cognitive bias wherein a good impression caused by 1 attribute leads to a good impression of other attributes that may or may not be related.32 Such effects have been described elsewhere in the surgical literature, but not for individual surgeons to our knowledge.33-35 The brand recognition of an excellent university is likely greater than the brand recognition of an excellent residency program, and the general public probably has more familiarity with university rankings than of residency rankings. As such, the halo effect may translate into better ratings for providers who attended highly ranked medical schools if patients perceive them to be better surgeons a priori. Limitations Our study is not without limitations. Due to the fluidity of online ratings, ratings one day may differ from ratings then next, making our analysis only relevant for a snapshot in time. Providers often change practice settings, which may confound our geographic analyses. We only analyzed data from the top 3 PRWs were included in the analysis, which may not be reflective of other PRWs. There is no verification process on such websites for board certification, which was one of our inclusion criteria for the providers we analyzed. There is no method in place on many of these websites to verify whether physicians listed are currently in practice, so providers who have subsequently retired may skew our data. We could not readily differentiate providers based on the type of surgeries they offered, eg, cranial vs spine, which may lead to comparisons between dissimilar surgical practices. We have no clinical outcomes data to analyze from these providers, and cannot therefore determine what outcomes driver surgeons’ ratings, if any. Moreover, we cannot determine whether factors outside of a surgeon's control, such as nursing staff or hospital facilities, impact a patient's impression of their care and their subsequent rating. Lastly, it should be noted that current medical school and residency rating systems are not perfect and similar to PRWs are only a snapshot in time, likely not reflecting the dynamic state of training over years past. Despite these limitations, our study is the first in the neurosurgical literature to examine PRWs, and to provide a context with which to interpret surgeons’ scores on PRWs, which is a topic of growing interest as patient satisfaction metrics continue to become more important. CONCLUSION Online ratings for neurosurgeons must be evaluated in context. Median ratings are approximately 4 on a 5-point scale, but vary between websites, and scores are not normally distributed. Ratings are more varied and more extreme when providers have fewer total ratings. Median scores also vary among regions and practice settings. Surgeons who graduated from highly ranked medical schools, who graduated from medical school more recently, and who are fellowship-trained tend to have higher ratings. Disclosure The authors have no personal, financial, or institutional interest in any of the drugs, materials, or devices described in this article. Notes Portions of this work were submitted as an abstract to the American Association of Neurosurgeons’ 2018 Annual Scientific Meeting in New Orleans, Louisiana, USA. REFERENCES 1. Hanauer DA, Zheng K, Singer DC, Gebremariam A, Davis MM. Public awareness, perception, and use of online physician rating sites. JAMA . 2014; 311( 7): 734- 735. Google Scholar CrossRef Search ADS PubMed  2. Huff DJ. Online physician rating websites. J Med Assoc Ga . 2013; 102( 2): 30, 34. 3. Jack RA 2nd, Burn MB, McCulloch PC, Liberman SR, Varner KE, Harris JD. Does experience matter? A meta-analysis of physician rating websites of orthopaedic surgeons. Musculoskelet Surg . 2017. 4. Hawkes N. Plethora of websites rating healthcare is unhelpful to public, says think tank. BMJ . 2015; 351: h5462. Google Scholar CrossRef Search ADS PubMed  5. Ricciardi BF, Waddell BS, Nodzo SR et al.  . Provider-initiated patient satisfaction reporting yields improved physician ratings relative to online rating websites. Orthopedics . 2017; 40( 5): 304- 310. Google Scholar CrossRef Search ADS PubMed  6. Galizzi MM, Miraldo M, Stavropoulou C et al.   Who is more likely to use doctor-rating websites, and why? A cross-sectional study in London. BMJ Open . 2012; 2( 6). 7. Adelhardt T, Emmert M, Sander U, Wambach V, Lindenthal J. Can patients rely on results of physician rating websites when selecting a physician? - A cross-sectional study assessing the association between online ratings and structural and quality of care measures from two German physician rating websites. Value Health . 2015; 18( 7): A545. Google Scholar CrossRef Search ADS PubMed  8. Emmert M, Adelhardt T, Sander U, Wambach V, Lindenthal J. A cross-sectional study assessing the association between online ratings and structural and quality of care measures: results from two German physician rating websites. BMC Health Serv Res . 2015; 15( 1): 414. Google Scholar CrossRef Search ADS PubMed  9. Emmert M, Meier F. An analysis of online evaluations on a physician rating website: evidence from a German public reporting instrument. J Med Internet Res . 2013; 15( 8): e157. Google Scholar CrossRef Search ADS PubMed  10. Ma L, Kaye AD, Bean M, Vo N, Ruan X. A five-star doctor? Online rating of physicians by patients in an internet driven world. Pain Physician . 2015; 18( 1): E15- E18. Google Scholar PubMed  11. Kirkpatrick W, Abboudi J, Kim N et al.  . An assessment of online reviews of hand surgeons. Arch Bone Jt Surg . 2017; 5( 3): 139- 144. Google Scholar PubMed  12. Lagu T, Metayer K, Moran M et al.   Website characteristics and physician reviews on commercial physician-rating websites. JAMA . 2017; 317( 7): 766- 768. Google Scholar CrossRef Search ADS PubMed  13. AlRuthia YS, Hong SH, Graff C, Kocak M, Solomon D, Nolly R. Exploring the factors that influence medication rating Web sites value to older adults: a cross-sectional study. Geriatric Nursing . 2016; 37( 1): 36- 43. Google Scholar CrossRef Search ADS PubMed  14. Burkle CM, Keegan MT. Popularity of internet physician rating sites and their apparent influence on patients’ choices of physicians. BMC Health Serv Res . 2015; 15( 1): 416. Google Scholar CrossRef Search ADS PubMed  15. Ramkumar PN, Navarro SM, Chughtai M, La T Jr., Fisch E, Mont MA. The patient experience: an analysis of orthopedic surgeon quality on physician-rating sites. J Arthroplasty . 2017; 32( 9): 2905- 2910. Google Scholar CrossRef Search ADS PubMed  16. Bakhsh W, Mesfin A. Online ratings of orthopedic surgeons: analysis of 2185 reviews. Am J Orthop . 2014; 43( 8): 359- 363. Google Scholar PubMed  17. Report UNW. Top Medical Schools- Research . 2017. Available at: https://www.usnews.com/best-graduate-schools/top-medical-schools/research-rankings. Accessed September 22, 2017. 18. Doximity. Neurological Surgery- Reputation . 2017. Available at: https://residency.doximity.com/programs?residency_specialty_id=46&sort_by=reputation. Accessed September 22, 2017. 19. Okike K, Peter-Bibb TK, Xie KC, Okike ON. Association between physician online rating and quality of care. J Med Internet Res . 2016; 18( 12): e324. Google Scholar CrossRef Search ADS PubMed  20. Lagu T, Hannon NS, Rothberg MB, Lindenauer PK. Patients’ evaluations of health care providers in the era of social networking: an analysis of physician-rating websites. J Gen Intern Med . 2010; 25( 9): 942- 946. Google Scholar CrossRef Search ADS PubMed  21. Ellimoottil C, Hart A, Greco K, Quek ML, Farooq A. Online reviews of 500 urologists. J Urol . 2013; 189( 6): 2269- 2273. Google Scholar CrossRef Search ADS PubMed  22. Kadry B, Santoro E, Zhang Q, Emmert M, Meier F. An analysis of online evaluations on a physician rating website: evidence from a german public reporting instrument. J Med Internet Res . 2013; 15( 8): e157. Google Scholar CrossRef Search ADS PubMed  23. Gao G, Greaves F, Emmert M, Sander U, Pisch F. Eight questions about physician-rating websites: a systematic review. J Med Internet Res . 2013; 15( 2): e24. Google Scholar CrossRef Search ADS PubMed  24. Tsugawa Y, Newhouse JP, Zaslavsky AM, Blumenthal DM, Jena AB. Physician age and outcomes in elderly patients in hospital in the US: observational study. BMJ . 2017; 357: j1797. Google Scholar CrossRef Search ADS PubMed  25. Kadry B, Chu LF, Kadry B, Gammas D, Macario A. Analysis of 4999 online physician ratings indicates that most patients give physicians a favorable rating. J Med Internet Res . 2011; 13( 4): e95. Google Scholar CrossRef Search ADS PubMed  26. Elrod JK, Fortenberry JL Jr. Centers of excellence in healthcare institutions: what they are and how to assemble them. BMC Health Serv Res . 2017; 17( S1): 425. Google Scholar CrossRef Search ADS PubMed  27. Sugerman DT. Centers of excellence. JAMA . 2013; 310( 9): 994. Google Scholar CrossRef Search ADS PubMed  28. Mehrotra A, Sloss EM, Hussey PS, Adams JL, Lovejoy S, SooHoo NF. Evaluation of a center of excellence program for spine surgery. Med Care . 2013; 51( 8): 748- 757. Google Scholar CrossRef Search ADS PubMed  29. Livingston EH. Bariatric surgery outcomes at designated centers of excellence vs nondesignated programs. Arch Surg . 2009; 144( 4): 319- 325; discussion 325. Google Scholar CrossRef Search ADS PubMed  30. Oliak D, Owens M, Schmidt HJ. Impact of fellowship training on the learning curve for laparoscopic gastric bypass. Obes Surg . 2004; 14( 2): 197- 200; 0960-8923. Google Scholar CrossRef Search ADS PubMed  31. Agrawal S. Impact of bariatric fellowship training on perioperative outcomes for laparoscopic Roux-en-Y gastric bypass in the first year as consultant surgeon. Obes Surg . 2011; 21( 12): 1817- 1821; 0960-8923. Google Scholar CrossRef Search ADS PubMed  32. Nisbett RE, Wilson TD. The halo effect: Evidence for unconscious alteration of judgments. J Person Social Psychol . 1977; 35( 4): 250- 256; 1939-1315. Google Scholar CrossRef Search ADS   33. Utter GH, Maier RV, Rivara FP, Nathens AB. Outcomes after ruptured abdominal aortic aneurysms: the “halo effect” of trauma center designation. J Am Coll Surg . 2006; 203( 4): 498- 505; 1072-7515. Google Scholar CrossRef Search ADS PubMed  34. Nagarajan N, Selvarajah S, Gani F et al.   “Halo effect” in trauma centers: does it extend to emergent colectomy? J Surg Res . 2016; 203( 1): 231- 237; 0022-4804. Google Scholar CrossRef Search ADS PubMed  35. Brown EG, Anderson JE, Burgess D, Bold RJ. Examining the “Halo Effect” of surgical care within health systems. JAMA Surg . 2016; 151( 10): 983- 984; 2168-6254. Google Scholar CrossRef Search ADS PubMed  COMMENTS This study shines the light of statistical analysis into the murky world of online physician rankings. This study is a first for neurosurgery, and the findings are consistent with other surgical specialties. While the results are not surprising, the demonstration of skew and bias in online rating platforms deserves notice. Based on this study, readers should consider the sources for perceptions of quality in neurosurgery. Hopefully this will direct further research into establishing concrete measures of quality and outcome within the specialty. Thomas Mattingly Richmond, Virginia Although this report is very limited in shedding any light on “physician rating websites”, it is an attempt to understand the trends in these now commonly used online instruments. There are no real surprises in the results in that surgeons who are younger, fellowship trained, and from better medical schools have better ratings. Urban practicing surgeons also have higher ratings, which fits the perception, especially in a highly specialized field such as neurosurgery, that surgeons in large medical centers are better. I think publishing this will spur others to look at this data in in a critical manner and may uncover inherent weaknesses or value in these systems. William F. Chandler Ann Arbor, Michigan Physician ranking is a currently unregulated and non-scientific method used by a biased population, patients. Current shifts in reimbursement are using patient satisfaction scores without understanding what, if any, usable objective data is present. This manuscript underscores this point and provides a framework to understand the bias involved in patient satisfaction scores. Jeffrey Steven Raskin Indianapolis, Indiana Research into various dimensions of quality in medicine has been undertaken for several decades.1 In 1998 the Institute of Medicine formed the Committee on Quality of Health Care in America. It set out to develop strategies that would improve the quality of health care by 2008.2, 3 Patient centered care rose from merely a high priority as measured by patient satisfaction surveys to a central part of health care quality assessment. Results of the patient satisfaction Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey comprise 30% of the total performance score of Hospital Value Based Purchasing, is tied to 2% of Medicare reimbursement.4 Not surprisingly, patient satisfaction with individual physicians and in the outpatient setting also came to command attention and substantial work has been done on the relationship between quality of outcomes and patient satisfaction.5, 6 The quality of physician-patient and nurse-patient interactions also clearly matter.7 With that said, patient satisfaction ratings (PSRs) are complicated instruments.8 The web-based systems described in this article are no less complicated and possibly more vulnerable to error because of the way their ratings are bestowed and disseminated. They are also a feature of the movement towards consumer-facing health care and media-based open information exchange. Like many rating systems, PSRs express subjective perceptions of quality whose vernacular authority derives from the ways they are used. Whether they matter because they exist or they exist because they matter cannot be determined. Irrespective of extent of validation against objective quality measures or any other rating standards, web-based PSRs will increasingly and unavoidably exert market influence. The question is how to respond. The practice of surgery and medicine is classified as a service. Services are characterized not only by the fact that they deal with the delivery of intangibles, unlike product-focused operations or industries that provide things that are tangible, but by the fact that the quality of the service cannot be fully assessed until it has been experienced. Insofar as all experiences are ultimately judged personally and subjectively, service industries rely heavily on client feedback for measures of quality and success. Reviews reflect the experiences of others. Review-based ratings, including the PSR, survey and aggregate the experiences of others so that potential clients might have a reasonable sense of what to expect, and a peer-anchored basis on which to decide whether to avail themselves of the services offered. The validity, usefulness, and relevance of any survey is limited by design specifications sampling considerations (including bias, sample size, and homogeneity) and by the range of questions asked and answered. In PSRs, for example, the patients most likely to respond are the most satisfied and the least satisfied, and this may constitute and important bias. Web-based PSRs are unlikely to undergo the testing and validation required for the HCAHPS (the Hospital Consumer Assessment of Healthcare Providers and Systems), for example, and it may be difficult to know how accurate and how representative they are. Nevertheless, for purposes of the casual guidance sought by most patients, they are deemed relevant enough because of inferred confidence in social media surveys. Relevance, validity, and meaning are attributed to consistency of the reported experience even though the questions that are asked may be biased and the questionnaire itself may be flawed. Surgeons have long responded to patient satisfaction surveys by saying that they are focused on saving lives and not on popularity contests. Their concerns follow patient outcomes and other traditional professional standards validated by the profession, even though they may not be fully appreciated either by the patient or by other relevant clients including payers. They accept the existence of disparities of value. How should surgeons respond in the era PSR and similar instruments of satisfaction? When McDonald's (a mixed model of service and food product) first opened in Moscow's Red Square, it had no competition. Service was poor and the lines were unimaginably long. Nevertheless, there were few complaints because it was Western. McDonald's was perceived to different from, and better than the alternatives, such as they may have been. And it probably was. McDonald's did not adhere to the same standard of service in Moscow as it did in St. Louis because it didn’t have to. As the level of consumer sophistication in Moscow rose, standards of service changed. Indeed, as a general rule, interest in and sensitivity to client feedback is heavily influenced by the competitive landscape and access to information about the competition. Health care is subject to the same rules. While health care has been historically construed as local phenomenon, geographic barriers are no longer as relevant as they once were. At one time, the characteristics of surgical services in Seattle were unlikely to be relevant in Key West, but in the era of consumer-facing health care, patients are increasingly encouraged to look around broadly before deciding where to have their care. They are led to integrate objective measures including price with subjective, client-based measures of satisfaction. For this and similar reasons, it is becoming increasing important to rethink standards of quality in neurosurgical practice. Patient outcomes are a given, but patient satisfaction must be targeted no less carefully. There is only 1 way to be sure that neurosurgical services successfully address patient needs, and that is to inquire, formally, thoughtfully, and consistently, and to respond appropriately. That is not necessarily the job of the surgeon, but in most cases, surgeons should oversee the process. There are good guidelines for team-based practices in this regard.9, 10, 11, 12 Whether web-based PSRs or other forms of patient satisfaction survey, current survey tools assess convenience, communication, promptness, access, and other traditional service-based quality measures. Patients are now encouraged explicitly to expect both good outcomes and a good experience. These are all factors that influence surgeon ratings. While consumer facing tools to assess quality and rate surgeons may be blunt, they cannot be avoided, and should not be ignored. It is important to understand them both in order to optimize patient care and to enhance patient satisfaction. T. Forcht Dagi Newton Centre, Massachusetts 1. Sitzia J, Wood N. Patient satisfaction: A review of issues and concepts. Social Science & Medicine . 1997: 45( i): 1829- 1843. Google Scholar CrossRef Search ADS   2. Kennedy GD, Tevis SE, Kent CK. Is There a Relationship Between Patient Satisfaction and Favorable Outcomes? Ann Surg . 2014; 260( 4): 592- 600. Google Scholar CrossRef Search ADS PubMed  3. Institute of Medicine (U.S.). Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century . 2001. Washington, D.C.: National Academy Press. 4. Petrullo K, Lamar S, Nwankwo-Otti O, Alexander-Mills K, Viola D. The Patient Satisfaction Survey: What does it mean to your bottom line? J Hosp Adm . 2012; 2: 1- 8. 5. Isaac T, Zaslavsky AM, Cleary PD et al. The relationship between patients' perception of care and measures of hospital quality and safety. Health Serv Res . 2010; 45: 1024- 1040. Google Scholar CrossRef Search ADS PubMed  6. Manary MP, Boulding W, Staelin R, Glickman SW. The Patient Experience and Health Outcomes. NEJM 2012 N Engl J Med  2013; 368: 201- 203. Google Scholar PubMed  7. Iannuzzi JC, Kahn SA, Zhang L, Gestring ML, Noyes K, Monson JR. Getting satisfaction: drivers of surgical Hospital Consumer Assessment of Health care Providers and Systems survey scores. J Surg Res . 2015 Jul; 197( 1): 155- 61. Epub 2015 Mar 24. 8. Bachman JW. The Problem with patient satisfaction scores. Fam Pract Manag . 2016; 23( 1): 23- 27. Google Scholar PubMed  9. Ogrinc GS, Headrick LA, Moore SM, Barton AJ, Dolansky MA, Madigosky WS. Fundamentals of Health Care Improvement: A Guide to Improving Your Patients' Care . 2nd ed. 2012. Oakbrook, IL: Joint Commission Resources and Institute for Healthcare Improvement. 10. Leon Guerrero CR, Anderson T, Zazulia AR. Education Research: Physician identification and patient satisfaction on an academic neurology inpatient service. Neurology . 2018. pii: 10.1212/WNL.0000000000004961. doi: 10.1212/WNL.0000000000004961. [Epub ahead of print]. 11. Fregene T, Wintle S, Venkat Raman V, Edmond H, Rizvi S. Making the experience of elective surgery better. BMJ Open Qual . 2017 Aug 9; 6( 2): e000079. doi: 10.1136/bmjoq-2017-000079. eCollection 2017. Google Scholar CrossRef Search ADS   12. de Vos MS, Hamming JF, Marang-van de Mheen PJ. The problem with using patient complaints for improvement. BMJ Qual Saf . 2018. pii: bmjqs-2017-0 07463. doi: 10.1136/bmjqs-2017-007463. [Epub ahead of print]. Copyright © 2018 by the Congress of Neurological Surgeons

Journal

NeurosurgeryOxford University Press

Published: Apr 3, 2018

References