Impact of the implementation of electronic health records on the quality of discharge summaries and on the coding of hospitalization episodes

Impact of the implementation of electronic health records on the quality of discharge summaries... Abstract Objective To determine whether the implementation and use of the electronic health records (EHR) modifies the quality, readability and/or the length of the discharge summaries (DS) and the average number of coded diagnosis and procedures per hospitalization episode. Design A pre–post-intervention descriptive study conducted between 2010 and 2014. Setting The ‘Hospital Universitario 12 de Octubre’ (H12O) of Madrid (Spain). A tertiary University Hospital of up to 1200 beds. Intervention Implementation and systematic use of the EHR. Main Outcome Measures The quality, length and readability of the DS and the number of diagnosis and procedures codes by raw and risk-adjusted data. Results A total of 200 DS were included in the present work. After the implementation of the EHR the DS had better quality per formal requirements, although were longer and harder to read (P < 0.001). The average number of coded diagnoses and procedures was increased, 9.48 in the PRE-INT and 10.77 in the POST-INT, and the difference was statistically significant (P < 0.001) in both raw and risk-adjusted data. Conclusions The implementation of EHR improves the formal quality of DS, although poor use of EHR functionalities might reduce its understandability. Having more clinical information immediately available due to EHR increases the number of diagnosis and procedure codes enhancing their utility for secondary uses. electronic health records, discharge summary, diagnosis and procedure codes, hospital care, quality improvement Introduction The discharge summary (DS) is an essential document for communication among healthcare professionals of different levels of care [1, 2]. It is considered indispensable for a proper continuing patient care and, in Spain, it is legally required and its contents defined. Until recently, DS were handwritten in a paper-based support; however, the progress of information technologies such as electronic health record (EHR) has enabled the automatic creation of the DS from clinical information registered during the hospitalization episode [3, 4] or any other information from the patient medical record. Moreover, the quality of the DS has implications other than the continuing patient care, since the Hospital Discharge Database, the main administrative database, also known as the Minimum Basic Data Set (MBDS) gathers the information from the coded diagnoses and procedures found in the DS. The MBDS is the largest set of administrative data of hospitalized patients and the major source of information of morbidity in Spain [5]. The systematic use of EHR in the clinical practice is spreading quickly [6, 7], therefore, as any other healthcare technology has to be evaluated and its impact in the processes of patient care and Healthcare Information Systems understood. Up to date, the comparison in the average number of coded diagnoses and procedures during hospitalization among different countries [8] as well as before and after the use of a new edition of the International Classification of Diseases [9] have been analyzed. As well, it has been shown that the utilization of EHR improves the quality of outpatient clinical notes [10] and of DS [11, 12]; however, to the best of our knowledge, this is the first study to show the impact of the EHR on coding of hospitalization episodes. Thus, the present study analyzed whether the implementation of EHR has modified the quality, readability and length of the DS as well as it has affected the number of coded diagnoses and procedures per hospitalization episode in a tertiary university Hospital of 1200 beds. Methods Study design and population This was a pre–post-intervention observational study of DS from the population of patients discharged out of the H12O since 1 January 2010 to 31 December 2014. The inclusion criterion was to have the hospitalization episode coded and registered in the MBDS. The exclusion criteria were not coded hospitalization episodes and patients grouped by all patient diagnosis-related groups (AP-DRG) with weight 0 (AP-DRG 469: principal diagnosis invalid as discharge diagnosis and AP-DRG 470: ungroupable) and indeterminate sex (Fig. 1). The data sources were the MBDS and the DS. The number of coded diagnoses and procedures were studied in all the episodes that met the inclusion criteria (N = 214 648). Figure 1 View largeDownload slide Inclusion and exclusion criteria used to determine the study population. Figure 1 View largeDownload slide Inclusion and exclusion criteria used to determine the study population. Intervention The intervention was the implementation and systematic use of EHR in all the H12O services and units. It was progressively deployed during 2012 and had full implementation with a systematic use by 2013. The years 2010 and 2011 were considered as pre-intervention period (PRE-INT); 2012 as transition period (TP); finally, 2013 and 2014 as post-intervention period (POST-INT). Insourced versus outsourced coding H12O has a specialized coding unit, but the task was partly outsourced from December of 2011 to August of 2013, to offset an unusual additional workload. The coexistence during this period of two distinct groups of coders was considered as a possible confounding factor and its impact on coding performance analyzed. Instrumentation, study variables and data source To evaluate the DS quality in PRE-INT and POST-INT, a sample was selected by simple random sampling. A review team consisting of three resident physicians was formed. Two senior physicians were consultants for the review team: the Coordinator of the Quality Unit and the Coordinator of the Patient Management Unit of the H12O. An initial training phase for the review team was considered and 50 DS of each period were analyzed and the reviewers performed an independent evaluation. However, the assessment was jointly validated, and the review criteria unified with the consensus of the two consultants. After the training phase, the sample size was calculated in 100 DS per period (200 in total) related to the populations of PRE-INT (42 182 DS) and POST-INT (43 928 DS) and considering the average number of diagnoses and procedure codes (NC) as well as their standard deviation (SD), obtained in the training phase, to detect a difference of means of 0.5 and a SD of 1.2, with a confidence level of 95% and a statistical power of 80%. The main variables analyzed from the selected sample were the quality, length and readability of the DS. To determine the quality, a method developed by the Hospital's Medical Charts Committee was used. It evaluates the compliance with the criteria established on the current legislation concerning to MBDS in the Spanish National Health System. It includes 13 DS items (admission and discharge dates; reason for admission; discharge status; health history; evolution; principal and secondary diagnoses surgical procedures; other procedures and therapeutic recommendations at discharge) scoring 0 points if the DS contained the minimum information required in each issue, 1 point if the information was incomplete or 2 points if it was missing, therefore, quality of DS=∑iVi and 0 ≤ QDS ≤ 26. Readability was quantified with the Spanish adaptation of the Flesch-Kincaid index, which measures formal readability between 0 and 100, considering the relationship between text difficulty, sentence length and words chosen [13]. Length was evaluated by a simple word count. Other variables studied were the NC according to the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM). Analyzing both, the codes registered in the MBDS previously not included in the DS and the diagnoses and procedures found in the DS but not registered in the MBDS. To measure other aspects of the EHR impact on coding, we considered the number of ICD-9-MC diagnosis and procedure codes as the outcome variable. The number of codes was directly obtained counting them in the H12O MBDS and to avoid confusion due to demographic and clinical characteristics of patients, raw data were risk-adjusted. The independent variables used for this adjustment obtained from the MBDS, are shown in Table 1. Table 1 Independent variables used to estimate the number of coded diagnoses and procedures for hospital episode based on the demographic and clinical characteristics of patients Variables Characteristics Number of Hospitalizations (per patient) 2010–14 Continuous External injury causes Dichotomous: 1 = Yes, 0 = No (Main diagnostic with ICD-9-MC) Reintervention during episode Dichotomous: 1 = Yes, 0 = No Major diagnostic category Categorical dichotomized. MDS 1 (reference) to MDS 25 (AP-DRG) Type of discharge service Categorical dichotomized. 1: Maternal-child (reference), 2: Critical care, 3: Medical, 4: Surgical; 5: Complex surgeries Weigh (AP-DRG) Continuous Length of stay Continuous: Discharge date–Hospitalization date (days) Age Continuous: Hospitalization date–date of birth (years) Sex Dichotomous: 1 = Women, 0 = Men Elective admissions Dichotomous: 1 = Yes, 0 = No Exitus Dichotomous: 1 = Yes, 0 = No Internal transfer Dichotomous: 1 = Yes, 0 = No Hospitalization in critical care Dichotomous: 1 = Yes, 0 = No Medical DRG Dichotomous: 1 = Yes, 0 = No Readmission in 30 days from discharge Dichotomous: 1 = Yes, 0 = No Charlson comorbidity index (CCI) Categorical dichotomized. 1: CCI = 0 (reference); 2: CCI = 1, 2; 3: CCI > 2 Variables Characteristics Number of Hospitalizations (per patient) 2010–14 Continuous External injury causes Dichotomous: 1 = Yes, 0 = No (Main diagnostic with ICD-9-MC) Reintervention during episode Dichotomous: 1 = Yes, 0 = No Major diagnostic category Categorical dichotomized. MDS 1 (reference) to MDS 25 (AP-DRG) Type of discharge service Categorical dichotomized. 1: Maternal-child (reference), 2: Critical care, 3: Medical, 4: Surgical; 5: Complex surgeries Weigh (AP-DRG) Continuous Length of stay Continuous: Discharge date–Hospitalization date (days) Age Continuous: Hospitalization date–date of birth (years) Sex Dichotomous: 1 = Women, 0 = Men Elective admissions Dichotomous: 1 = Yes, 0 = No Exitus Dichotomous: 1 = Yes, 0 = No Internal transfer Dichotomous: 1 = Yes, 0 = No Hospitalization in critical care Dichotomous: 1 = Yes, 0 = No Medical DRG Dichotomous: 1 = Yes, 0 = No Readmission in 30 days from discharge Dichotomous: 1 = Yes, 0 = No Charlson comorbidity index (CCI) Categorical dichotomized. 1: CCI = 0 (reference); 2: CCI = 1, 2; 3: CCI > 2 Table 1 Independent variables used to estimate the number of coded diagnoses and procedures for hospital episode based on the demographic and clinical characteristics of patients Variables Characteristics Number of Hospitalizations (per patient) 2010–14 Continuous External injury causes Dichotomous: 1 = Yes, 0 = No (Main diagnostic with ICD-9-MC) Reintervention during episode Dichotomous: 1 = Yes, 0 = No Major diagnostic category Categorical dichotomized. MDS 1 (reference) to MDS 25 (AP-DRG) Type of discharge service Categorical dichotomized. 1: Maternal-child (reference), 2: Critical care, 3: Medical, 4: Surgical; 5: Complex surgeries Weigh (AP-DRG) Continuous Length of stay Continuous: Discharge date–Hospitalization date (days) Age Continuous: Hospitalization date–date of birth (years) Sex Dichotomous: 1 = Women, 0 = Men Elective admissions Dichotomous: 1 = Yes, 0 = No Exitus Dichotomous: 1 = Yes, 0 = No Internal transfer Dichotomous: 1 = Yes, 0 = No Hospitalization in critical care Dichotomous: 1 = Yes, 0 = No Medical DRG Dichotomous: 1 = Yes, 0 = No Readmission in 30 days from discharge Dichotomous: 1 = Yes, 0 = No Charlson comorbidity index (CCI) Categorical dichotomized. 1: CCI = 0 (reference); 2: CCI = 1, 2; 3: CCI > 2 Variables Characteristics Number of Hospitalizations (per patient) 2010–14 Continuous External injury causes Dichotomous: 1 = Yes, 0 = No (Main diagnostic with ICD-9-MC) Reintervention during episode Dichotomous: 1 = Yes, 0 = No Major diagnostic category Categorical dichotomized. MDS 1 (reference) to MDS 25 (AP-DRG) Type of discharge service Categorical dichotomized. 1: Maternal-child (reference), 2: Critical care, 3: Medical, 4: Surgical; 5: Complex surgeries Weigh (AP-DRG) Continuous Length of stay Continuous: Discharge date–Hospitalization date (days) Age Continuous: Hospitalization date–date of birth (years) Sex Dichotomous: 1 = Women, 0 = Men Elective admissions Dichotomous: 1 = Yes, 0 = No Exitus Dichotomous: 1 = Yes, 0 = No Internal transfer Dichotomous: 1 = Yes, 0 = No Hospitalization in critical care Dichotomous: 1 = Yes, 0 = No Medical DRG Dichotomous: 1 = Yes, 0 = No Readmission in 30 days from discharge Dichotomous: 1 = Yes, 0 = No Charlson comorbidity index (CCI) Categorical dichotomized. 1: CCI = 0 (reference); 2: CCI = 1, 2; 3: CCI > 2 The average number of codes adjusted by case mix and standard performance were also calculated. The number of codes case mix adjusted was defined as the average number of codes that each period would have had if the episodes of the whole sample (reference standard) were codified with the average number of codes that each AP-DRG presents in such period. Conversely, the number of codes standard performance adjusted was defined as the average number of codes that each period would have had if their episodes were codified with the average number of codes that each AP-DRG presents in the whole sample. The difference of the number of codes case mix adjusted minus the average number of codes of the whole sample and the difference of the number of codes standard performance adjusted minus the average number of codes of the period were obtained for each one of the periods analyzed. Data analysis A descriptive analysis was performed using mean, median and standard deviations for continuous variables and frequencies distribution for the categorical ones. Comparison among PRE-INT, POST-INT and TP was performed by one-way analysis of variance or Kruskal–Wallis test, as appropriate. Similarly, the difference between insourced and outsourced coding performance in the studied period was analyzed using Mann–Whitney U or Student’s t-test. The homogeneity of variance between groups was tested using Levene's test. Models of multiple linear regression, Poisson regression and negative binomial regression were specified to adjust by risk the number of codes per individual characteristics. The goodness of fit of each model was examined to select the most appropriate. In Poisson regression, overdispersion was analyzed and compared with negative binomial regression through the Chi-square test. Moreover, as Poisson and negative binomial regressions allow dependent variables to take null values and the underlying distributions may result to be biased; zero truncated models were used in both cases. In the final fit, robust standard errors were used for parameter estimations [14]. In all cases, P-values <0.05 were considered statistically significant. Data analysis was performed using SPSS v21 and STATA v12.0. Results Demographic and clinical characteristics of patients The total coded DS during the period studied were 214 648 (99.86%) of which 97 901 (45.63%) were women and the average age of the patients was 50.1 years. The complexity of the cases attended ranged from 1.99 (2011) to 2.08 (2014) according to the average weight AP-DRG, and 0.98 (2010)–1.17 (2013) according to the Charlson comorbidity index (CCI) [15, 16]. The most frequent pathologies, grouped into major diagnostic categories (MDC AP-DRG) were: pregnancy, birth and postpartum (12.83%), circulatory system diseases (12.71%) and digestive system diseases (12.27%). The length of the stay decreased steadily 0.26 days in average annually from 8.52 in 2010 to 7.26 in 2014, the 30-days readmissions rate increased from 10.0% in 2010 to 12.42% in 2014 and the crude mortality rate remained stable throughout the study period with an average value of 2.96%. Quality of discharge summaries The reviewers achieved agreement by consensus in the 97 DS of PRE-INT (three were not found) and 100 of POST-INT evaluated. Statistically significant differences (P < 0.001) were found between pre- and post-intervention DS in quality (mean [SD]; PRE-INT = 2.49 (2.48); POST-INT = 1.54 (2.23)), length (PRE-INTE = 510.24 [374.03]; POST-INT = 1057.58 [683.96]) and readability (PRE-INT = 25.46 [7.70]; POST-INT = 19.59 [8.74]) (Fig. 2). Figure 2 View largeDownload slide Differences in DS quality, readability and length between pre- and post-intervention periods. Figure 2 View largeDownload slide Differences in DS quality, readability and length between pre- and post-intervention periods. Discharge summaries coding The average number of codes per episode of hospitalization for the whole studied period (2010–14) was 9.99 [5.99]. However, statistically significant differences (P = 0.001) were found in raw data between the PRE-INT, TP and POST-INT, 9.48 [5.94], 9.91 [6.03] and 10.77 [5.97], respectively. However, the implementation of the EHR neither changed the average number of codes not found in the DS per episode of hospitalization (PRE-INT = 0.505; POST-INT = 0.510) (P = 0.98) nor the number of codes not registered in the MBDS per DS (PRE-INT = 0.56; POST-INT = 0.96); (P = 0.11). The multiple linear regression model for risk-adjustment number of codes showed an acceptable coefficient of determination (0.631; P < 0.001) with neither multicollinearity (the values of the variance inflation factor were <5 in all variables) nor autocorrelation (Durbin–Watson = 1.97), the hypothesis of normality was rejected by the Shapiro–Wilk test (P < 0.001) and the existence of heterocedasticity was detected by Breusch–Pagan/Cook–Weisberg test (P < 0.001). The zero truncated Poisson regression fit was significant according to the Chi-square test (P < 0.001), with pseudo-R2 = 0.2837. However, the existence of overdispersion was accepted because the alpha parameter negative binomial regression was significantly different from 0 according to Chi-square test (P < 0.001). The zero truncated negative binomial regression also provided a good fit (coefficient of determination = 0.99) between the medians of the quintiles of the number of codes and the mean of the estimated values for each quintile. Consequently, the difference in risk-adjusted number of codes was calculated from the estimates obtained with this model (Table 2) and the mean difference between POST-INT and PRE-INT were the only ones found to be statistically significant (P < 0.001) (Table 3). Table 2 Adjustment of the negative binomial regression model for the number of diagnosis and procedure codes (NC) per episode Predictors IRR SE z P CI Number of hospitalizations (per patient) 2010–14 1.004 0.000 18.870 0.000 1.003 1.004 External injury causes 1.189 0.003 68.650 0.000 1.184 1.195 Reintervention during period 1.081 0.010 8.290 0.000 1.062 1.102 Major diagnostic category  2 0.896 0.010 –9.450 0.000 0.876 0.917  3 0.875 0.006 –18.070 0.000 0.862 0.887  4 1.072 0.005 16.200 0.000 1.063 1.081  5 1.047 0.005 10.490 0.000 1.038 1.056  6 1.003 0.005 0.550 0.586 0.993 1.012  7 1.003 0.005 0.630 0.530 0.993 1.013  8 0.913 0.005 –15.740 0.000 0.902 0.923  9 0.887 0.006 –16.780 0.000 0.874 0.899  10 1.009 0.007 1.230 0.220 0.995 1.023  11 1.073 0.005 14.390 0.000 1.063 1.083  12 0.900 0.008 –12.370 0.000 0.885 0.915  13 0.839 0.007 –22.630 0.000 0.826 0.852  14 0.891 0.006 –18.450 0.000 0.881 0.902  15 1.092 0.011 8.770 0.000 1.070 1.113  16 0.972 0.008 –3.510 0.000 0.957 0.988  17 0.810 0.006 –29.410 0.000 0.799 0.821  18 1.102 0.007 15.520 0.000 1.088 1.115  19 1.057 0.008 7.710 0.000 1.043 1.073  20 1.114 0.018 6.650 0.000 1.079 1.150  21 0.894 0.009 –10.740 0.000 0.876 0.913  22 1.020 0.065 0.320 0.751 0.900 1.157  23 0.962 0.012 –3.020 0.003 0.938 0.987  24 1.105 0.014 7.660 0.000 1.077 1.134  25 1.728 0.031 30.410 0.000 1.668 1.790 Type of discharge service  2 1.159 0.008 22.040 0.000 1.144 1.174  3 1.182 0.006 31.530 0.000 1.170 1.194  4 0.707 0.004 –58.270 0.000 0.699 0.715  5 0.842 0.005 –30.170 0.000 0.833 0.852 Weight (AP-DRG) 1.017 0.000 37.320 0.000 1.016 1.018 Length of stay 1.008 0.000 31.330 0.000 1.007 1.008 Age 1.006 0.000 103.070 0.000 1.005 1.006 Sex 1.012 0.002 6.170 0.000 1.008 1.015 Elective admissions 0.886 0.002 –47.420 0.000 0.881 0.89 Exitus 1.000 0.005 0.100 0.922 0.991 1.010 Handoff during hospitalization admission 1.098 0.003 32.600 0.000 1.092 1.105 Hospitalization in critical care 1.103 0.004 30.080 0.000 1.096 1.110 Medical DRG 1.066 0.003 22.420 0.000 1.060 1.072 Readmission in 30 days from discharge 0.995 0.003 –1.720 0.086 0.990 1.001 Charlson comorbidity index (CCI)  2 1.174 0.003 72.900 0.000 1.169 1.180  3 1.310 0.003 100.980 0.000 1.303 1.317 Intercept 5.484 0.036 258.530 0.000 5.414 5.555 /lnalpha –3.198 0.019 –3.236 –3.161 Alpha 0.041 0.001 1.003 1.004 Pseudo-R2 = 0.1519 (P = 0.001) Predictors IRR SE z P CI Number of hospitalizations (per patient) 2010–14 1.004 0.000 18.870 0.000 1.003 1.004 External injury causes 1.189 0.003 68.650 0.000 1.184 1.195 Reintervention during period 1.081 0.010 8.290 0.000 1.062 1.102 Major diagnostic category  2 0.896 0.010 –9.450 0.000 0.876 0.917  3 0.875 0.006 –18.070 0.000 0.862 0.887  4 1.072 0.005 16.200 0.000 1.063 1.081  5 1.047 0.005 10.490 0.000 1.038 1.056  6 1.003 0.005 0.550 0.586 0.993 1.012  7 1.003 0.005 0.630 0.530 0.993 1.013  8 0.913 0.005 –15.740 0.000 0.902 0.923  9 0.887 0.006 –16.780 0.000 0.874 0.899  10 1.009 0.007 1.230 0.220 0.995 1.023  11 1.073 0.005 14.390 0.000 1.063 1.083  12 0.900 0.008 –12.370 0.000 0.885 0.915  13 0.839 0.007 –22.630 0.000 0.826 0.852  14 0.891 0.006 –18.450 0.000 0.881 0.902  15 1.092 0.011 8.770 0.000 1.070 1.113  16 0.972 0.008 –3.510 0.000 0.957 0.988  17 0.810 0.006 –29.410 0.000 0.799 0.821  18 1.102 0.007 15.520 0.000 1.088 1.115  19 1.057 0.008 7.710 0.000 1.043 1.073  20 1.114 0.018 6.650 0.000 1.079 1.150  21 0.894 0.009 –10.740 0.000 0.876 0.913  22 1.020 0.065 0.320 0.751 0.900 1.157  23 0.962 0.012 –3.020 0.003 0.938 0.987  24 1.105 0.014 7.660 0.000 1.077 1.134  25 1.728 0.031 30.410 0.000 1.668 1.790 Type of discharge service  2 1.159 0.008 22.040 0.000 1.144 1.174  3 1.182 0.006 31.530 0.000 1.170 1.194  4 0.707 0.004 –58.270 0.000 0.699 0.715  5 0.842 0.005 –30.170 0.000 0.833 0.852 Weight (AP-DRG) 1.017 0.000 37.320 0.000 1.016 1.018 Length of stay 1.008 0.000 31.330 0.000 1.007 1.008 Age 1.006 0.000 103.070 0.000 1.005 1.006 Sex 1.012 0.002 6.170 0.000 1.008 1.015 Elective admissions 0.886 0.002 –47.420 0.000 0.881 0.89 Exitus 1.000 0.005 0.100 0.922 0.991 1.010 Handoff during hospitalization admission 1.098 0.003 32.600 0.000 1.092 1.105 Hospitalization in critical care 1.103 0.004 30.080 0.000 1.096 1.110 Medical DRG 1.066 0.003 22.420 0.000 1.060 1.072 Readmission in 30 days from discharge 0.995 0.003 –1.720 0.086 0.990 1.001 Charlson comorbidity index (CCI)  2 1.174 0.003 72.900 0.000 1.169 1.180  3 1.310 0.003 100.980 0.000 1.303 1.317 Intercept 5.484 0.036 258.530 0.000 5.414 5.555 /lnalpha –3.198 0.019 –3.236 –3.161 Alpha 0.041 0.001 1.003 1.004 Pseudo-R2 = 0.1519 (P = 0.001) IRR, incidence rate ratio; SE, standard error (robust); CI, confidence interval for the mean (95%). Table 2 Adjustment of the negative binomial regression model for the number of diagnosis and procedure codes (NC) per episode Predictors IRR SE z P CI Number of hospitalizations (per patient) 2010–14 1.004 0.000 18.870 0.000 1.003 1.004 External injury causes 1.189 0.003 68.650 0.000 1.184 1.195 Reintervention during period 1.081 0.010 8.290 0.000 1.062 1.102 Major diagnostic category  2 0.896 0.010 –9.450 0.000 0.876 0.917  3 0.875 0.006 –18.070 0.000 0.862 0.887  4 1.072 0.005 16.200 0.000 1.063 1.081  5 1.047 0.005 10.490 0.000 1.038 1.056  6 1.003 0.005 0.550 0.586 0.993 1.012  7 1.003 0.005 0.630 0.530 0.993 1.013  8 0.913 0.005 –15.740 0.000 0.902 0.923  9 0.887 0.006 –16.780 0.000 0.874 0.899  10 1.009 0.007 1.230 0.220 0.995 1.023  11 1.073 0.005 14.390 0.000 1.063 1.083  12 0.900 0.008 –12.370 0.000 0.885 0.915  13 0.839 0.007 –22.630 0.000 0.826 0.852  14 0.891 0.006 –18.450 0.000 0.881 0.902  15 1.092 0.011 8.770 0.000 1.070 1.113  16 0.972 0.008 –3.510 0.000 0.957 0.988  17 0.810 0.006 –29.410 0.000 0.799 0.821  18 1.102 0.007 15.520 0.000 1.088 1.115  19 1.057 0.008 7.710 0.000 1.043 1.073  20 1.114 0.018 6.650 0.000 1.079 1.150  21 0.894 0.009 –10.740 0.000 0.876 0.913  22 1.020 0.065 0.320 0.751 0.900 1.157  23 0.962 0.012 –3.020 0.003 0.938 0.987  24 1.105 0.014 7.660 0.000 1.077 1.134  25 1.728 0.031 30.410 0.000 1.668 1.790 Type of discharge service  2 1.159 0.008 22.040 0.000 1.144 1.174  3 1.182 0.006 31.530 0.000 1.170 1.194  4 0.707 0.004 –58.270 0.000 0.699 0.715  5 0.842 0.005 –30.170 0.000 0.833 0.852 Weight (AP-DRG) 1.017 0.000 37.320 0.000 1.016 1.018 Length of stay 1.008 0.000 31.330 0.000 1.007 1.008 Age 1.006 0.000 103.070 0.000 1.005 1.006 Sex 1.012 0.002 6.170 0.000 1.008 1.015 Elective admissions 0.886 0.002 –47.420 0.000 0.881 0.89 Exitus 1.000 0.005 0.100 0.922 0.991 1.010 Handoff during hospitalization admission 1.098 0.003 32.600 0.000 1.092 1.105 Hospitalization in critical care 1.103 0.004 30.080 0.000 1.096 1.110 Medical DRG 1.066 0.003 22.420 0.000 1.060 1.072 Readmission in 30 days from discharge 0.995 0.003 –1.720 0.086 0.990 1.001 Charlson comorbidity index (CCI)  2 1.174 0.003 72.900 0.000 1.169 1.180  3 1.310 0.003 100.980 0.000 1.303 1.317 Intercept 5.484 0.036 258.530 0.000 5.414 5.555 /lnalpha –3.198 0.019 –3.236 –3.161 Alpha 0.041 0.001 1.003 1.004 Pseudo-R2 = 0.1519 (P = 0.001) Predictors IRR SE z P CI Number of hospitalizations (per patient) 2010–14 1.004 0.000 18.870 0.000 1.003 1.004 External injury causes 1.189 0.003 68.650 0.000 1.184 1.195 Reintervention during period 1.081 0.010 8.290 0.000 1.062 1.102 Major diagnostic category  2 0.896 0.010 –9.450 0.000 0.876 0.917  3 0.875 0.006 –18.070 0.000 0.862 0.887  4 1.072 0.005 16.200 0.000 1.063 1.081  5 1.047 0.005 10.490 0.000 1.038 1.056  6 1.003 0.005 0.550 0.586 0.993 1.012  7 1.003 0.005 0.630 0.530 0.993 1.013  8 0.913 0.005 –15.740 0.000 0.902 0.923  9 0.887 0.006 –16.780 0.000 0.874 0.899  10 1.009 0.007 1.230 0.220 0.995 1.023  11 1.073 0.005 14.390 0.000 1.063 1.083  12 0.900 0.008 –12.370 0.000 0.885 0.915  13 0.839 0.007 –22.630 0.000 0.826 0.852  14 0.891 0.006 –18.450 0.000 0.881 0.902  15 1.092 0.011 8.770 0.000 1.070 1.113  16 0.972 0.008 –3.510 0.000 0.957 0.988  17 0.810 0.006 –29.410 0.000 0.799 0.821  18 1.102 0.007 15.520 0.000 1.088 1.115  19 1.057 0.008 7.710 0.000 1.043 1.073  20 1.114 0.018 6.650 0.000 1.079 1.150  21 0.894 0.009 –10.740 0.000 0.876 0.913  22 1.020 0.065 0.320 0.751 0.900 1.157  23 0.962 0.012 –3.020 0.003 0.938 0.987  24 1.105 0.014 7.660 0.000 1.077 1.134  25 1.728 0.031 30.410 0.000 1.668 1.790 Type of discharge service  2 1.159 0.008 22.040 0.000 1.144 1.174  3 1.182 0.006 31.530 0.000 1.170 1.194  4 0.707 0.004 –58.270 0.000 0.699 0.715  5 0.842 0.005 –30.170 0.000 0.833 0.852 Weight (AP-DRG) 1.017 0.000 37.320 0.000 1.016 1.018 Length of stay 1.008 0.000 31.330 0.000 1.007 1.008 Age 1.006 0.000 103.070 0.000 1.005 1.006 Sex 1.012 0.002 6.170 0.000 1.008 1.015 Elective admissions 0.886 0.002 –47.420 0.000 0.881 0.89 Exitus 1.000 0.005 0.100 0.922 0.991 1.010 Handoff during hospitalization admission 1.098 0.003 32.600 0.000 1.092 1.105 Hospitalization in critical care 1.103 0.004 30.080 0.000 1.096 1.110 Medical DRG 1.066 0.003 22.420 0.000 1.060 1.072 Readmission in 30 days from discharge 0.995 0.003 –1.720 0.086 0.990 1.001 Charlson comorbidity index (CCI)  2 1.174 0.003 72.900 0.000 1.169 1.180  3 1.310 0.003 100.980 0.000 1.303 1.317 Intercept 5.484 0.036 258.530 0.000 5.414 5.555 /lnalpha –3.198 0.019 –3.236 –3.161 Alpha 0.041 0.001 1.003 1.004 Pseudo-R2 = 0.1519 (P = 0.001) IRR, incidence rate ratio; SE, standard error (robust); CI, confidence interval for the mean (95%). Table 3 Differences in diagnosis and procedure codes per episode adjusted for risk (zero truncated negative binomial regression) and multiple comparisons between periods Period N Mean SD SE CI Period II Mean dif P PRE-INT 42 182 –0.418 50.077 0.244 –0.896 –0.896 PT –0.30 0.461 POST-INT –0.74 0.001 TP 42 126 –0.118 4.303 0.021 –0.159 –0.077 PRE-INT 0.30 0.461 POST-INT –0.44 0.108 POST-INT 43 928 0.319 17.194 0.082 0.158 0.480 PRE-INT –0.74 0.001 PT 0.44 0.108 Total 128 236 –0.067 30.534 0.085 –0.234 –0.1 Period N Mean SD SE CI Period II Mean dif P PRE-INT 42 182 –0.418 50.077 0.244 –0.896 –0.896 PT –0.30 0.461 POST-INT –0.74 0.001 TP 42 126 –0.118 4.303 0.021 –0.159 –0.077 PRE-INT 0.30 0.461 POST-INT –0.44 0.108 POST-INT 43 928 0.319 17.194 0.082 0.158 0.480 PRE-INT –0.74 0.001 PT 0.44 0.108 Total 128 236 –0.067 30.534 0.085 –0.234 –0.1 N, number of episodes; SD, standard deviation; SE, standard error; CI, confidence Interval for the average value (95%); Mean dif, mean differences; PRE-INT, pre-intervention period; TP, transition period; POST-INT, post implementation period. F = 524.948 (P = 0.001). Table 3 Differences in diagnosis and procedure codes per episode adjusted for risk (zero truncated negative binomial regression) and multiple comparisons between periods Period N Mean SD SE CI Period II Mean dif P PRE-INT 42 182 –0.418 50.077 0.244 –0.896 –0.896 PT –0.30 0.461 POST-INT –0.74 0.001 TP 42 126 –0.118 4.303 0.021 –0.159 –0.077 PRE-INT 0.30 0.461 POST-INT –0.44 0.108 POST-INT 43 928 0.319 17.194 0.082 0.158 0.480 PRE-INT –0.74 0.001 PT 0.44 0.108 Total 128 236 –0.067 30.534 0.085 –0.234 –0.1 Period N Mean SD SE CI Period II Mean dif P PRE-INT 42 182 –0.418 50.077 0.244 –0.896 –0.896 PT –0.30 0.461 POST-INT –0.74 0.001 TP 42 126 –0.118 4.303 0.021 –0.159 –0.077 PRE-INT 0.30 0.461 POST-INT –0.44 0.108 POST-INT 43 928 0.319 17.194 0.082 0.158 0.480 PRE-INT –0.74 0.001 PT 0.44 0.108 Total 128 236 –0.067 30.534 0.085 –0.234 –0.1 N, number of episodes; SD, standard deviation; SE, standard error; CI, confidence Interval for the average value (95%); Mean dif, mean differences; PRE-INT, pre-intervention period; TP, transition period; POST-INT, post implementation period. F = 524.948 (P = 0.001). The binomial negative regression showed (Table 2) statistically significant higher incidence rates ratio (IRR > 1) in diagnosis and procedure codes in discharges from women, hospitalization in critical care, external injury causes, reintervention during period, handoff during hospitalization admission, high AP-DRG weight and MDC 4, 5, 11, 15, 18, 19, 20, 24, 25 and lower incidence (IRR < 1) in episodes with elective admissions or readmissions, while in exitus and in MDC 6, 7, 10, 16, 17 differences were not statistically significant (P > 0.05). In the outsourcing period, the average number of codes per episode coded by insourced coders (n = 12.253) was higher than those coded by outsourced coders (n = 8.359) and the difference was statistically significant (P < 0.001). After risk-adjustment, the difference in average number of risk-adjusted codes was also significantly higher in episodes coded by insourced coders (0.035 vs. −0.142; P < 0.001). Finally, differences in number of codes adjusted by either case mix and or standard performance were higher in the POST-INT than in TP as well as higher than in PRE-INT (Table 4). Therefore, the observed difference in the average number of codes is unrelated to changes in casuistry or complexity in the studied periods. Table 4 Differences in diagnosis and procedure codes per hospitalization episode adjusted by case mix and standard performance PRE-INT TP POST-INT Mean NC (population study) 9.48 9.91 10.77 Mean (CI) NC (sample for evaluation of DS) 9.14 (8.64–9.64) 10.33 (9.83–10.83) NCCMA 9.80 9.91 10.44 NCSPA 9.68 9.98 10.32 DNCCMA −0.51 −0.08 0.78 DNCSPA −0.20 −0.07 0.45 PRE-INT TP POST-INT Mean NC (population study) 9.48 9.91 10.77 Mean (CI) NC (sample for evaluation of DS) 9.14 (8.64–9.64) 10.33 (9.83–10.83) NCCMA 9.80 9.91 10.44 NCSPA 9.68 9.98 10.32 DNCCMA −0.51 −0.08 0.78 DNCSPA −0.20 −0.07 0.45 PRE-INT, pre-intervention period; TP, transition period; POST-INT, post-intervention period; CI, confidence interval (95%); NC, number of diagnosis and procedure codes; NCCMA, NC mean (case mix adjusted); NCSPA, mean NC (standard performance adjusted); DNCCMA, difference adjusted by case mix; DNCSPA, difference adjusted by standard performance. Table 4 Differences in diagnosis and procedure codes per hospitalization episode adjusted by case mix and standard performance PRE-INT TP POST-INT Mean NC (population study) 9.48 9.91 10.77 Mean (CI) NC (sample for evaluation of DS) 9.14 (8.64–9.64) 10.33 (9.83–10.83) NCCMA 9.80 9.91 10.44 NCSPA 9.68 9.98 10.32 DNCCMA −0.51 −0.08 0.78 DNCSPA −0.20 −0.07 0.45 PRE-INT TP POST-INT Mean NC (population study) 9.48 9.91 10.77 Mean (CI) NC (sample for evaluation of DS) 9.14 (8.64–9.64) 10.33 (9.83–10.83) NCCMA 9.80 9.91 10.44 NCSPA 9.68 9.98 10.32 DNCCMA −0.51 −0.08 0.78 DNCSPA −0.20 −0.07 0.45 PRE-INT, pre-intervention period; TP, transition period; POST-INT, post-intervention period; CI, confidence interval (95%); NC, number of diagnosis and procedure codes; NCCMA, NC mean (case mix adjusted); NCSPA, mean NC (standard performance adjusted); DNCCMA, difference adjusted by case mix; DNCSPA, difference adjusted by standard performance. Discussion Health systems need reliable information for multiple purposes. The essential is undoubtedly the use of clinical information to provide the best possible healthcare. The relationship between standardization of care documentation and records with the improvement of clinical practices has been proved [17] as well as the relationship between amount and quality of information on patient care results [18]. Administrative data, usually recorded by governmental initiatives [19], manage relevant information on services utilization [20], health outcomes [21], healthcare quality [22, 23] and their costs [24] for planning, supporting decisions and evaluating the effectiveness of policies. The implementation of EHR has potential to greatly improve patient safety, efficiency and effectiveness of healthcare. Although it has raised some concerns about data quality and its implications [25], it seems to provide more complete and detailed clinical documentation [6]. Moreover, the DS, an essential element of the EHR, benefits from a standard format that uses the information previously recorded in templates and progress notes to compose the final document at the time of patient discharge. Our results show that after implementation of EHR the DS had higher quality but were longer and harder to read than previously. The quality improvement is consistent with the literature. Several authors had evaluated the impact of similar interventions. Thus, O’Leary et al. [11] concluded that the use of a composition tool in the EHR improved the quality and readability of DS and Reinke et al. [12] showed that the intervention also improved the quality of DS; but they were shorter and harder to read. In our view, the improvement in quality results from both the widespread use of EHR structured templates (easier data recording) and the simplicity of the content importing technologies (data transfer to DS). The results variability in terms of length and readability may depend on the proper use of the EHR. Thus, it can be partially explained by the routine inclusion of diagnostic tests results, particularly lab tests, which take up a significant space with little real useful information and complicate the reading. It can be also be due to the improper use of functionalities, such as the ‘copy and paste’ in the composition of DS [26], particularly in the patients personal and family history. The implementation of EHR in our hospital did not change the proportion of diagnoses and procedures coded in the DS to the total registered in the MBDS. This fact suggests that the nature and amount of information available to the physician at the time of discharge did not change with the EHR implementation. However, the number of diagnoses and procedures coded per DS has significantly increased with EHR, using both raw and risk-adjusted data. This effect was not observed during the TP, but it might be related to the lower performance of the outsourced coding. Thus, our data suggest that the implementation of EHR improves availability of clinical information for the DS and coding performance. The importance of this finding hinges on the utility of the administrative databases such as the MBDS in any of the many aspects related to healthcare. The appropriateness of the use of administrative databases in health research projects has been debated. However, its availability and low cost makes them attractive tools for healthcare research. Thus, validation studies of results obtained from administrative data versus clinical records [27] or clinical chart reviews [28] had spurred. On one hand, inaccuracies have been described in the validation of certain diseases and comorbidities [29]; on the other hand, it has been concluded that there are no significant differences between risk-adjusted mortality and readmissions for example in acute myocardial infarction or heart failure [30]. Our study supports the notion that adoption of EHR improves MBDS reliability, and therefore its utility for secondary uses. Limitations Our study has some limitations. It is a pre–post-intervention design without control group (the analyzed DS are done either in one period or another) and the observed effect is attributed to the intervention but it does not dismiss any other associated factors. To control for this, we have adjusted by risk the dependent variable by different methods, analyzed if there were changes in the proportion of diagnoses and procedures present in the DS that were not coded, and therefore not recorded in the MBDS; and assessed the differences between coders (insourced and outsourced) in the period when the codification was partially outsourced. However, given that during the pre- and post-intervention periods no outsourcing took place, we understand that the results, especially risk-adjusted differences in favor of the group of H12O coders, indicate the coding pattern has been stable. Consequently, we can exclude the one factor that in our opinion could be the main cause of confusion. Another limitation was inherent to the format of the DS since the intervention changed the DS aspect the reviewers could easily identify the period of the reports evaluated. Finally, the quality evaluation method is internal to the H12O and has not been externally validated, although we have followed the criteria specified in our country regulations. Regarding the quality of the DS and the availability of clinical documentation for secondary uses, future research on the impact of EHR implementation is needed to deep our understanding. It would be advisable to study the external validity in other healthcare institutions, including, if possible, studies with a control group. It is as well interesting to ascertain whether the use of ‘copy and paste’ tools available in the EHR affects the understandability of the DS to the extent to compromise patient safety. Thus, it would be necessary to investigate ways to improve the transmission of clinical information between healthcare settings, patients and professionals. Conclusions The implementation of EHR improves the quality of the DS composed from clinical information recorded during the episode of hospitalization. The improper use of the content importing technologies and inclusion of non-relevant content can make them more extensive and less legible. The increased availability of clinical information in DS allows the codification of greater number of diagnoses and procedures and, ultimately, improves utility of MBDS for secondary use. Acknowledgements The authors have performed this work with the support of the Coding Unit and Management Control Service of the Hospital Universitario 12 de Octubre. It is kindly appreciated the dedication and effort of their staff. Funding There were no funding sources that contributed to this study. References 1 Wilson S , Ruscoe W , Chapman M et al. . General practitioner–hospital communications: a review of discharge summaries . J Qual Clin Pract 2001 ; 21 : 104 – 8 . http://www.ncbi.nlm.nih.gov/pubmed/11856404. Google Scholar Crossref Search ADS PubMed 2 Buurman BM , Verhaegh KJ , Smeulers M et al. . Improving handoff communication from hospital to home: the development, implementation and evaluation of a personalized patient discharge letter . Int J Qual Health Care 2016 ; 28 : 384 – 90 . http://www.ncbi.nlm.nih.gov/pubmed/27224995. Google Scholar Crossref Search ADS PubMed 3 Callen JL , Alderton M , McIntosh J . Evaluation of electronic discharge summaries: a comparison of documentation in electronic and handwritten discharge summaries . Int J Med Inform 2008 ; 77 : 613 – 20 . http://www.ncbi.nlm.nih.gov/pubmed/18294904. Google Scholar Crossref Search ADS PubMed 4 Gagnier JJ , Derosier JM , Maratt JD et al. . Development, implementation and evaluation of a patient handoff tool to improve safety in orthopaedic surgery . Int J Qual Health Care 2016 ; 28 : 363 – 70 . http://www.ncbi.nlm.nih.gov/pubmed/27090398. Google Scholar Crossref Search ADS PubMed 5 Ministerio de Sanidad y Consumo . Metodología de análisis de la hospitalización en el Sistema Nacional de Salud. Modelo de indicadores basado en el Registro de Altas (CMBD). 2007 ;126. http://www.msssi.gob.es/estadEstudios/estadisticas/docs/metod_modelo_cmbd_pub.pdf 6 Häyrinen K , Saranto K , Nykänen P . Definition, structure, content, use and impacts of electronic health records: a review of the research literature . Int J Med Inform 2008 ; 77 : 291 – 304 . http://www.ncbi.nlm.nih.gov/pubmed/17951106. Google Scholar Crossref Search ADS PubMed 7 Marca G , Perez AJ , Blanco-Garcia MG et al. . The use of electronic health records in Spanish hospitals . Health Inf Manag 2014 ; 43 : 37 – 44 . http://www.ncbi.nlm.nih.gov/pubmed/24682737. Google Scholar PubMed 8 Reid B , Kelly Z , Westbrook J . A preliminary analysis of differences in coded data from Australia and Maryland . Health Inf Manag 2002 ; 30 : 9 – 20 . http://www.ncbi.nlm.nih.gov/pubmed/19468136. Google Scholar PubMed 9 Walker RL , Hennessy DA , Johansen H et al. . Implementation of ICD-10 in Canada: how has it impacted coded hospital discharge data? BMC Health Serv Res 2012 ; 12 : 149 . http://www.ncbi.nlm.nih.gov/pubmed/22682405. Google Scholar Crossref Search ADS PubMed 10 Burke HB , Sessums LL , Hoang A et al. . Electronic health records improve clinical note quality . J Am Med Inform Assoc 2015 ; 22 : 199 – 205 . http://www.ncbi.nlm.nih.gov/pubmed/25342178. Google Scholar PubMed 11 O’Leary KJ , Liebovitz DM , Feinglass J et al. . Creating a better discharge summary: improvement in quality and timeliness using an electronic discharge summary . J Hosp Med 2009 ; 4 : 219 – 25 . http://www.ncbi.nlm.nih.gov/pubmed/19267397. Google Scholar Crossref Search ADS PubMed 12 Reinke CE , Kelz RR , Baillie CA et al. . Timeliness and quality of surgical discharge summaries after the implementation of an electronic format . Am J Surg 2014 ; 207 : 7 – 16 . http://www.ncbi.nlm.nih.gov/pubmed/24269034. Google Scholar Crossref Search ADS PubMed 13 Lorda PS , Barrio Cantalejo IM , Concheiro Carro L . Legibilidad de los formularios escritos de consentimiento informado . Med Clin (Barc) 2016 ; 107 : 524 – 9 . http://cat.inist.fr/?aModele=afficheN&cpsidt=11023799. 14 Cameron AC , Trivedi PK . Stata Version 11 and / Microeconometrics using Stata. 2010 ;4–7. 15 Charlson ME , Pompei P , Ales KL et al. . A new method of classifying prognostic comorbidity in longitudinal studies: development and validation . J Chronic Dis 1987 ; 40 : 373 – 83 . http://www.ncbi.nlm.nih.gov/pubmed/3558716. Google Scholar Crossref Search ADS PubMed 16 Deyo RA , Cherkin DC , Ciol MA . Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases . J Clin Epidemiol 1992 ; 45 : 613 – 9 . http://www.ncbi.nlm.nih.gov/pubmed/1607900. Google Scholar Crossref Search ADS PubMed 17 Wright P , Jansen C , Wyatt JC . How to limit clinical errors in interpretation of data . Lancet 1998 ; 352 : 1539 – 43 . http://www.ncbi.nlm.nih.gov/pubmed/9820319. Google Scholar Crossref Search ADS PubMed 18 Dunlay SM , Alexander KP , Melloni C et al. . Medical records and quality of care in acute coronary syndromes: results from CRUSADE . Arch Intern Med 2008 ; 168 : 1692 – 8 . http://www.ncbi.nlm.nih.gov/pubmed/18695085. Google Scholar Crossref Search ADS PubMed 19 Chowdhury TT , Hemmelgarn B . Evidence-based decision-making 6: utilization of administrative databases for health services research . Methods Mol Biol 2015 ; 1281 : 469 – 84 . http://www.ncbi.nlm.nih.gov/pubmed/25694328. Google Scholar Crossref Search ADS PubMed 20 Ruiz M , Bottle A , Aylin PP . The Global Comparators project: international comparison of 30-day in-hospital mortality by day of the week . BMJ Qual Saf 2015 ; 24 : 492 – 504 . http://www.ncbi.nlm.nih.gov/pubmed/26150550. Google Scholar Crossref Search ADS PubMed 21 Nuti SV , Wang Y , Masoudi FA et al. . Improvements in the distribution of hospital performance for the care of patients with acute myocardial infarction, heart failure, and pneumonia, 2006–2011 . Med Care 2015 ; 53 : 485 – 91 . http://www.ncbi.nlm.nih.gov/pubmed/25906012. Google Scholar Crossref Search ADS PubMed 22 Iezzoni LI . Assessing quality using administrative data . Ann Intern Med 1997 ; 127 : 666 – 74 . http://www.ncbi.nlm.nih.gov/pubmed/9382378. Google Scholar Crossref Search ADS PubMed 23 Knight HE , Gurol-Urganci I , Mahmood TA et al. . Evaluating maternity care using national administrative health datasets: how are statistics affected by the quality of data on method of delivery? BMC Health Serv Res 2013 ; 13 : 200 . http://www.ncbi.nlm.nih.gov/pubmed/23721128. Google Scholar Crossref Search ADS PubMed 24 Riley GF . Administrative and claims records as sources of health care cost data . Med Care 2009 ; 47 : S51 – 5 . http://www.ncbi.nlm.nih.gov/pubmed/19536019. Google Scholar Crossref Search ADS PubMed 25 Callen J . What is the impact of electronic health records on the quality of health data? Heal Inf Manag J 2014 ; 43 : 42 – 3 . 26 Hirschtick RE . A piece of my mind. Copy-and-paste . JAMA 2006 ; 295 : 2335 – 6 . http://www.ncbi.nlm.nih.gov/pubmed/16720812. Google Scholar Crossref Search ADS PubMed 27 Keenan PS , Normand S-LT , Lin Z et al. . An administrative claims measure suitable for profiling hospital performance on the basis of 30-day all-cause readmission rates among patients with heart failure . Circ Cardiovasc Qual Outcomes 2008 ; 1 : 29 – 37 . http://www.ncbi.nlm.nih.gov/pubmed/20031785. Google Scholar Crossref Search ADS PubMed 28 Horwitz LI , Grady JN , Cohen DB et al. . Development and validation of an algorithm to identify planned readmissions from claims data . J Hosp Med 2015 ; 10 : 670 – 7 . http://www.ncbi.nlm.nih.gov/pubmed/26149225. Google Scholar Crossref Search ADS PubMed 29 Hannan EL , Kilburn H , Lindsey ML et al. . Clinical versus administrative data bases for CABG surgery. Does it matter? Med Care 1992 ; 30 : 892 – 907 . http://www.ncbi.nlm.nih.gov/pubmed/1405795. Google Scholar Crossref Search ADS PubMed 30 Krumholz HM , Wang Y , Mattera JA et al. . An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with an acute myocardial infarction . Circulation 2006 ; 113 : 1683 – 92 . http://www.ncbi.nlm.nih.gov/pubmed/16549637. Google Scholar Crossref Search ADS PubMed © The Author(s) 2018. Published by Oxford University Press in association with the International Society for Quality in Health Care. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png International Journal for Quality in Health Care Oxford University Press

Impact of the implementation of electronic health records on the quality of discharge summaries and on the coding of hospitalization episodes

Loading next page...
 
/lp/ou_press/impact-of-the-implementation-of-electronic-health-records-on-the-djmltqMQIa
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press in association with the International Society for Quality in Health Care. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com
ISSN
1353-4505
eISSN
1464-3677
D.O.I.
10.1093/intqhc/mzy075
Publisher site
See Article on Publisher Site

Abstract

Abstract Objective To determine whether the implementation and use of the electronic health records (EHR) modifies the quality, readability and/or the length of the discharge summaries (DS) and the average number of coded diagnosis and procedures per hospitalization episode. Design A pre–post-intervention descriptive study conducted between 2010 and 2014. Setting The ‘Hospital Universitario 12 de Octubre’ (H12O) of Madrid (Spain). A tertiary University Hospital of up to 1200 beds. Intervention Implementation and systematic use of the EHR. Main Outcome Measures The quality, length and readability of the DS and the number of diagnosis and procedures codes by raw and risk-adjusted data. Results A total of 200 DS were included in the present work. After the implementation of the EHR the DS had better quality per formal requirements, although were longer and harder to read (P < 0.001). The average number of coded diagnoses and procedures was increased, 9.48 in the PRE-INT and 10.77 in the POST-INT, and the difference was statistically significant (P < 0.001) in both raw and risk-adjusted data. Conclusions The implementation of EHR improves the formal quality of DS, although poor use of EHR functionalities might reduce its understandability. Having more clinical information immediately available due to EHR increases the number of diagnosis and procedure codes enhancing their utility for secondary uses. electronic health records, discharge summary, diagnosis and procedure codes, hospital care, quality improvement Introduction The discharge summary (DS) is an essential document for communication among healthcare professionals of different levels of care [1, 2]. It is considered indispensable for a proper continuing patient care and, in Spain, it is legally required and its contents defined. Until recently, DS were handwritten in a paper-based support; however, the progress of information technologies such as electronic health record (EHR) has enabled the automatic creation of the DS from clinical information registered during the hospitalization episode [3, 4] or any other information from the patient medical record. Moreover, the quality of the DS has implications other than the continuing patient care, since the Hospital Discharge Database, the main administrative database, also known as the Minimum Basic Data Set (MBDS) gathers the information from the coded diagnoses and procedures found in the DS. The MBDS is the largest set of administrative data of hospitalized patients and the major source of information of morbidity in Spain [5]. The systematic use of EHR in the clinical practice is spreading quickly [6, 7], therefore, as any other healthcare technology has to be evaluated and its impact in the processes of patient care and Healthcare Information Systems understood. Up to date, the comparison in the average number of coded diagnoses and procedures during hospitalization among different countries [8] as well as before and after the use of a new edition of the International Classification of Diseases [9] have been analyzed. As well, it has been shown that the utilization of EHR improves the quality of outpatient clinical notes [10] and of DS [11, 12]; however, to the best of our knowledge, this is the first study to show the impact of the EHR on coding of hospitalization episodes. Thus, the present study analyzed whether the implementation of EHR has modified the quality, readability and length of the DS as well as it has affected the number of coded diagnoses and procedures per hospitalization episode in a tertiary university Hospital of 1200 beds. Methods Study design and population This was a pre–post-intervention observational study of DS from the population of patients discharged out of the H12O since 1 January 2010 to 31 December 2014. The inclusion criterion was to have the hospitalization episode coded and registered in the MBDS. The exclusion criteria were not coded hospitalization episodes and patients grouped by all patient diagnosis-related groups (AP-DRG) with weight 0 (AP-DRG 469: principal diagnosis invalid as discharge diagnosis and AP-DRG 470: ungroupable) and indeterminate sex (Fig. 1). The data sources were the MBDS and the DS. The number of coded diagnoses and procedures were studied in all the episodes that met the inclusion criteria (N = 214 648). Figure 1 View largeDownload slide Inclusion and exclusion criteria used to determine the study population. Figure 1 View largeDownload slide Inclusion and exclusion criteria used to determine the study population. Intervention The intervention was the implementation and systematic use of EHR in all the H12O services and units. It was progressively deployed during 2012 and had full implementation with a systematic use by 2013. The years 2010 and 2011 were considered as pre-intervention period (PRE-INT); 2012 as transition period (TP); finally, 2013 and 2014 as post-intervention period (POST-INT). Insourced versus outsourced coding H12O has a specialized coding unit, but the task was partly outsourced from December of 2011 to August of 2013, to offset an unusual additional workload. The coexistence during this period of two distinct groups of coders was considered as a possible confounding factor and its impact on coding performance analyzed. Instrumentation, study variables and data source To evaluate the DS quality in PRE-INT and POST-INT, a sample was selected by simple random sampling. A review team consisting of three resident physicians was formed. Two senior physicians were consultants for the review team: the Coordinator of the Quality Unit and the Coordinator of the Patient Management Unit of the H12O. An initial training phase for the review team was considered and 50 DS of each period were analyzed and the reviewers performed an independent evaluation. However, the assessment was jointly validated, and the review criteria unified with the consensus of the two consultants. After the training phase, the sample size was calculated in 100 DS per period (200 in total) related to the populations of PRE-INT (42 182 DS) and POST-INT (43 928 DS) and considering the average number of diagnoses and procedure codes (NC) as well as their standard deviation (SD), obtained in the training phase, to detect a difference of means of 0.5 and a SD of 1.2, with a confidence level of 95% and a statistical power of 80%. The main variables analyzed from the selected sample were the quality, length and readability of the DS. To determine the quality, a method developed by the Hospital's Medical Charts Committee was used. It evaluates the compliance with the criteria established on the current legislation concerning to MBDS in the Spanish National Health System. It includes 13 DS items (admission and discharge dates; reason for admission; discharge status; health history; evolution; principal and secondary diagnoses surgical procedures; other procedures and therapeutic recommendations at discharge) scoring 0 points if the DS contained the minimum information required in each issue, 1 point if the information was incomplete or 2 points if it was missing, therefore, quality of DS=∑iVi and 0 ≤ QDS ≤ 26. Readability was quantified with the Spanish adaptation of the Flesch-Kincaid index, which measures formal readability between 0 and 100, considering the relationship between text difficulty, sentence length and words chosen [13]. Length was evaluated by a simple word count. Other variables studied were the NC according to the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM). Analyzing both, the codes registered in the MBDS previously not included in the DS and the diagnoses and procedures found in the DS but not registered in the MBDS. To measure other aspects of the EHR impact on coding, we considered the number of ICD-9-MC diagnosis and procedure codes as the outcome variable. The number of codes was directly obtained counting them in the H12O MBDS and to avoid confusion due to demographic and clinical characteristics of patients, raw data were risk-adjusted. The independent variables used for this adjustment obtained from the MBDS, are shown in Table 1. Table 1 Independent variables used to estimate the number of coded diagnoses and procedures for hospital episode based on the demographic and clinical characteristics of patients Variables Characteristics Number of Hospitalizations (per patient) 2010–14 Continuous External injury causes Dichotomous: 1 = Yes, 0 = No (Main diagnostic with ICD-9-MC) Reintervention during episode Dichotomous: 1 = Yes, 0 = No Major diagnostic category Categorical dichotomized. MDS 1 (reference) to MDS 25 (AP-DRG) Type of discharge service Categorical dichotomized. 1: Maternal-child (reference), 2: Critical care, 3: Medical, 4: Surgical; 5: Complex surgeries Weigh (AP-DRG) Continuous Length of stay Continuous: Discharge date–Hospitalization date (days) Age Continuous: Hospitalization date–date of birth (years) Sex Dichotomous: 1 = Women, 0 = Men Elective admissions Dichotomous: 1 = Yes, 0 = No Exitus Dichotomous: 1 = Yes, 0 = No Internal transfer Dichotomous: 1 = Yes, 0 = No Hospitalization in critical care Dichotomous: 1 = Yes, 0 = No Medical DRG Dichotomous: 1 = Yes, 0 = No Readmission in 30 days from discharge Dichotomous: 1 = Yes, 0 = No Charlson comorbidity index (CCI) Categorical dichotomized. 1: CCI = 0 (reference); 2: CCI = 1, 2; 3: CCI > 2 Variables Characteristics Number of Hospitalizations (per patient) 2010–14 Continuous External injury causes Dichotomous: 1 = Yes, 0 = No (Main diagnostic with ICD-9-MC) Reintervention during episode Dichotomous: 1 = Yes, 0 = No Major diagnostic category Categorical dichotomized. MDS 1 (reference) to MDS 25 (AP-DRG) Type of discharge service Categorical dichotomized. 1: Maternal-child (reference), 2: Critical care, 3: Medical, 4: Surgical; 5: Complex surgeries Weigh (AP-DRG) Continuous Length of stay Continuous: Discharge date–Hospitalization date (days) Age Continuous: Hospitalization date–date of birth (years) Sex Dichotomous: 1 = Women, 0 = Men Elective admissions Dichotomous: 1 = Yes, 0 = No Exitus Dichotomous: 1 = Yes, 0 = No Internal transfer Dichotomous: 1 = Yes, 0 = No Hospitalization in critical care Dichotomous: 1 = Yes, 0 = No Medical DRG Dichotomous: 1 = Yes, 0 = No Readmission in 30 days from discharge Dichotomous: 1 = Yes, 0 = No Charlson comorbidity index (CCI) Categorical dichotomized. 1: CCI = 0 (reference); 2: CCI = 1, 2; 3: CCI > 2 Table 1 Independent variables used to estimate the number of coded diagnoses and procedures for hospital episode based on the demographic and clinical characteristics of patients Variables Characteristics Number of Hospitalizations (per patient) 2010–14 Continuous External injury causes Dichotomous: 1 = Yes, 0 = No (Main diagnostic with ICD-9-MC) Reintervention during episode Dichotomous: 1 = Yes, 0 = No Major diagnostic category Categorical dichotomized. MDS 1 (reference) to MDS 25 (AP-DRG) Type of discharge service Categorical dichotomized. 1: Maternal-child (reference), 2: Critical care, 3: Medical, 4: Surgical; 5: Complex surgeries Weigh (AP-DRG) Continuous Length of stay Continuous: Discharge date–Hospitalization date (days) Age Continuous: Hospitalization date–date of birth (years) Sex Dichotomous: 1 = Women, 0 = Men Elective admissions Dichotomous: 1 = Yes, 0 = No Exitus Dichotomous: 1 = Yes, 0 = No Internal transfer Dichotomous: 1 = Yes, 0 = No Hospitalization in critical care Dichotomous: 1 = Yes, 0 = No Medical DRG Dichotomous: 1 = Yes, 0 = No Readmission in 30 days from discharge Dichotomous: 1 = Yes, 0 = No Charlson comorbidity index (CCI) Categorical dichotomized. 1: CCI = 0 (reference); 2: CCI = 1, 2; 3: CCI > 2 Variables Characteristics Number of Hospitalizations (per patient) 2010–14 Continuous External injury causes Dichotomous: 1 = Yes, 0 = No (Main diagnostic with ICD-9-MC) Reintervention during episode Dichotomous: 1 = Yes, 0 = No Major diagnostic category Categorical dichotomized. MDS 1 (reference) to MDS 25 (AP-DRG) Type of discharge service Categorical dichotomized. 1: Maternal-child (reference), 2: Critical care, 3: Medical, 4: Surgical; 5: Complex surgeries Weigh (AP-DRG) Continuous Length of stay Continuous: Discharge date–Hospitalization date (days) Age Continuous: Hospitalization date–date of birth (years) Sex Dichotomous: 1 = Women, 0 = Men Elective admissions Dichotomous: 1 = Yes, 0 = No Exitus Dichotomous: 1 = Yes, 0 = No Internal transfer Dichotomous: 1 = Yes, 0 = No Hospitalization in critical care Dichotomous: 1 = Yes, 0 = No Medical DRG Dichotomous: 1 = Yes, 0 = No Readmission in 30 days from discharge Dichotomous: 1 = Yes, 0 = No Charlson comorbidity index (CCI) Categorical dichotomized. 1: CCI = 0 (reference); 2: CCI = 1, 2; 3: CCI > 2 The average number of codes adjusted by case mix and standard performance were also calculated. The number of codes case mix adjusted was defined as the average number of codes that each period would have had if the episodes of the whole sample (reference standard) were codified with the average number of codes that each AP-DRG presents in such period. Conversely, the number of codes standard performance adjusted was defined as the average number of codes that each period would have had if their episodes were codified with the average number of codes that each AP-DRG presents in the whole sample. The difference of the number of codes case mix adjusted minus the average number of codes of the whole sample and the difference of the number of codes standard performance adjusted minus the average number of codes of the period were obtained for each one of the periods analyzed. Data analysis A descriptive analysis was performed using mean, median and standard deviations for continuous variables and frequencies distribution for the categorical ones. Comparison among PRE-INT, POST-INT and TP was performed by one-way analysis of variance or Kruskal–Wallis test, as appropriate. Similarly, the difference between insourced and outsourced coding performance in the studied period was analyzed using Mann–Whitney U or Student’s t-test. The homogeneity of variance between groups was tested using Levene's test. Models of multiple linear regression, Poisson regression and negative binomial regression were specified to adjust by risk the number of codes per individual characteristics. The goodness of fit of each model was examined to select the most appropriate. In Poisson regression, overdispersion was analyzed and compared with negative binomial regression through the Chi-square test. Moreover, as Poisson and negative binomial regressions allow dependent variables to take null values and the underlying distributions may result to be biased; zero truncated models were used in both cases. In the final fit, robust standard errors were used for parameter estimations [14]. In all cases, P-values <0.05 were considered statistically significant. Data analysis was performed using SPSS v21 and STATA v12.0. Results Demographic and clinical characteristics of patients The total coded DS during the period studied were 214 648 (99.86%) of which 97 901 (45.63%) were women and the average age of the patients was 50.1 years. The complexity of the cases attended ranged from 1.99 (2011) to 2.08 (2014) according to the average weight AP-DRG, and 0.98 (2010)–1.17 (2013) according to the Charlson comorbidity index (CCI) [15, 16]. The most frequent pathologies, grouped into major diagnostic categories (MDC AP-DRG) were: pregnancy, birth and postpartum (12.83%), circulatory system diseases (12.71%) and digestive system diseases (12.27%). The length of the stay decreased steadily 0.26 days in average annually from 8.52 in 2010 to 7.26 in 2014, the 30-days readmissions rate increased from 10.0% in 2010 to 12.42% in 2014 and the crude mortality rate remained stable throughout the study period with an average value of 2.96%. Quality of discharge summaries The reviewers achieved agreement by consensus in the 97 DS of PRE-INT (three were not found) and 100 of POST-INT evaluated. Statistically significant differences (P < 0.001) were found between pre- and post-intervention DS in quality (mean [SD]; PRE-INT = 2.49 (2.48); POST-INT = 1.54 (2.23)), length (PRE-INTE = 510.24 [374.03]; POST-INT = 1057.58 [683.96]) and readability (PRE-INT = 25.46 [7.70]; POST-INT = 19.59 [8.74]) (Fig. 2). Figure 2 View largeDownload slide Differences in DS quality, readability and length between pre- and post-intervention periods. Figure 2 View largeDownload slide Differences in DS quality, readability and length between pre- and post-intervention periods. Discharge summaries coding The average number of codes per episode of hospitalization for the whole studied period (2010–14) was 9.99 [5.99]. However, statistically significant differences (P = 0.001) were found in raw data between the PRE-INT, TP and POST-INT, 9.48 [5.94], 9.91 [6.03] and 10.77 [5.97], respectively. However, the implementation of the EHR neither changed the average number of codes not found in the DS per episode of hospitalization (PRE-INT = 0.505; POST-INT = 0.510) (P = 0.98) nor the number of codes not registered in the MBDS per DS (PRE-INT = 0.56; POST-INT = 0.96); (P = 0.11). The multiple linear regression model for risk-adjustment number of codes showed an acceptable coefficient of determination (0.631; P < 0.001) with neither multicollinearity (the values of the variance inflation factor were <5 in all variables) nor autocorrelation (Durbin–Watson = 1.97), the hypothesis of normality was rejected by the Shapiro–Wilk test (P < 0.001) and the existence of heterocedasticity was detected by Breusch–Pagan/Cook–Weisberg test (P < 0.001). The zero truncated Poisson regression fit was significant according to the Chi-square test (P < 0.001), with pseudo-R2 = 0.2837. However, the existence of overdispersion was accepted because the alpha parameter negative binomial regression was significantly different from 0 according to Chi-square test (P < 0.001). The zero truncated negative binomial regression also provided a good fit (coefficient of determination = 0.99) between the medians of the quintiles of the number of codes and the mean of the estimated values for each quintile. Consequently, the difference in risk-adjusted number of codes was calculated from the estimates obtained with this model (Table 2) and the mean difference between POST-INT and PRE-INT were the only ones found to be statistically significant (P < 0.001) (Table 3). Table 2 Adjustment of the negative binomial regression model for the number of diagnosis and procedure codes (NC) per episode Predictors IRR SE z P CI Number of hospitalizations (per patient) 2010–14 1.004 0.000 18.870 0.000 1.003 1.004 External injury causes 1.189 0.003 68.650 0.000 1.184 1.195 Reintervention during period 1.081 0.010 8.290 0.000 1.062 1.102 Major diagnostic category  2 0.896 0.010 –9.450 0.000 0.876 0.917  3 0.875 0.006 –18.070 0.000 0.862 0.887  4 1.072 0.005 16.200 0.000 1.063 1.081  5 1.047 0.005 10.490 0.000 1.038 1.056  6 1.003 0.005 0.550 0.586 0.993 1.012  7 1.003 0.005 0.630 0.530 0.993 1.013  8 0.913 0.005 –15.740 0.000 0.902 0.923  9 0.887 0.006 –16.780 0.000 0.874 0.899  10 1.009 0.007 1.230 0.220 0.995 1.023  11 1.073 0.005 14.390 0.000 1.063 1.083  12 0.900 0.008 –12.370 0.000 0.885 0.915  13 0.839 0.007 –22.630 0.000 0.826 0.852  14 0.891 0.006 –18.450 0.000 0.881 0.902  15 1.092 0.011 8.770 0.000 1.070 1.113  16 0.972 0.008 –3.510 0.000 0.957 0.988  17 0.810 0.006 –29.410 0.000 0.799 0.821  18 1.102 0.007 15.520 0.000 1.088 1.115  19 1.057 0.008 7.710 0.000 1.043 1.073  20 1.114 0.018 6.650 0.000 1.079 1.150  21 0.894 0.009 –10.740 0.000 0.876 0.913  22 1.020 0.065 0.320 0.751 0.900 1.157  23 0.962 0.012 –3.020 0.003 0.938 0.987  24 1.105 0.014 7.660 0.000 1.077 1.134  25 1.728 0.031 30.410 0.000 1.668 1.790 Type of discharge service  2 1.159 0.008 22.040 0.000 1.144 1.174  3 1.182 0.006 31.530 0.000 1.170 1.194  4 0.707 0.004 –58.270 0.000 0.699 0.715  5 0.842 0.005 –30.170 0.000 0.833 0.852 Weight (AP-DRG) 1.017 0.000 37.320 0.000 1.016 1.018 Length of stay 1.008 0.000 31.330 0.000 1.007 1.008 Age 1.006 0.000 103.070 0.000 1.005 1.006 Sex 1.012 0.002 6.170 0.000 1.008 1.015 Elective admissions 0.886 0.002 –47.420 0.000 0.881 0.89 Exitus 1.000 0.005 0.100 0.922 0.991 1.010 Handoff during hospitalization admission 1.098 0.003 32.600 0.000 1.092 1.105 Hospitalization in critical care 1.103 0.004 30.080 0.000 1.096 1.110 Medical DRG 1.066 0.003 22.420 0.000 1.060 1.072 Readmission in 30 days from discharge 0.995 0.003 –1.720 0.086 0.990 1.001 Charlson comorbidity index (CCI)  2 1.174 0.003 72.900 0.000 1.169 1.180  3 1.310 0.003 100.980 0.000 1.303 1.317 Intercept 5.484 0.036 258.530 0.000 5.414 5.555 /lnalpha –3.198 0.019 –3.236 –3.161 Alpha 0.041 0.001 1.003 1.004 Pseudo-R2 = 0.1519 (P = 0.001) Predictors IRR SE z P CI Number of hospitalizations (per patient) 2010–14 1.004 0.000 18.870 0.000 1.003 1.004 External injury causes 1.189 0.003 68.650 0.000 1.184 1.195 Reintervention during period 1.081 0.010 8.290 0.000 1.062 1.102 Major diagnostic category  2 0.896 0.010 –9.450 0.000 0.876 0.917  3 0.875 0.006 –18.070 0.000 0.862 0.887  4 1.072 0.005 16.200 0.000 1.063 1.081  5 1.047 0.005 10.490 0.000 1.038 1.056  6 1.003 0.005 0.550 0.586 0.993 1.012  7 1.003 0.005 0.630 0.530 0.993 1.013  8 0.913 0.005 –15.740 0.000 0.902 0.923  9 0.887 0.006 –16.780 0.000 0.874 0.899  10 1.009 0.007 1.230 0.220 0.995 1.023  11 1.073 0.005 14.390 0.000 1.063 1.083  12 0.900 0.008 –12.370 0.000 0.885 0.915  13 0.839 0.007 –22.630 0.000 0.826 0.852  14 0.891 0.006 –18.450 0.000 0.881 0.902  15 1.092 0.011 8.770 0.000 1.070 1.113  16 0.972 0.008 –3.510 0.000 0.957 0.988  17 0.810 0.006 –29.410 0.000 0.799 0.821  18 1.102 0.007 15.520 0.000 1.088 1.115  19 1.057 0.008 7.710 0.000 1.043 1.073  20 1.114 0.018 6.650 0.000 1.079 1.150  21 0.894 0.009 –10.740 0.000 0.876 0.913  22 1.020 0.065 0.320 0.751 0.900 1.157  23 0.962 0.012 –3.020 0.003 0.938 0.987  24 1.105 0.014 7.660 0.000 1.077 1.134  25 1.728 0.031 30.410 0.000 1.668 1.790 Type of discharge service  2 1.159 0.008 22.040 0.000 1.144 1.174  3 1.182 0.006 31.530 0.000 1.170 1.194  4 0.707 0.004 –58.270 0.000 0.699 0.715  5 0.842 0.005 –30.170 0.000 0.833 0.852 Weight (AP-DRG) 1.017 0.000 37.320 0.000 1.016 1.018 Length of stay 1.008 0.000 31.330 0.000 1.007 1.008 Age 1.006 0.000 103.070 0.000 1.005 1.006 Sex 1.012 0.002 6.170 0.000 1.008 1.015 Elective admissions 0.886 0.002 –47.420 0.000 0.881 0.89 Exitus 1.000 0.005 0.100 0.922 0.991 1.010 Handoff during hospitalization admission 1.098 0.003 32.600 0.000 1.092 1.105 Hospitalization in critical care 1.103 0.004 30.080 0.000 1.096 1.110 Medical DRG 1.066 0.003 22.420 0.000 1.060 1.072 Readmission in 30 days from discharge 0.995 0.003 –1.720 0.086 0.990 1.001 Charlson comorbidity index (CCI)  2 1.174 0.003 72.900 0.000 1.169 1.180  3 1.310 0.003 100.980 0.000 1.303 1.317 Intercept 5.484 0.036 258.530 0.000 5.414 5.555 /lnalpha –3.198 0.019 –3.236 –3.161 Alpha 0.041 0.001 1.003 1.004 Pseudo-R2 = 0.1519 (P = 0.001) IRR, incidence rate ratio; SE, standard error (robust); CI, confidence interval for the mean (95%). Table 2 Adjustment of the negative binomial regression model for the number of diagnosis and procedure codes (NC) per episode Predictors IRR SE z P CI Number of hospitalizations (per patient) 2010–14 1.004 0.000 18.870 0.000 1.003 1.004 External injury causes 1.189 0.003 68.650 0.000 1.184 1.195 Reintervention during period 1.081 0.010 8.290 0.000 1.062 1.102 Major diagnostic category  2 0.896 0.010 –9.450 0.000 0.876 0.917  3 0.875 0.006 –18.070 0.000 0.862 0.887  4 1.072 0.005 16.200 0.000 1.063 1.081  5 1.047 0.005 10.490 0.000 1.038 1.056  6 1.003 0.005 0.550 0.586 0.993 1.012  7 1.003 0.005 0.630 0.530 0.993 1.013  8 0.913 0.005 –15.740 0.000 0.902 0.923  9 0.887 0.006 –16.780 0.000 0.874 0.899  10 1.009 0.007 1.230 0.220 0.995 1.023  11 1.073 0.005 14.390 0.000 1.063 1.083  12 0.900 0.008 –12.370 0.000 0.885 0.915  13 0.839 0.007 –22.630 0.000 0.826 0.852  14 0.891 0.006 –18.450 0.000 0.881 0.902  15 1.092 0.011 8.770 0.000 1.070 1.113  16 0.972 0.008 –3.510 0.000 0.957 0.988  17 0.810 0.006 –29.410 0.000 0.799 0.821  18 1.102 0.007 15.520 0.000 1.088 1.115  19 1.057 0.008 7.710 0.000 1.043 1.073  20 1.114 0.018 6.650 0.000 1.079 1.150  21 0.894 0.009 –10.740 0.000 0.876 0.913  22 1.020 0.065 0.320 0.751 0.900 1.157  23 0.962 0.012 –3.020 0.003 0.938 0.987  24 1.105 0.014 7.660 0.000 1.077 1.134  25 1.728 0.031 30.410 0.000 1.668 1.790 Type of discharge service  2 1.159 0.008 22.040 0.000 1.144 1.174  3 1.182 0.006 31.530 0.000 1.170 1.194  4 0.707 0.004 –58.270 0.000 0.699 0.715  5 0.842 0.005 –30.170 0.000 0.833 0.852 Weight (AP-DRG) 1.017 0.000 37.320 0.000 1.016 1.018 Length of stay 1.008 0.000 31.330 0.000 1.007 1.008 Age 1.006 0.000 103.070 0.000 1.005 1.006 Sex 1.012 0.002 6.170 0.000 1.008 1.015 Elective admissions 0.886 0.002 –47.420 0.000 0.881 0.89 Exitus 1.000 0.005 0.100 0.922 0.991 1.010 Handoff during hospitalization admission 1.098 0.003 32.600 0.000 1.092 1.105 Hospitalization in critical care 1.103 0.004 30.080 0.000 1.096 1.110 Medical DRG 1.066 0.003 22.420 0.000 1.060 1.072 Readmission in 30 days from discharge 0.995 0.003 –1.720 0.086 0.990 1.001 Charlson comorbidity index (CCI)  2 1.174 0.003 72.900 0.000 1.169 1.180  3 1.310 0.003 100.980 0.000 1.303 1.317 Intercept 5.484 0.036 258.530 0.000 5.414 5.555 /lnalpha –3.198 0.019 –3.236 –3.161 Alpha 0.041 0.001 1.003 1.004 Pseudo-R2 = 0.1519 (P = 0.001) Predictors IRR SE z P CI Number of hospitalizations (per patient) 2010–14 1.004 0.000 18.870 0.000 1.003 1.004 External injury causes 1.189 0.003 68.650 0.000 1.184 1.195 Reintervention during period 1.081 0.010 8.290 0.000 1.062 1.102 Major diagnostic category  2 0.896 0.010 –9.450 0.000 0.876 0.917  3 0.875 0.006 –18.070 0.000 0.862 0.887  4 1.072 0.005 16.200 0.000 1.063 1.081  5 1.047 0.005 10.490 0.000 1.038 1.056  6 1.003 0.005 0.550 0.586 0.993 1.012  7 1.003 0.005 0.630 0.530 0.993 1.013  8 0.913 0.005 –15.740 0.000 0.902 0.923  9 0.887 0.006 –16.780 0.000 0.874 0.899  10 1.009 0.007 1.230 0.220 0.995 1.023  11 1.073 0.005 14.390 0.000 1.063 1.083  12 0.900 0.008 –12.370 0.000 0.885 0.915  13 0.839 0.007 –22.630 0.000 0.826 0.852  14 0.891 0.006 –18.450 0.000 0.881 0.902  15 1.092 0.011 8.770 0.000 1.070 1.113  16 0.972 0.008 –3.510 0.000 0.957 0.988  17 0.810 0.006 –29.410 0.000 0.799 0.821  18 1.102 0.007 15.520 0.000 1.088 1.115  19 1.057 0.008 7.710 0.000 1.043 1.073  20 1.114 0.018 6.650 0.000 1.079 1.150  21 0.894 0.009 –10.740 0.000 0.876 0.913  22 1.020 0.065 0.320 0.751 0.900 1.157  23 0.962 0.012 –3.020 0.003 0.938 0.987  24 1.105 0.014 7.660 0.000 1.077 1.134  25 1.728 0.031 30.410 0.000 1.668 1.790 Type of discharge service  2 1.159 0.008 22.040 0.000 1.144 1.174  3 1.182 0.006 31.530 0.000 1.170 1.194  4 0.707 0.004 –58.270 0.000 0.699 0.715  5 0.842 0.005 –30.170 0.000 0.833 0.852 Weight (AP-DRG) 1.017 0.000 37.320 0.000 1.016 1.018 Length of stay 1.008 0.000 31.330 0.000 1.007 1.008 Age 1.006 0.000 103.070 0.000 1.005 1.006 Sex 1.012 0.002 6.170 0.000 1.008 1.015 Elective admissions 0.886 0.002 –47.420 0.000 0.881 0.89 Exitus 1.000 0.005 0.100 0.922 0.991 1.010 Handoff during hospitalization admission 1.098 0.003 32.600 0.000 1.092 1.105 Hospitalization in critical care 1.103 0.004 30.080 0.000 1.096 1.110 Medical DRG 1.066 0.003 22.420 0.000 1.060 1.072 Readmission in 30 days from discharge 0.995 0.003 –1.720 0.086 0.990 1.001 Charlson comorbidity index (CCI)  2 1.174 0.003 72.900 0.000 1.169 1.180  3 1.310 0.003 100.980 0.000 1.303 1.317 Intercept 5.484 0.036 258.530 0.000 5.414 5.555 /lnalpha –3.198 0.019 –3.236 –3.161 Alpha 0.041 0.001 1.003 1.004 Pseudo-R2 = 0.1519 (P = 0.001) IRR, incidence rate ratio; SE, standard error (robust); CI, confidence interval for the mean (95%). Table 3 Differences in diagnosis and procedure codes per episode adjusted for risk (zero truncated negative binomial regression) and multiple comparisons between periods Period N Mean SD SE CI Period II Mean dif P PRE-INT 42 182 –0.418 50.077 0.244 –0.896 –0.896 PT –0.30 0.461 POST-INT –0.74 0.001 TP 42 126 –0.118 4.303 0.021 –0.159 –0.077 PRE-INT 0.30 0.461 POST-INT –0.44 0.108 POST-INT 43 928 0.319 17.194 0.082 0.158 0.480 PRE-INT –0.74 0.001 PT 0.44 0.108 Total 128 236 –0.067 30.534 0.085 –0.234 –0.1 Period N Mean SD SE CI Period II Mean dif P PRE-INT 42 182 –0.418 50.077 0.244 –0.896 –0.896 PT –0.30 0.461 POST-INT –0.74 0.001 TP 42 126 –0.118 4.303 0.021 –0.159 –0.077 PRE-INT 0.30 0.461 POST-INT –0.44 0.108 POST-INT 43 928 0.319 17.194 0.082 0.158 0.480 PRE-INT –0.74 0.001 PT 0.44 0.108 Total 128 236 –0.067 30.534 0.085 –0.234 –0.1 N, number of episodes; SD, standard deviation; SE, standard error; CI, confidence Interval for the average value (95%); Mean dif, mean differences; PRE-INT, pre-intervention period; TP, transition period; POST-INT, post implementation period. F = 524.948 (P = 0.001). Table 3 Differences in diagnosis and procedure codes per episode adjusted for risk (zero truncated negative binomial regression) and multiple comparisons between periods Period N Mean SD SE CI Period II Mean dif P PRE-INT 42 182 –0.418 50.077 0.244 –0.896 –0.896 PT –0.30 0.461 POST-INT –0.74 0.001 TP 42 126 –0.118 4.303 0.021 –0.159 –0.077 PRE-INT 0.30 0.461 POST-INT –0.44 0.108 POST-INT 43 928 0.319 17.194 0.082 0.158 0.480 PRE-INT –0.74 0.001 PT 0.44 0.108 Total 128 236 –0.067 30.534 0.085 –0.234 –0.1 Period N Mean SD SE CI Period II Mean dif P PRE-INT 42 182 –0.418 50.077 0.244 –0.896 –0.896 PT –0.30 0.461 POST-INT –0.74 0.001 TP 42 126 –0.118 4.303 0.021 –0.159 –0.077 PRE-INT 0.30 0.461 POST-INT –0.44 0.108 POST-INT 43 928 0.319 17.194 0.082 0.158 0.480 PRE-INT –0.74 0.001 PT 0.44 0.108 Total 128 236 –0.067 30.534 0.085 –0.234 –0.1 N, number of episodes; SD, standard deviation; SE, standard error; CI, confidence Interval for the average value (95%); Mean dif, mean differences; PRE-INT, pre-intervention period; TP, transition period; POST-INT, post implementation period. F = 524.948 (P = 0.001). The binomial negative regression showed (Table 2) statistically significant higher incidence rates ratio (IRR > 1) in diagnosis and procedure codes in discharges from women, hospitalization in critical care, external injury causes, reintervention during period, handoff during hospitalization admission, high AP-DRG weight and MDC 4, 5, 11, 15, 18, 19, 20, 24, 25 and lower incidence (IRR < 1) in episodes with elective admissions or readmissions, while in exitus and in MDC 6, 7, 10, 16, 17 differences were not statistically significant (P > 0.05). In the outsourcing period, the average number of codes per episode coded by insourced coders (n = 12.253) was higher than those coded by outsourced coders (n = 8.359) and the difference was statistically significant (P < 0.001). After risk-adjustment, the difference in average number of risk-adjusted codes was also significantly higher in episodes coded by insourced coders (0.035 vs. −0.142; P < 0.001). Finally, differences in number of codes adjusted by either case mix and or standard performance were higher in the POST-INT than in TP as well as higher than in PRE-INT (Table 4). Therefore, the observed difference in the average number of codes is unrelated to changes in casuistry or complexity in the studied periods. Table 4 Differences in diagnosis and procedure codes per hospitalization episode adjusted by case mix and standard performance PRE-INT TP POST-INT Mean NC (population study) 9.48 9.91 10.77 Mean (CI) NC (sample for evaluation of DS) 9.14 (8.64–9.64) 10.33 (9.83–10.83) NCCMA 9.80 9.91 10.44 NCSPA 9.68 9.98 10.32 DNCCMA −0.51 −0.08 0.78 DNCSPA −0.20 −0.07 0.45 PRE-INT TP POST-INT Mean NC (population study) 9.48 9.91 10.77 Mean (CI) NC (sample for evaluation of DS) 9.14 (8.64–9.64) 10.33 (9.83–10.83) NCCMA 9.80 9.91 10.44 NCSPA 9.68 9.98 10.32 DNCCMA −0.51 −0.08 0.78 DNCSPA −0.20 −0.07 0.45 PRE-INT, pre-intervention period; TP, transition period; POST-INT, post-intervention period; CI, confidence interval (95%); NC, number of diagnosis and procedure codes; NCCMA, NC mean (case mix adjusted); NCSPA, mean NC (standard performance adjusted); DNCCMA, difference adjusted by case mix; DNCSPA, difference adjusted by standard performance. Table 4 Differences in diagnosis and procedure codes per hospitalization episode adjusted by case mix and standard performance PRE-INT TP POST-INT Mean NC (population study) 9.48 9.91 10.77 Mean (CI) NC (sample for evaluation of DS) 9.14 (8.64–9.64) 10.33 (9.83–10.83) NCCMA 9.80 9.91 10.44 NCSPA 9.68 9.98 10.32 DNCCMA −0.51 −0.08 0.78 DNCSPA −0.20 −0.07 0.45 PRE-INT TP POST-INT Mean NC (population study) 9.48 9.91 10.77 Mean (CI) NC (sample for evaluation of DS) 9.14 (8.64–9.64) 10.33 (9.83–10.83) NCCMA 9.80 9.91 10.44 NCSPA 9.68 9.98 10.32 DNCCMA −0.51 −0.08 0.78 DNCSPA −0.20 −0.07 0.45 PRE-INT, pre-intervention period; TP, transition period; POST-INT, post-intervention period; CI, confidence interval (95%); NC, number of diagnosis and procedure codes; NCCMA, NC mean (case mix adjusted); NCSPA, mean NC (standard performance adjusted); DNCCMA, difference adjusted by case mix; DNCSPA, difference adjusted by standard performance. Discussion Health systems need reliable information for multiple purposes. The essential is undoubtedly the use of clinical information to provide the best possible healthcare. The relationship between standardization of care documentation and records with the improvement of clinical practices has been proved [17] as well as the relationship between amount and quality of information on patient care results [18]. Administrative data, usually recorded by governmental initiatives [19], manage relevant information on services utilization [20], health outcomes [21], healthcare quality [22, 23] and their costs [24] for planning, supporting decisions and evaluating the effectiveness of policies. The implementation of EHR has potential to greatly improve patient safety, efficiency and effectiveness of healthcare. Although it has raised some concerns about data quality and its implications [25], it seems to provide more complete and detailed clinical documentation [6]. Moreover, the DS, an essential element of the EHR, benefits from a standard format that uses the information previously recorded in templates and progress notes to compose the final document at the time of patient discharge. Our results show that after implementation of EHR the DS had higher quality but were longer and harder to read than previously. The quality improvement is consistent with the literature. Several authors had evaluated the impact of similar interventions. Thus, O’Leary et al. [11] concluded that the use of a composition tool in the EHR improved the quality and readability of DS and Reinke et al. [12] showed that the intervention also improved the quality of DS; but they were shorter and harder to read. In our view, the improvement in quality results from both the widespread use of EHR structured templates (easier data recording) and the simplicity of the content importing technologies (data transfer to DS). The results variability in terms of length and readability may depend on the proper use of the EHR. Thus, it can be partially explained by the routine inclusion of diagnostic tests results, particularly lab tests, which take up a significant space with little real useful information and complicate the reading. It can be also be due to the improper use of functionalities, such as the ‘copy and paste’ in the composition of DS [26], particularly in the patients personal and family history. The implementation of EHR in our hospital did not change the proportion of diagnoses and procedures coded in the DS to the total registered in the MBDS. This fact suggests that the nature and amount of information available to the physician at the time of discharge did not change with the EHR implementation. However, the number of diagnoses and procedures coded per DS has significantly increased with EHR, using both raw and risk-adjusted data. This effect was not observed during the TP, but it might be related to the lower performance of the outsourced coding. Thus, our data suggest that the implementation of EHR improves availability of clinical information for the DS and coding performance. The importance of this finding hinges on the utility of the administrative databases such as the MBDS in any of the many aspects related to healthcare. The appropriateness of the use of administrative databases in health research projects has been debated. However, its availability and low cost makes them attractive tools for healthcare research. Thus, validation studies of results obtained from administrative data versus clinical records [27] or clinical chart reviews [28] had spurred. On one hand, inaccuracies have been described in the validation of certain diseases and comorbidities [29]; on the other hand, it has been concluded that there are no significant differences between risk-adjusted mortality and readmissions for example in acute myocardial infarction or heart failure [30]. Our study supports the notion that adoption of EHR improves MBDS reliability, and therefore its utility for secondary uses. Limitations Our study has some limitations. It is a pre–post-intervention design without control group (the analyzed DS are done either in one period or another) and the observed effect is attributed to the intervention but it does not dismiss any other associated factors. To control for this, we have adjusted by risk the dependent variable by different methods, analyzed if there were changes in the proportion of diagnoses and procedures present in the DS that were not coded, and therefore not recorded in the MBDS; and assessed the differences between coders (insourced and outsourced) in the period when the codification was partially outsourced. However, given that during the pre- and post-intervention periods no outsourcing took place, we understand that the results, especially risk-adjusted differences in favor of the group of H12O coders, indicate the coding pattern has been stable. Consequently, we can exclude the one factor that in our opinion could be the main cause of confusion. Another limitation was inherent to the format of the DS since the intervention changed the DS aspect the reviewers could easily identify the period of the reports evaluated. Finally, the quality evaluation method is internal to the H12O and has not been externally validated, although we have followed the criteria specified in our country regulations. Regarding the quality of the DS and the availability of clinical documentation for secondary uses, future research on the impact of EHR implementation is needed to deep our understanding. It would be advisable to study the external validity in other healthcare institutions, including, if possible, studies with a control group. It is as well interesting to ascertain whether the use of ‘copy and paste’ tools available in the EHR affects the understandability of the DS to the extent to compromise patient safety. Thus, it would be necessary to investigate ways to improve the transmission of clinical information between healthcare settings, patients and professionals. Conclusions The implementation of EHR improves the quality of the DS composed from clinical information recorded during the episode of hospitalization. The improper use of the content importing technologies and inclusion of non-relevant content can make them more extensive and less legible. The increased availability of clinical information in DS allows the codification of greater number of diagnoses and procedures and, ultimately, improves utility of MBDS for secondary use. Acknowledgements The authors have performed this work with the support of the Coding Unit and Management Control Service of the Hospital Universitario 12 de Octubre. It is kindly appreciated the dedication and effort of their staff. Funding There were no funding sources that contributed to this study. References 1 Wilson S , Ruscoe W , Chapman M et al. . General practitioner–hospital communications: a review of discharge summaries . J Qual Clin Pract 2001 ; 21 : 104 – 8 . http://www.ncbi.nlm.nih.gov/pubmed/11856404. Google Scholar Crossref Search ADS PubMed 2 Buurman BM , Verhaegh KJ , Smeulers M et al. . Improving handoff communication from hospital to home: the development, implementation and evaluation of a personalized patient discharge letter . Int J Qual Health Care 2016 ; 28 : 384 – 90 . http://www.ncbi.nlm.nih.gov/pubmed/27224995. Google Scholar Crossref Search ADS PubMed 3 Callen JL , Alderton M , McIntosh J . Evaluation of electronic discharge summaries: a comparison of documentation in electronic and handwritten discharge summaries . Int J Med Inform 2008 ; 77 : 613 – 20 . http://www.ncbi.nlm.nih.gov/pubmed/18294904. Google Scholar Crossref Search ADS PubMed 4 Gagnier JJ , Derosier JM , Maratt JD et al. . Development, implementation and evaluation of a patient handoff tool to improve safety in orthopaedic surgery . Int J Qual Health Care 2016 ; 28 : 363 – 70 . http://www.ncbi.nlm.nih.gov/pubmed/27090398. Google Scholar Crossref Search ADS PubMed 5 Ministerio de Sanidad y Consumo . Metodología de análisis de la hospitalización en el Sistema Nacional de Salud. Modelo de indicadores basado en el Registro de Altas (CMBD). 2007 ;126. http://www.msssi.gob.es/estadEstudios/estadisticas/docs/metod_modelo_cmbd_pub.pdf 6 Häyrinen K , Saranto K , Nykänen P . Definition, structure, content, use and impacts of electronic health records: a review of the research literature . Int J Med Inform 2008 ; 77 : 291 – 304 . http://www.ncbi.nlm.nih.gov/pubmed/17951106. Google Scholar Crossref Search ADS PubMed 7 Marca G , Perez AJ , Blanco-Garcia MG et al. . The use of electronic health records in Spanish hospitals . Health Inf Manag 2014 ; 43 : 37 – 44 . http://www.ncbi.nlm.nih.gov/pubmed/24682737. Google Scholar PubMed 8 Reid B , Kelly Z , Westbrook J . A preliminary analysis of differences in coded data from Australia and Maryland . Health Inf Manag 2002 ; 30 : 9 – 20 . http://www.ncbi.nlm.nih.gov/pubmed/19468136. Google Scholar PubMed 9 Walker RL , Hennessy DA , Johansen H et al. . Implementation of ICD-10 in Canada: how has it impacted coded hospital discharge data? BMC Health Serv Res 2012 ; 12 : 149 . http://www.ncbi.nlm.nih.gov/pubmed/22682405. Google Scholar Crossref Search ADS PubMed 10 Burke HB , Sessums LL , Hoang A et al. . Electronic health records improve clinical note quality . J Am Med Inform Assoc 2015 ; 22 : 199 – 205 . http://www.ncbi.nlm.nih.gov/pubmed/25342178. Google Scholar PubMed 11 O’Leary KJ , Liebovitz DM , Feinglass J et al. . Creating a better discharge summary: improvement in quality and timeliness using an electronic discharge summary . J Hosp Med 2009 ; 4 : 219 – 25 . http://www.ncbi.nlm.nih.gov/pubmed/19267397. Google Scholar Crossref Search ADS PubMed 12 Reinke CE , Kelz RR , Baillie CA et al. . Timeliness and quality of surgical discharge summaries after the implementation of an electronic format . Am J Surg 2014 ; 207 : 7 – 16 . http://www.ncbi.nlm.nih.gov/pubmed/24269034. Google Scholar Crossref Search ADS PubMed 13 Lorda PS , Barrio Cantalejo IM , Concheiro Carro L . Legibilidad de los formularios escritos de consentimiento informado . Med Clin (Barc) 2016 ; 107 : 524 – 9 . http://cat.inist.fr/?aModele=afficheN&cpsidt=11023799. 14 Cameron AC , Trivedi PK . Stata Version 11 and / Microeconometrics using Stata. 2010 ;4–7. 15 Charlson ME , Pompei P , Ales KL et al. . A new method of classifying prognostic comorbidity in longitudinal studies: development and validation . J Chronic Dis 1987 ; 40 : 373 – 83 . http://www.ncbi.nlm.nih.gov/pubmed/3558716. Google Scholar Crossref Search ADS PubMed 16 Deyo RA , Cherkin DC , Ciol MA . Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases . J Clin Epidemiol 1992 ; 45 : 613 – 9 . http://www.ncbi.nlm.nih.gov/pubmed/1607900. Google Scholar Crossref Search ADS PubMed 17 Wright P , Jansen C , Wyatt JC . How to limit clinical errors in interpretation of data . Lancet 1998 ; 352 : 1539 – 43 . http://www.ncbi.nlm.nih.gov/pubmed/9820319. Google Scholar Crossref Search ADS PubMed 18 Dunlay SM , Alexander KP , Melloni C et al. . Medical records and quality of care in acute coronary syndromes: results from CRUSADE . Arch Intern Med 2008 ; 168 : 1692 – 8 . http://www.ncbi.nlm.nih.gov/pubmed/18695085. Google Scholar Crossref Search ADS PubMed 19 Chowdhury TT , Hemmelgarn B . Evidence-based decision-making 6: utilization of administrative databases for health services research . Methods Mol Biol 2015 ; 1281 : 469 – 84 . http://www.ncbi.nlm.nih.gov/pubmed/25694328. Google Scholar Crossref Search ADS PubMed 20 Ruiz M , Bottle A , Aylin PP . The Global Comparators project: international comparison of 30-day in-hospital mortality by day of the week . BMJ Qual Saf 2015 ; 24 : 492 – 504 . http://www.ncbi.nlm.nih.gov/pubmed/26150550. Google Scholar Crossref Search ADS PubMed 21 Nuti SV , Wang Y , Masoudi FA et al. . Improvements in the distribution of hospital performance for the care of patients with acute myocardial infarction, heart failure, and pneumonia, 2006–2011 . Med Care 2015 ; 53 : 485 – 91 . http://www.ncbi.nlm.nih.gov/pubmed/25906012. Google Scholar Crossref Search ADS PubMed 22 Iezzoni LI . Assessing quality using administrative data . Ann Intern Med 1997 ; 127 : 666 – 74 . http://www.ncbi.nlm.nih.gov/pubmed/9382378. Google Scholar Crossref Search ADS PubMed 23 Knight HE , Gurol-Urganci I , Mahmood TA et al. . Evaluating maternity care using national administrative health datasets: how are statistics affected by the quality of data on method of delivery? BMC Health Serv Res 2013 ; 13 : 200 . http://www.ncbi.nlm.nih.gov/pubmed/23721128. Google Scholar Crossref Search ADS PubMed 24 Riley GF . Administrative and claims records as sources of health care cost data . Med Care 2009 ; 47 : S51 – 5 . http://www.ncbi.nlm.nih.gov/pubmed/19536019. Google Scholar Crossref Search ADS PubMed 25 Callen J . What is the impact of electronic health records on the quality of health data? Heal Inf Manag J 2014 ; 43 : 42 – 3 . 26 Hirschtick RE . A piece of my mind. Copy-and-paste . JAMA 2006 ; 295 : 2335 – 6 . http://www.ncbi.nlm.nih.gov/pubmed/16720812. Google Scholar Crossref Search ADS PubMed 27 Keenan PS , Normand S-LT , Lin Z et al. . An administrative claims measure suitable for profiling hospital performance on the basis of 30-day all-cause readmission rates among patients with heart failure . Circ Cardiovasc Qual Outcomes 2008 ; 1 : 29 – 37 . http://www.ncbi.nlm.nih.gov/pubmed/20031785. Google Scholar Crossref Search ADS PubMed 28 Horwitz LI , Grady JN , Cohen DB et al. . Development and validation of an algorithm to identify planned readmissions from claims data . J Hosp Med 2015 ; 10 : 670 – 7 . http://www.ncbi.nlm.nih.gov/pubmed/26149225. Google Scholar Crossref Search ADS PubMed 29 Hannan EL , Kilburn H , Lindsey ML et al. . Clinical versus administrative data bases for CABG surgery. Does it matter? Med Care 1992 ; 30 : 892 – 907 . http://www.ncbi.nlm.nih.gov/pubmed/1405795. Google Scholar Crossref Search ADS PubMed 30 Krumholz HM , Wang Y , Mattera JA et al. . An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with an acute myocardial infarction . Circulation 2006 ; 113 : 1683 – 92 . http://www.ncbi.nlm.nih.gov/pubmed/16549637. Google Scholar Crossref Search ADS PubMed © The Author(s) 2018. Published by Oxford University Press in association with the International Society for Quality in Health Care. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

Journal

International Journal for Quality in Health CareOxford University Press

Published: Oct 1, 2018

References

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off