Development and External Validation of Risk Scores for Cardiovascular Hospitalization and Rehospitalization in Patients With Diabetes

Development and External Validation of Risk Scores for Cardiovascular Hospitalization and... Abstract Context Cardiovascular disease (CVD) is a common and costly reason for hospitalization and rehospitalization among patients with type 2 diabetes. Objective This study aimed to develop and externally validate two risk-prediction models for cardiovascular hospitalization and cardiovascular rehospitalization. Design Two independent prospective cohorts. Setting The derivation cohort includes 4704 patients with type 2 diabetes from 18 general practices in Cambridgeshire. The validation cohort comprises 1121 patients with type 2 diabetes from post-trial follow-up data. Main Outcome Measure Cardiovascular hospitalization over 2 years and cardiovascular rehospitalization after 90 days of the prior CVD hospitalization. Results The absolute rate of cardiovascular hospitalization and rehospitalization was 12.5% and 6.7% in the derivation cohort and 16.3% and 7.0% in the validation cohort. Discrimination of the models was similar in both cohorts, with C statistics above 0.70 and excellent calibration of observed and predicted risks. Conclusion Two prediction models that quantify risks of cardiovascular hospitalization and rehospitalization have been developed and externally validated. They are based on a small number of clinical measurements that are available for patients with type 2 diabetes in many developed countries in primary care settings and could serve as the tools to screen the population at high risk of cardiovascular hospitalization and rehospitalization. The prevalence and cost of diabetes are growing rapidly worldwide (1). People with diabetes are twice as likely to be admitted to a hospital, and at least 10% of those in a hospital have diabetes at any one time (2). In some age groups, it is as many as one in five (3). The associated costs of excess admissions, as well as increased costs per admission, are important contributors to the financial burden borne by health care systems from diabetes and often reflect preventable morbidity suffered by patients (4). Previously, two prediction tools have been developed, both based on secondary care data, to identify those with diabetes, at high risk of either all-cause excessive length of stay or all-cause inpatient mortality over 4 years (5) or all-cause readmission within 30 days among hospitalized patients (6). However, the practical application of both prediction models was limited by a lack of external validation, nonspecificity for people with type 2 diabetes, the use of predictors derived from secondary care rather than primary care data, variations on predictors recorded in different datasets (e.g., comorbidity), and a relatively short time gap between baseline and outcome (readmission within 30 days). Among hospital admissions, cardiovascular events are the major cause for hospitalization in people with type 2 diabetes (7). Although risk factors, such as blood pressure and hemoglobin A1c (HbA1c), are recognized as warranting intervention on their own (8, 9), there has been no current algorithm to estimate the absolute risk of cardiovascular hospitalization and rehospitalization in people with type 2 diabetes. The use of a model to make predictions for individual patients with type 2 diabetes is more comprehensive than the use of individual risk factors and is preferred to the risk-grouping approach (10, 11). The aim of our study was to develop and externally validate different prediction models based on reliable clinical measurements in primary care settings for cardiovascular hospitalization over the next 2 years and cardiovascular rehospitalization up to 90 days following a prior cardiovascular hospitalization. Materials and Methods Data source and study population We used two cohorts from Cambridgeshire, United Kingdom: one (Derivation) based on the electronic health record data from primary care settings to develop our cardiovascular hospitalization and rehospitalization risk scores and another (Validation) based on post-trial cohort data for external validation. Derivation cohort Patient lists from 18 general practices across Cambridgeshire in 2008/2009 were collated and linked with hospital admissions (Secondary Uses Service) data, as part of an evaluation of diabetes care across the county by the local health board, National Health Service (NHS) Cambridgeshire. This cohort was limited to volunteer practices using the Egton Medical Information Systems general practitioner software system, from which a predefined set of data could be extracted. There was no systematic selection process for these surgeries, and data extracted were for their entire diabetes population. All patients with diabetes had follow-up hospitalization data from 2010 to 2011. Hospital admissions to NHS and private hospitals within and outside Cambridgeshire were followed up. No personal identifiers were released to researchers, and all subsequent analyses were conducted on anonymous datasets. Validation cohort The design and methods of the Randomized Controlled Trial of Peer Support in Type 2 Diabetes trial have been published previously (12), as have its Consolidated Standards of Reporting Trials diagram and the results of its primary outcomes (12). In brief, Randomized Controlled Trial of Peer Support in Type 2 Diabetes was a 2 × 2 factorial cluster randomized controlled trial comparing four groups: controls, 1:1 (individual) peer support, group peer support, and combined 1:1 and group peer support among patients with type 2 diabetes. Participants had their diabetes for at least 12 months, and those with dementia or psychotic illness were excluded. Participants were recruited from communities across Cambridgeshire and neighboring areas of Essex and Hertfordshire. Follow-up data were only available for participants in Cambridgeshire and neighboring areas of Hertfordshire that are served by the Cambridgeshire and Peterborough Clinical Commissioning Group. Clusters were defined by local government (“parish council”) boundaries. The intervention was developed following a pilot (13), using a framework defined by Peers for Progress (14). Peers facilitating peer support were termed “peer support facilitators,” and their selection, training, support, and the overall program are described elsewhere (15). The intervention lasted 8 to 12 months and was commenced and concluded, cluster by cluster, between 6 February 2011 and 4 December 2012. Ethics approval was received from the Cambridgeshire REC2 Committee (10/H0308/72), and signed consent included agreement for access to hospital data. At baseline, demographic data, blood pressure, and HbA1c and lipid profiles were collected. Each participant was followed up until June 2015 (0.91 to 4.07 years’ follow-up from beginning/entry into the trial). Hospitalization (NHS hospitals and private hospitals), Accident and Emergency, and outpatient visits within/outside Cambridgeshire and the included areas of Hertfordshire were completely collected through Cambridgeshire and Peterborough Clinical Commissioning Group (16) and the elective/nonelective status and International Classification of Diseases (ICD-10) codes (8). Defining cardiovascular hospitalization and rehospitalization The primary outcome of the study was having at least one hospitalization with cardiovascular disease (CVD) as the primary diagnosis (ICD-10: I20–I25, I60–I69, and I73 in the first ICD field) over the 2-year follow-up and having at least one CVD rehospitalization after 90 days of prior CVD hospitalization. Candidate predictors, missing data, and power calculations To achieve the maximum extrapolation application of our risk algorithm, objective clinical measurements were used as predictors in the model, including body mass index (BMI), blood pressure [systolic (SBP) and diastolic (DBP)], and the metabolic variables, glycated hemoglobin (HbA1c) and lipid profiles. We also included demographic characteristics (age and sex) and whether the patient was on lipid-lowering treatment. Patients with diabetes were invited to have their blood pressure and metabolic variables measured at least once a year after the diagnosis of diabetes, and the most recent was taken before 1 April 2009 (a minimum of 50 days before the first admission). Diabetes duration was not universally recorded and hence, was not usefully available for analysis. Diabetes therapy was not included in the dataset. Lipid-lowering treatment was recorded. Our derivation cohort had missing information on BMI (3.17%), SBP (9.95%), DBP (9.95%), total cholesterol (12.35%), high-density lipoprotein (HDL; 14.56%), and low-density lipoprotein (LDL; 16.27%). We used multiple imputation to replace missing values by using a chained equation approach based on all candidate predictors and outcomes. We created 16 imputed datasets for missing variables that were then combined across all datasets by using Rubin’s rule (17) to obtain final model estimates. Limited information was missing (<1%) in our external validation dataset, and the complete dataset was used in our analysis. On the basis of an estimated 588 cardiovascular hospitalizations and 316 cardiovascular rehospitalizations and 16 predictors or levels in our derivation cohort, we had an effective sample size of 37 cardiovascular hospitalization and 21 cardiovascular rehospitalization per predictor or level, above the minimum requirement suggested by Peduzzi et al. (18). Ethical approval The derivation cohort work had approval from the Cambridgeshire Research Ethics Committee as part of a wider service evaluation. Ethics approval for the validation cohort was received from the Cambridgeshire REC2 Committee (10/H0308/72), and signed consent included agreement for access to hospital data. Statistical analysis for model derivation and external validation We treated incidence occurrence of cardiovascular hospitalization after the first 90 days since the start of follow-up and the incident occurrence of cardiovascular rehospitalization as binary outcome measures. For each of the 15 candidate predictors or levels, we used a univariate logistic regression model to calculate the unadjusted odds ratios. For derivation of the risk-prediction model, we initially included all candidate predictors in a multivariable logistic regression model. We used fractional polynomials to model potential nonlinear relationships between continuous predictors and outcome. Through backward elimination, we excluded lower lipid treatment from the multivariate model, as it was not statistically significant (P > 0.1, based on change in log likelihood). After elimination, we reinserted the excluded predictor into the final model to check further whether it became statistically significant. We also rechecked fractional polynomial terms at this stage and re-estimated them if necessary. We formed the risk equations for predicting the log odds of cardiovascular hospitalization and cardiovascular rehospitalization by using the estimated regression coefficients multiplied by the corresponding predictors included in our models, together with the intercepts. This process ultimately led to equations for the predicted risk = 1/(1 + e−risk score), whether the “risk score” is the predicted log odds of cardiovascular hospitalization or cardiovascular rehospitalization from the developed models. To facilitate model use in clinical practice, the logistic regression equations were transformed into prognostic score charts. The coefficients in the logistic regression equation were multiplied by 50 and rounded to the nearest integer to obtain the prognostic score per predictor. Multiplication by 50 was chosen to get the majority of the coefficients close to an integer, thereby minimizing the effects of rounding. The sum of all prognostic scores reflects patients’ probability of cardiovascular hospitalization or cardiovascular rehospitalization. We assessed the performance of the models in terms of the C statistic and calibration slope (where 1.00 is ideal). The C statistic represents the probability that for any randomly selected pair of people with type 2 diabetes, with and without outcomes, the patient with outcomes had a higher predicted risk (19). A value of 0.50 indicated no discrimination, and 1.00 represents perfect discrimination. We then undertook internal validation to correct measures of predictive performance for optimism (overfitting) by bootstrapping 100 samples of the derivation data. We repeated the model derivation process in each bootstrap sample to produce a model, applied the model to the same bootstrap sample to quantify apparent performance, and applied the model to the original dataset to test model performance (calibration slope and C statistics) and optimism (difference in the test performance and apparent performance). We then estimated the overall optimism across all models. We applied our risk-prediction model to each patient with type 2 diabetes in the external validation cohort on the basis of the presence of one or more predictors. We examined the performance of this final model, both in the derivation dataset and then in the external validation dataset in terms of discrimination by calculating the C statistics. We examined calibration by plotting agreement between predicted and observed risks across one-tenth of the predicted risks. We used Stata V14.0 for all statistical analyses. This study was conducted and reported in line with the Transparent Reporting of a Multivariate Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guidelines (20). Role of the funding source The sponsors of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. Results Study participants In our derivation cohort, we analyzed information on 4704 patients with type 2 diabetes with 588 cardiovascular hospitalizations within 2 years and 316 rehospitalizations after 90 days since a prior cardiovascular hospitalization. Our validated cohort had information on 1121 patients with type 2 diabetes with 183 cardiovascular hospitalizations and 78 rehospitalizations. Table 1 summarizes the basic characteristics and potential predictors of the study population. Patients with type 2 diabetes in both cohorts had similar age, sex, blood pressure, and total cholesterol. Patients in the derived cohort had a higher level of HDL, LDL, and HbA1c. Compared with the derivation cohort, those in the validation cohort were more likely to be prescribed lowering lipid medicine and had more cardiovascular hospitalization and rehospitalization. Table 1. Baseline Characteristics of Study Populations   Derivation Cohort  External Validation Cohort  n  4704  1121  Cardiovascular hospitalization, n (%)  588 (12.5)  183 (16.3)  Cardiovascular rehospitalization, n (%)  316 (6.7)  78 (7.0)  Age, y  65.0 ± 16.3  65.5 ± 11.4  Female, n (%)  1919 (40.8)  444 (39.6)  SBP, mm Hg  134.5 ± 16.0  139.7 ± 20.2  DBP, mm Hg  76.3 ± 10.0  75.5 ± 11.5  Total cholesterol, mmol/L  4.3 ± 1.2  4.2 ± 1.7  HDL, mmol/L  1.3 ± 0.6  1.1 ± 1.2  LDL, mmol/L  2.5 ± 1.4  1.4 ± 3.0  BMI, kg/m2  30.8 ± 6.9  32.2 ± 6.0  HbA1c, mmol/mol  61.5 ± 17.2  56.2 ± 15.1  Lipid-lowering treatment, n (%)  3342 (71.4)  731 (65.2)    Derivation Cohort  External Validation Cohort  n  4704  1121  Cardiovascular hospitalization, n (%)  588 (12.5)  183 (16.3)  Cardiovascular rehospitalization, n (%)  316 (6.7)  78 (7.0)  Age, y  65.0 ± 16.3  65.5 ± 11.4  Female, n (%)  1919 (40.8)  444 (39.6)  SBP, mm Hg  134.5 ± 16.0  139.7 ± 20.2  DBP, mm Hg  76.3 ± 10.0  75.5 ± 11.5  Total cholesterol, mmol/L  4.3 ± 1.2  4.2 ± 1.7  HDL, mmol/L  1.3 ± 0.6  1.1 ± 1.2  LDL, mmol/L  2.5 ± 1.4  1.4 ± 3.0  BMI, kg/m2  30.8 ± 6.9  32.2 ± 6.0  HbA1c, mmol/mol  61.5 ± 17.2  56.2 ± 15.1  Lipid-lowering treatment, n (%)  3342 (71.4)  731 (65.2)  View Large Model derivation, performance measure, and validation In the derivation dataset, the absolute risks of cardiovascular hospitalization within 2 years and rehospitalization within 90 days postcardiovascular hospitalization were 12.5% and 6.7%, respectively. Univariable associations between cardiovascular hospitalization and cardiovascular rehospitalization are listed in Supplemental Table 1. Of the 10 candidate predictors (16 categories), nine predictors (15 categories) were statistically, significantly associated with cardiovascular hospitalization and rehospitalization in the final multivariable model (Table 2). Table 2 shows apparent and internal validation performance statistics of the risk-prediction model. After adjustment for optimism, the final risk-prediction model was able to discriminate patients with type 2 diabetes, with and without cardiovascular hospitalization, with a C statistic of 0.7094 (95% confidence interval 0.7067 to 0.7205), and discriminate patients with type 2 diabetes, with and without cardiovascular rehospitalization, with a C statistic 0.7118 (0.7077 to 0.7159). The agreement between the observed and predicted proportion of cardiovascular hospitalization and rehospitalization showed good apparent calibration (Fig. 1, upper left for cardiovascular hospitalization and lower left for cardiovascular rehospitalization). The optimism-adjusted calibration slope was 1.0301 (0.9856 to 1.0747) and 1.0001 (0.9711 to 1.0247) for cardiovascular hospitalization and rehospitalization, respectively (Table 3). Table 2. Final Multivariate Analysis for Cardiovascular Hospitalization and Rehospitalization Risk Among People With Type 2 Diabetes in the Derivation Cohort Predictors  Coefficient  95% Confidence Interval  Cardiovascular hospitalization   Age ≥ 70 y  0.815914  (0.793045–0.838784)   Male  0.228943  (0.206719–0.251168)   HbA1c ≥ 57 mmol/mol (7.4%)  −0.03967  (−0.06088 to −0.01846)   (BMI/10)−2  −1.85384  (−2.39533 to −1.31235)   (BMI/10)0.5  0.690585  (0.551284–0.829887)   (SBP/100)2  −0.40302  (−0.58492 to −0.22111)   (SBP/100)2 × ln(SBP/100)  0.966205  (0.758028–1.174381)   (DBP/100)−2  0.474014  (0.387498–0.56053)   (DBP/100)−2 × ln(DBP/100)  0.2724  (0.188226–0.356575)   ln(Total cholesterol/10)  0.514695  (0.27381–0.75558)   (Total cholesterol/10)0.5  −1.05803  (−1.86382 to −0.25223)   ln(HDL)  0.073489  (0.04377–0.103208)   (HDL)3  −0.02384  (−0.02699 to −0.02069)   (LDL/10)0.5  −0.55634  (−0.67239 to −0.44028)   ln(LDL/10) × (LDL/10)0.5  −0.83161  (−1.01001 to −0.65322)   Constant  −3.80246  (−4.67529 to −2.92963)  Cardiovascular rehospitalization   Age ≥ 70 y  0.90054  (0.86384–0.93724)   Male  0.22328  (0.188299–0.258261)   HbA1c ≥ 57 mmol/mol (7.4%)  0.004076  (−0.0294 to 0.037547)   (BMI/10)−2  −4.17347  (−4.62492 to −3.72202)   (BMI/10)3  0.001821  (0.001318–0.002324)   SBP/100)2  −1.16118  (−1.46728 to −0.85507)   SBP/100)3  0.773551  (0.637616–0.909486)   (DBP/100)−2  0.5875  (0.439237–0.735763)   (DBP/100)−2 × ln(DBP/100)  0.4095  (0.260667–0.558332)   (Total cholesterol/10)−2  −0.00798  (−0.01031 to −0.00565)   (Total cholesterol/10)2  −0.02734  (−0.23117 to 0.176482)   ln(HDL/10)  0.051443  (0.004285–0.0986)   (HDL/10)3  −0.02718  (−0.03277 to −0.02159)   LDL/10  −1.34491  (−1.56307 to −1.12675)   ln(LDL/10)  −0.88347  (−1.28497 to −0.48196)   Constant  −4.55873  (−4.8866 to −4.23086)  Predictors  Coefficient  95% Confidence Interval  Cardiovascular hospitalization   Age ≥ 70 y  0.815914  (0.793045–0.838784)   Male  0.228943  (0.206719–0.251168)   HbA1c ≥ 57 mmol/mol (7.4%)  −0.03967  (−0.06088 to −0.01846)   (BMI/10)−2  −1.85384  (−2.39533 to −1.31235)   (BMI/10)0.5  0.690585  (0.551284–0.829887)   (SBP/100)2  −0.40302  (−0.58492 to −0.22111)   (SBP/100)2 × ln(SBP/100)  0.966205  (0.758028–1.174381)   (DBP/100)−2  0.474014  (0.387498–0.56053)   (DBP/100)−2 × ln(DBP/100)  0.2724  (0.188226–0.356575)   ln(Total cholesterol/10)  0.514695  (0.27381–0.75558)   (Total cholesterol/10)0.5  −1.05803  (−1.86382 to −0.25223)   ln(HDL)  0.073489  (0.04377–0.103208)   (HDL)3  −0.02384  (−0.02699 to −0.02069)   (LDL/10)0.5  −0.55634  (−0.67239 to −0.44028)   ln(LDL/10) × (LDL/10)0.5  −0.83161  (−1.01001 to −0.65322)   Constant  −3.80246  (−4.67529 to −2.92963)  Cardiovascular rehospitalization   Age ≥ 70 y  0.90054  (0.86384–0.93724)   Male  0.22328  (0.188299–0.258261)   HbA1c ≥ 57 mmol/mol (7.4%)  0.004076  (−0.0294 to 0.037547)   (BMI/10)−2  −4.17347  (−4.62492 to −3.72202)   (BMI/10)3  0.001821  (0.001318–0.002324)   SBP/100)2  −1.16118  (−1.46728 to −0.85507)   SBP/100)3  0.773551  (0.637616–0.909486)   (DBP/100)−2  0.5875  (0.439237–0.735763)   (DBP/100)−2 × ln(DBP/100)  0.4095  (0.260667–0.558332)   (Total cholesterol/10)−2  −0.00798  (−0.01031 to −0.00565)   (Total cholesterol/10)2  −0.02734  (−0.23117 to 0.176482)   ln(HDL/10)  0.051443  (0.004285–0.0986)   (HDL/10)3  −0.02718  (−0.03277 to −0.02159)   LDL/10  −1.34491  (−1.56307 to −1.12675)   ln(LDL/10)  −0.88347  (−1.28497 to −0.48196)   Constant  −4.55873  (−4.8866 to −4.23086)  Abbreviation: ln, natural logarithm. View Large Figure 1. View largeDownload slide Assessing calibration in the (left) derivation and (right) validation cohorts for cardiovascular (upper) hospitalization and (lower) rehospitalization. Figure 1. View largeDownload slide Assessing calibration in the (left) derivation and (right) validation cohorts for cardiovascular (upper) hospitalization and (lower) rehospitalization. Table 3. Model Diagnostics (With 95% CI) Measure  Derivation  Validation  Apparent Performance  Test Performance  Average Optimism  Optimism Corrected  Cardiovascular hospitalization     C statistic  0.7163 (0.7136–0.7190)  0.7027 (0.6996–0.7058)  +0.0069  0.7094 (0.7067–0.7205)  0.7092 (0.7033–0.7151)   Calibration slope  1.0000 (0.9806–1.0194)  0.9933 (0.9899–0.9966)  +0.0067  0.9933 (0.9739–1.0127)  1.0001 (0.9807–1.0195)  Cardiovascular rehospitalization     C statistic  0.7154 (0.7113–0.7195)  0.7136 (0.7105–0.7167)  +0.0036  0.7118 (0.7077–0.7159)  0.7098 (0.7014–0.7182)   Calibration slope  1.0000 (0.9766–1.0234)  0.9976 (0.9949–1.0003)  +0.0024  0.9976 (0.9742–0.9796)  0.9981 (0.9948–1.0482)  Measure  Derivation  Validation  Apparent Performance  Test Performance  Average Optimism  Optimism Corrected  Cardiovascular hospitalization     C statistic  0.7163 (0.7136–0.7190)  0.7027 (0.6996–0.7058)  +0.0069  0.7094 (0.7067–0.7205)  0.7092 (0.7033–0.7151)   Calibration slope  1.0000 (0.9806–1.0194)  0.9933 (0.9899–0.9966)  +0.0067  0.9933 (0.9739–1.0127)  1.0001 (0.9807–1.0195)  Cardiovascular rehospitalization     C statistic  0.7154 (0.7113–0.7195)  0.7136 (0.7105–0.7167)  +0.0036  0.7118 (0.7077–0.7159)  0.7098 (0.7014–0.7182)   Calibration slope  1.0000 (0.9766–1.0234)  0.9976 (0.9949–1.0003)  +0.0024  0.9976 (0.9742–0.9796)  0.9981 (0.9948–1.0482)  Abbreviation: CI, confidence interval. View Large External validation In the external validation cohort, the absolute risks for cardiovascular hospitalization and rehospitalization were 16.3% and 7.0%, respectively. The application of our final risk-prediction model to the independent population gave a C statistic of 0.7092 (0.7033 to 0.7151) for cardiovascular hospitalization and 0.7098 (0.7014 to 0.7182) for cardiovascular rehospitalization and good calibration (Fig. 1, upper right for cardiovascular hospitalization and lower right for cardiovascular rehospitalization), with the calibration slope 1.0001 (0.9807 to 1.0195) and 0.9981 (0.9948 to 1.0482) for cardiovascular hospitalization and rehospitalization, respectively. Performance at the threshold for 10% and 20% of patients at highest risk Table 4 shows the sensitivity, specificity, and observed risk for the 5%, 10%, 15%, 20%, and 25% of patients at the highest predicted risk of each outcome in the validation cohort, shown for illustrative purposes. For example, when a risk threshold of 24.53% for cardiovascular hospitalization and 7.93% for cardiovascular rehospitalization is used to identify the 20% at the highest predicted risk, the sensitivity was 33.40% for cardiovascular hospitalization and 45.20% for cardiovascular rehospitalization, the specificity was 84.60% for cardiovascular hospitalization and 75.90% for cardiovascular rehospitalization, and the observed risk was 30.09% for cardiovascular hospitalization and 11.98% for cardiovascular rehospitalization, respectively. Table 4. Predicted Risk of Cardiovascular Hospitalization and Rehospitalization in the Validation Cohort Based on Various Cut-Offs   Cut-Off, %, for Risk  Mean Predicted Risk, %  Sensitivity, %  Specificity, %  Positive Predictive Value, %  Observed Risk, %  Cardiovascular hospitalization               Top 5%  38.17  51.96  10.30 (9.70–10.90)  97.40 (97.20–97.50)  43.50 (41.50–45.50)  43.48   Top 10%  31.73  43.35  17.50 (16.80–18.30)  94.60 (94.40–94.80)  38.60 (37.20–40.10)  38.62   Top 15%  27.54  37.71  24.70 (23.90–25.60)  90.10 (89.80–90.40)  32.80 (31.80–33.90)  32.83   Top 20%  24.53  33.77  34.00 (33.10–35.00)  84.60 (84.20–84.90)  30.10 (29.20–31.00)  30.09   Top 25%  22.22  31.05  42.80 (41.80–43.80)  78.40 (78.00–78.70)  27.90 (27.20–28.60)  27.89  Cardiovascular rehospitalization               Top 5%  11.34  15.86  26.20 (24.90–27.50)  91.20 (91.00–91.50)  18.30 (17.40–19.30)  18.33   Top 10%  9.67  13.63  34.50 (33.10–36.00)  84.30 (84.00–84.60)  14.20 (13.50–14.90)  14.22   Top 15%  8.69  12.59  40.50 (39.00–42.00)  79.10 (78.80–79.50)  12.70 (12.20–13.30)  12.73   Top 20%  7.93  12.02  45.20 (43.70–46.70)  75.90 (75.50–76.30)  12.40 (11.90–12.90)  12.37   Top 25%  7.16  11.46  50.00 (48.50–51.50)  72.40 (72.00–72.70)  12.00 (11.50–12.50)  11.98    Cut-Off, %, for Risk  Mean Predicted Risk, %  Sensitivity, %  Specificity, %  Positive Predictive Value, %  Observed Risk, %  Cardiovascular hospitalization               Top 5%  38.17  51.96  10.30 (9.70–10.90)  97.40 (97.20–97.50)  43.50 (41.50–45.50)  43.48   Top 10%  31.73  43.35  17.50 (16.80–18.30)  94.60 (94.40–94.80)  38.60 (37.20–40.10)  38.62   Top 15%  27.54  37.71  24.70 (23.90–25.60)  90.10 (89.80–90.40)  32.80 (31.80–33.90)  32.83   Top 20%  24.53  33.77  34.00 (33.10–35.00)  84.60 (84.20–84.90)  30.10 (29.20–31.00)  30.09   Top 25%  22.22  31.05  42.80 (41.80–43.80)  78.40 (78.00–78.70)  27.90 (27.20–28.60)  27.89  Cardiovascular rehospitalization               Top 5%  11.34  15.86  26.20 (24.90–27.50)  91.20 (91.00–91.50)  18.30 (17.40–19.30)  18.33   Top 10%  9.67  13.63  34.50 (33.10–36.00)  84.30 (84.00–84.60)  14.20 (13.50–14.90)  14.22   Top 15%  8.69  12.59  40.50 (39.00–42.00)  79.10 (78.80–79.50)  12.70 (12.20–13.30)  12.73   Top 20%  7.93  12.02  45.20 (43.70–46.70)  75.90 (75.50–76.30)  12.40 (11.90–12.90)  12.37   Top 25%  7.16  11.46  50.00 (48.50–51.50)  72.40 (72.00–72.70)  12.00 (11.50–12.50)  11.98  View Large Clinical examples Supplemental Chart 1 gives a clinical example of the application of prognostic score charts with graphical illustrations for cardiovascular hospitalization and rehospitalization risk-prediction models to predict a 2-year risk of cardiovascular hospitalization and risk of rehospitalization within 90 days of a prior cardiovascular hospitalization. Discussion We have developed two risk-prediction models to estimate the absolute risk of cardiovascular hospitalization within 2 years and cardiovascular rehospitalization after 90 days of prior cardiovascular hospitalization in a cohort of patients with type 2 diabetes in England. We then externally validated this model in another English cohort. The two prediction models had excellent calibration and useful discrimination, with C statistics of >0.70, both in the derivation cohort and external validation cohort. The two prediction models were built from clinical variables, usually recorded and accessible in primary care settings, implying that they can be readily applied in routine primary care. Strengths and limitations Our two risk algorithms have several advantages over those in use in many developed countries. Our models are based on absolute risks, determined and validated in two independent populations. The models are developed from routinely recorded demographic and clinical measurements in primary care settings, which suggests that they can be straightforwardly applied in general practice and are readily amenable for further external validations in countries that have routine recorded data accessible for such aims. Furthermore, the two risk algorithms can be easily integrated into online calculators for implementation in general practices. The methods used to derive and validate the model are similar to those for other risk-prediction algorithms derived from the Clinical Practice Research Datalink and QResearch databases (21, 22). The majority of predictors in our final model are accurate and reliable clinical measurements (23), routinely recorded in primary care settings and updated and reviewed for patients with type 2 diabetes, and are less varied than in other datasets. Moreover, the proportion of missing values was low, which would lead to little variation in external applications, although multiple imputation was still applied in our study. We acknowledge that our prediction models do not take into account diabetes duration, antidiabetes treatments, antihypertensive treatments, prior history of CVDs, other diabetes complications (e.g., renal failure), lifestyle risk factors (such as smoking), and other comorbidities as a result of limitations in the original data, but we feel that the clinical measurements included in our models could be proxies for missing predictors. Data limitations also prevented the extension of our model to all diabetes complications rather than those relating to cardiovascular hospitalization. The relatively low sensitivities of our models to identify individuals at high risk of cardiovascular hospitalization and rehospitalization is another limitation of the study. As a result of the similarity between the derivation and validation cohorts, further external validation (e.g., cohorts from other countries) is warranted. Comparison with other studies Nirantharakumar et al. (5) developed a prediction model among patients with diabetes to estimate adverse events (either excessive length of stay or inpatient mortality) over 4 years using a secondary care dataset in Birmingham, England. The predictors applied in this model covered demographic characteristics, clinical pathological test results, and use of insulin, recorded within 72 hours of hospitalization. That population represented the people with at least previous inpatient hospitalization and probably reflects a cohort with more severe conditions and likely higher prior probabilities of an event. The ranges of clinical measurements during a hospital admission would tend to be greater than in the community, as patients would be sicker and, e.g., blood glucose control could be the reason for hospitalization or exacerbated by acute illness, making the dataset difficult to use as a basis for a prediction tool in routine care. Most importantly, this prediction model has not been externally validated, and the model performance needs to be evaluated further in external populations before its application in clinical practices. Rubin et al. (6) developed a tool to predict the risk of all-cause readmission within 30 days among hospitalized patients with diabetes using hospitalized data. The short time gap between predictor measurements and outcome made the tool less useful for clinical practice. The reasons for hospitalization could be quite mixed, with different pathway and potential interventions. Therefore, the use of the all-cause hospitalization risk as the outcome provides different information and allows less-targeted interventions. As with the model of Nirantharakumar et al. (5), this model has also not been externally validated in any independent population. Previous studies have not focused on CVD as both a major cause and cost for hospital admission among patients with diabetes. To understand the potential risk of cardiovascular hospitalization in the next year and the risk of a new episode (within 90 days) of a cardiovascular event (rehospitalization), it could be helpful for clinicians to facilitate tailored, more intensive care to those with high-risk profiles and to reduce hospitalization inpatient cost. Conclusion and policy implication Our study developed prediction tools to estimate the 2-year risk of cardiovascular hospitalization and rehospitalization within 90 days of a previous hospitalization. Our two prediction models have two important implications for clinical practice. First, they can be used as tools to screen populations at high risk of cardiovascular hospitalization and rehospitalization. Both algorithms are based on readily accessible clinical data, routinely recorded in primary care and reviewed by diabetes management teams. They can be readily integrated into primary care computer systems or developed into an application for a handheld device for ease of use. Secondly, our risk-prediction models could be used to establish different treatment thresholds in clinical practice through consensus development of national guidelines. Abbreviations: BMI body mass index CVD cardiovascular disease DBP diastolic blood pressure HbA1c hemoglobin A1c HDL high-density lipoprotein ICD International Classification of Diseases LDL low-density lipoprotein NHS National Health Service SBP systolic blood pressure. Acknowledgments We thank Toby Prevost, Chris Bunn, Simon Cohn, Sarah Donald, Charlotte Paddison, Candice Ward, Peers for Progress, West Anglia CLRN, Cambridgeshire & Peterborough PCT, Primary Care Research Network–East of England, Eastern Diabetes Research Network, MRC Epidemiology Unit, participating general practices, Jackie Williams, Caroline Taylor, Kym Mercer, Kevin Baker, Ben Bowers, Kalsoom Akhter (CUH Wolfson Diabetes & Endocrinology Clinic), James Brimicombe (Cambridge University), Kim Birch of Trumpington St General Practice, CUH Wolfson Diabetes & Endocrinology Clinic Educators, The RAPSID Patient Committee (Phillip Jones, Liz Carvlin, and Roger Smith), and the peers and peer-support participants. The views expressed are those of the authors and not necessarily those of the National Health Service, National Institute for Health Research, or Department of Health. Financial Support: This work was supported by National Institute for Health Research under its Research for Patient Benefit Programme Grants PB-PG-0808-17303 and PB-PG-0610-22311. Disclosure Summary: The authors have nothing to disclose. References 1. American Diabetes Association. Standards of medical care in diabetes--2012 [published erratum appears in Diabetes Care. 35(3): 660–660]. Diabetes Care . 2012; 35( Suppl 1): S11– S63. CrossRef Search ADS PubMed  2. Sampson MJ, Dozio N, Ferguson B, Dhatariya K. Total and excess bed occupancy by age, specialty and insulin use for nearly one million diabetes patients discharged from all English acute hospitals. Diabetes Res Clin Pract . 2007; 77( 1): 92– 98. Google Scholar CrossRef Search ADS PubMed  3. Simmons D, English P, Robins P, Craig A, Addicott R. Should diabetes be commissioned through multidisciplinary networks, rather than practice based commissioning? Prim Care Diabetes . 2011; 5( 1): 39– 44. Google Scholar CrossRef Search ADS PubMed  4. Vamos EP, Millett C, Parsons C, Aylin P, Majeed A, Bottle A. Nationwide study on trends in hospital admissions for major cardiovascular events and procedures among people with and without diabetes in England, 2004–2009. Diabetes Care . 2012; 35( 2): 265– 272. Google Scholar CrossRef Search ADS PubMed  5. Nirantharakumar K, Hemming K, Narendran P, Marshall T, Coleman JJ. A prediction model for adverse outcome in hospitalized patients with diabetes. Diabetes Care . 2013; 36( 11): 3566– 3572. Google Scholar CrossRef Search ADS PubMed  6. Rubin DJ, Handorf EA, Golden SH, Nelson DB, McDonnell ME, Zhao H. Development and validation of a novel tool to predict hospital readmission risk among patients with diabetes. Endocr Pract . 2016; 22( 10): 1204– 1215. Google Scholar CrossRef Search ADS PubMed  7. Khalid JM, Raluy-Callado M, Curtis BH, Boye KS, Maguire A, Reaney M. Rates and risk of hospitalisation among patients with type 2 diabetes: retrospective cohort study using the UK General Practice Research Database linked to English Hospital Episode Statistics. Int J Clin Pract . 2014; 68( 1): 40– 48. Google Scholar CrossRef Search ADS PubMed  8. Yu D, Simmons D. Association between blood pressure and risk of cardiovascular hospital admissions among people with type 2 diabetes. Heart . 2014; 100( 18): 1444– 1449. Google Scholar CrossRef Search ADS PubMed  9. Yu D, Simmons D. Relationship between HbA1c and risk of all-cause hospital admissions among people with Type 2 diabetes. Diabet Med . 2013; 30( 12): 1407– 1411. Google Scholar CrossRef Search ADS PubMed  10. Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S, Riley RD, Hemingway H, Altman DG, PROGRESS Group. Prognosis research strategy (PROGRESS) 3: prognostic model research. PLoS Med . 2013; 10( 2): e1001381. Google Scholar CrossRef Search ADS PubMed  11. Hemingway H, Croft P, Perel P, Hayden JA, Abrams K, Timmis A, Briggs A, Udumyan R, Moons KG, Steyerberg EW, Roberts I, Schroter S, Altman DG, Riley RD, PROGRESS Group. Prognosis research strategy (PROGRESS) 1: a framework for researching clinical outcomes. BMJ . 2013; 346: e5595. Google Scholar CrossRef Search ADS PubMed  12. Simmons D, Prevost AT, Bunn C, Holman D, Parker RA, Cohn S, Donald S, Paddison CA, Ward C, Robins P, Graffy J. Impact of community based peer support in type 2 diabetes: a cluster randomised controlled trial of individual and/or group approaches. PLoS One . 2015; 10( 3): e0120277. Google Scholar CrossRef Search ADS PubMed  13. Simmons D, Cohn S, Bunn C, Birch K, Donald S, Paddison C, Ward C, Robins P, Prevost AT, Graffy J. Testing a peer support intervention for people with type 2 diabetes: A pilot for a randomised controlled trial. BMC Fam Pract . 2013; 14: 5. Google Scholar CrossRef Search ADS PubMed  14. Fisher EB, Ayala GX, Ibarra L, Cherrington AL, Elder JP, Tang TS, Heisler M, Safford MM, Simmons D; Peers for Progress Investigator Group. Contributions of peer support to health, health care, and prevention: Papers from peers for progress. Ann Fam Med . 2015; 13( Suppl 1): S2– S8. Google Scholar CrossRef Search ADS PubMed  15. Dale JR, Williams SM, Bowyer V. What is the effect of peer support on diabetes outcomes in adults? A systematic review. Diabet Med . 2012; 29( 11): 1361– 1377. Google Scholar CrossRef Search ADS PubMed  16. Simmons D, Yu D, Wenzel H. Changes in hospital admissions and inpatient tariff associated with a Diabetes Integrated Care Initiative: preliminary findings. J Diabetes . 2014; 6( 1): 81– 89. Google Scholar CrossRef Search ADS PubMed  17. Rubin DB. Multiple Imputation for Nonresponse in Surveys. Hoboken, NJ: John Wiley & Sons, Inc.; 1987. 18. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol . 1996; 49( 12): 1373– 1379. Google Scholar CrossRef Search ADS PubMed  19. Yu D, Simmons D. Association between pulse pressure and risk of hospital admissions for cardiovascular events among people with type 2 diabetes: a population-based case-control study. Diabet Med . 2015; 32( 9): 1201– 1206. Google Scholar CrossRef Search ADS PubMed  20. Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, Vickers AJ, Ransohoff DF, Collins GS. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med . 2015; 162( 1): W1– W73. Google Scholar CrossRef Search ADS PubMed  21. Sultan AA, West J, Grainge MJ, Riley RD, Tata LJ, Stephansson O, Fleming KM, Nelson-Piercy C, Ludvigsson JF. Development and validation of risk prediction model for venous thromboembolism in postpartum women: multinational cohort study. BMJ . 2016; 355: i6253. Google Scholar CrossRef Search ADS PubMed  22. Hippisley-Cox J, Coupland C. Derivation and validation of updated QFracture algorithm to predict risk of osteoporotic fracture in primary care in the United Kingdom: prospective open cohort study. BMJ . 2012; 344: e3427. Google Scholar CrossRef Search ADS PubMed  23. Hemke AC, Heemskerk MB, van Diepen M, Dekker FW, Hoitsma AJ. Improved mortality prediction in dialysis patients using specific clinical and laboratory data. Am J Nephrol . 2015; 42( 2): 158– 167. Google Scholar CrossRef Search ADS PubMed  Copyright © 2018 Endocrine Society http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Clinical Endocrinology and Metabolism Oxford University Press

Development and External Validation of Risk Scores for Cardiovascular Hospitalization and Rehospitalization in Patients With Diabetes

Loading next page...
 
/lp/ou_press/development-and-external-validation-of-risk-scores-for-cardiovascular-B8kFHpppwq
Publisher
Oxford University Press
Copyright
Copyright © 2018 Endocrine Society
ISSN
0021-972X
eISSN
1945-7197
D.O.I.
10.1210/jc.2017-02293
Publisher site
See Article on Publisher Site

Abstract

Abstract Context Cardiovascular disease (CVD) is a common and costly reason for hospitalization and rehospitalization among patients with type 2 diabetes. Objective This study aimed to develop and externally validate two risk-prediction models for cardiovascular hospitalization and cardiovascular rehospitalization. Design Two independent prospective cohorts. Setting The derivation cohort includes 4704 patients with type 2 diabetes from 18 general practices in Cambridgeshire. The validation cohort comprises 1121 patients with type 2 diabetes from post-trial follow-up data. Main Outcome Measure Cardiovascular hospitalization over 2 years and cardiovascular rehospitalization after 90 days of the prior CVD hospitalization. Results The absolute rate of cardiovascular hospitalization and rehospitalization was 12.5% and 6.7% in the derivation cohort and 16.3% and 7.0% in the validation cohort. Discrimination of the models was similar in both cohorts, with C statistics above 0.70 and excellent calibration of observed and predicted risks. Conclusion Two prediction models that quantify risks of cardiovascular hospitalization and rehospitalization have been developed and externally validated. They are based on a small number of clinical measurements that are available for patients with type 2 diabetes in many developed countries in primary care settings and could serve as the tools to screen the population at high risk of cardiovascular hospitalization and rehospitalization. The prevalence and cost of diabetes are growing rapidly worldwide (1). People with diabetes are twice as likely to be admitted to a hospital, and at least 10% of those in a hospital have diabetes at any one time (2). In some age groups, it is as many as one in five (3). The associated costs of excess admissions, as well as increased costs per admission, are important contributors to the financial burden borne by health care systems from diabetes and often reflect preventable morbidity suffered by patients (4). Previously, two prediction tools have been developed, both based on secondary care data, to identify those with diabetes, at high risk of either all-cause excessive length of stay or all-cause inpatient mortality over 4 years (5) or all-cause readmission within 30 days among hospitalized patients (6). However, the practical application of both prediction models was limited by a lack of external validation, nonspecificity for people with type 2 diabetes, the use of predictors derived from secondary care rather than primary care data, variations on predictors recorded in different datasets (e.g., comorbidity), and a relatively short time gap between baseline and outcome (readmission within 30 days). Among hospital admissions, cardiovascular events are the major cause for hospitalization in people with type 2 diabetes (7). Although risk factors, such as blood pressure and hemoglobin A1c (HbA1c), are recognized as warranting intervention on their own (8, 9), there has been no current algorithm to estimate the absolute risk of cardiovascular hospitalization and rehospitalization in people with type 2 diabetes. The use of a model to make predictions for individual patients with type 2 diabetes is more comprehensive than the use of individual risk factors and is preferred to the risk-grouping approach (10, 11). The aim of our study was to develop and externally validate different prediction models based on reliable clinical measurements in primary care settings for cardiovascular hospitalization over the next 2 years and cardiovascular rehospitalization up to 90 days following a prior cardiovascular hospitalization. Materials and Methods Data source and study population We used two cohorts from Cambridgeshire, United Kingdom: one (Derivation) based on the electronic health record data from primary care settings to develop our cardiovascular hospitalization and rehospitalization risk scores and another (Validation) based on post-trial cohort data for external validation. Derivation cohort Patient lists from 18 general practices across Cambridgeshire in 2008/2009 were collated and linked with hospital admissions (Secondary Uses Service) data, as part of an evaluation of diabetes care across the county by the local health board, National Health Service (NHS) Cambridgeshire. This cohort was limited to volunteer practices using the Egton Medical Information Systems general practitioner software system, from which a predefined set of data could be extracted. There was no systematic selection process for these surgeries, and data extracted were for their entire diabetes population. All patients with diabetes had follow-up hospitalization data from 2010 to 2011. Hospital admissions to NHS and private hospitals within and outside Cambridgeshire were followed up. No personal identifiers were released to researchers, and all subsequent analyses were conducted on anonymous datasets. Validation cohort The design and methods of the Randomized Controlled Trial of Peer Support in Type 2 Diabetes trial have been published previously (12), as have its Consolidated Standards of Reporting Trials diagram and the results of its primary outcomes (12). In brief, Randomized Controlled Trial of Peer Support in Type 2 Diabetes was a 2 × 2 factorial cluster randomized controlled trial comparing four groups: controls, 1:1 (individual) peer support, group peer support, and combined 1:1 and group peer support among patients with type 2 diabetes. Participants had their diabetes for at least 12 months, and those with dementia or psychotic illness were excluded. Participants were recruited from communities across Cambridgeshire and neighboring areas of Essex and Hertfordshire. Follow-up data were only available for participants in Cambridgeshire and neighboring areas of Hertfordshire that are served by the Cambridgeshire and Peterborough Clinical Commissioning Group. Clusters were defined by local government (“parish council”) boundaries. The intervention was developed following a pilot (13), using a framework defined by Peers for Progress (14). Peers facilitating peer support were termed “peer support facilitators,” and their selection, training, support, and the overall program are described elsewhere (15). The intervention lasted 8 to 12 months and was commenced and concluded, cluster by cluster, between 6 February 2011 and 4 December 2012. Ethics approval was received from the Cambridgeshire REC2 Committee (10/H0308/72), and signed consent included agreement for access to hospital data. At baseline, demographic data, blood pressure, and HbA1c and lipid profiles were collected. Each participant was followed up until June 2015 (0.91 to 4.07 years’ follow-up from beginning/entry into the trial). Hospitalization (NHS hospitals and private hospitals), Accident and Emergency, and outpatient visits within/outside Cambridgeshire and the included areas of Hertfordshire were completely collected through Cambridgeshire and Peterborough Clinical Commissioning Group (16) and the elective/nonelective status and International Classification of Diseases (ICD-10) codes (8). Defining cardiovascular hospitalization and rehospitalization The primary outcome of the study was having at least one hospitalization with cardiovascular disease (CVD) as the primary diagnosis (ICD-10: I20–I25, I60–I69, and I73 in the first ICD field) over the 2-year follow-up and having at least one CVD rehospitalization after 90 days of prior CVD hospitalization. Candidate predictors, missing data, and power calculations To achieve the maximum extrapolation application of our risk algorithm, objective clinical measurements were used as predictors in the model, including body mass index (BMI), blood pressure [systolic (SBP) and diastolic (DBP)], and the metabolic variables, glycated hemoglobin (HbA1c) and lipid profiles. We also included demographic characteristics (age and sex) and whether the patient was on lipid-lowering treatment. Patients with diabetes were invited to have their blood pressure and metabolic variables measured at least once a year after the diagnosis of diabetes, and the most recent was taken before 1 April 2009 (a minimum of 50 days before the first admission). Diabetes duration was not universally recorded and hence, was not usefully available for analysis. Diabetes therapy was not included in the dataset. Lipid-lowering treatment was recorded. Our derivation cohort had missing information on BMI (3.17%), SBP (9.95%), DBP (9.95%), total cholesterol (12.35%), high-density lipoprotein (HDL; 14.56%), and low-density lipoprotein (LDL; 16.27%). We used multiple imputation to replace missing values by using a chained equation approach based on all candidate predictors and outcomes. We created 16 imputed datasets for missing variables that were then combined across all datasets by using Rubin’s rule (17) to obtain final model estimates. Limited information was missing (<1%) in our external validation dataset, and the complete dataset was used in our analysis. On the basis of an estimated 588 cardiovascular hospitalizations and 316 cardiovascular rehospitalizations and 16 predictors or levels in our derivation cohort, we had an effective sample size of 37 cardiovascular hospitalization and 21 cardiovascular rehospitalization per predictor or level, above the minimum requirement suggested by Peduzzi et al. (18). Ethical approval The derivation cohort work had approval from the Cambridgeshire Research Ethics Committee as part of a wider service evaluation. Ethics approval for the validation cohort was received from the Cambridgeshire REC2 Committee (10/H0308/72), and signed consent included agreement for access to hospital data. Statistical analysis for model derivation and external validation We treated incidence occurrence of cardiovascular hospitalization after the first 90 days since the start of follow-up and the incident occurrence of cardiovascular rehospitalization as binary outcome measures. For each of the 15 candidate predictors or levels, we used a univariate logistic regression model to calculate the unadjusted odds ratios. For derivation of the risk-prediction model, we initially included all candidate predictors in a multivariable logistic regression model. We used fractional polynomials to model potential nonlinear relationships between continuous predictors and outcome. Through backward elimination, we excluded lower lipid treatment from the multivariate model, as it was not statistically significant (P > 0.1, based on change in log likelihood). After elimination, we reinserted the excluded predictor into the final model to check further whether it became statistically significant. We also rechecked fractional polynomial terms at this stage and re-estimated them if necessary. We formed the risk equations for predicting the log odds of cardiovascular hospitalization and cardiovascular rehospitalization by using the estimated regression coefficients multiplied by the corresponding predictors included in our models, together with the intercepts. This process ultimately led to equations for the predicted risk = 1/(1 + e−risk score), whether the “risk score” is the predicted log odds of cardiovascular hospitalization or cardiovascular rehospitalization from the developed models. To facilitate model use in clinical practice, the logistic regression equations were transformed into prognostic score charts. The coefficients in the logistic regression equation were multiplied by 50 and rounded to the nearest integer to obtain the prognostic score per predictor. Multiplication by 50 was chosen to get the majority of the coefficients close to an integer, thereby minimizing the effects of rounding. The sum of all prognostic scores reflects patients’ probability of cardiovascular hospitalization or cardiovascular rehospitalization. We assessed the performance of the models in terms of the C statistic and calibration slope (where 1.00 is ideal). The C statistic represents the probability that for any randomly selected pair of people with type 2 diabetes, with and without outcomes, the patient with outcomes had a higher predicted risk (19). A value of 0.50 indicated no discrimination, and 1.00 represents perfect discrimination. We then undertook internal validation to correct measures of predictive performance for optimism (overfitting) by bootstrapping 100 samples of the derivation data. We repeated the model derivation process in each bootstrap sample to produce a model, applied the model to the same bootstrap sample to quantify apparent performance, and applied the model to the original dataset to test model performance (calibration slope and C statistics) and optimism (difference in the test performance and apparent performance). We then estimated the overall optimism across all models. We applied our risk-prediction model to each patient with type 2 diabetes in the external validation cohort on the basis of the presence of one or more predictors. We examined the performance of this final model, both in the derivation dataset and then in the external validation dataset in terms of discrimination by calculating the C statistics. We examined calibration by plotting agreement between predicted and observed risks across one-tenth of the predicted risks. We used Stata V14.0 for all statistical analyses. This study was conducted and reported in line with the Transparent Reporting of a Multivariate Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guidelines (20). Role of the funding source The sponsors of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. Results Study participants In our derivation cohort, we analyzed information on 4704 patients with type 2 diabetes with 588 cardiovascular hospitalizations within 2 years and 316 rehospitalizations after 90 days since a prior cardiovascular hospitalization. Our validated cohort had information on 1121 patients with type 2 diabetes with 183 cardiovascular hospitalizations and 78 rehospitalizations. Table 1 summarizes the basic characteristics and potential predictors of the study population. Patients with type 2 diabetes in both cohorts had similar age, sex, blood pressure, and total cholesterol. Patients in the derived cohort had a higher level of HDL, LDL, and HbA1c. Compared with the derivation cohort, those in the validation cohort were more likely to be prescribed lowering lipid medicine and had more cardiovascular hospitalization and rehospitalization. Table 1. Baseline Characteristics of Study Populations   Derivation Cohort  External Validation Cohort  n  4704  1121  Cardiovascular hospitalization, n (%)  588 (12.5)  183 (16.3)  Cardiovascular rehospitalization, n (%)  316 (6.7)  78 (7.0)  Age, y  65.0 ± 16.3  65.5 ± 11.4  Female, n (%)  1919 (40.8)  444 (39.6)  SBP, mm Hg  134.5 ± 16.0  139.7 ± 20.2  DBP, mm Hg  76.3 ± 10.0  75.5 ± 11.5  Total cholesterol, mmol/L  4.3 ± 1.2  4.2 ± 1.7  HDL, mmol/L  1.3 ± 0.6  1.1 ± 1.2  LDL, mmol/L  2.5 ± 1.4  1.4 ± 3.0  BMI, kg/m2  30.8 ± 6.9  32.2 ± 6.0  HbA1c, mmol/mol  61.5 ± 17.2  56.2 ± 15.1  Lipid-lowering treatment, n (%)  3342 (71.4)  731 (65.2)    Derivation Cohort  External Validation Cohort  n  4704  1121  Cardiovascular hospitalization, n (%)  588 (12.5)  183 (16.3)  Cardiovascular rehospitalization, n (%)  316 (6.7)  78 (7.0)  Age, y  65.0 ± 16.3  65.5 ± 11.4  Female, n (%)  1919 (40.8)  444 (39.6)  SBP, mm Hg  134.5 ± 16.0  139.7 ± 20.2  DBP, mm Hg  76.3 ± 10.0  75.5 ± 11.5  Total cholesterol, mmol/L  4.3 ± 1.2  4.2 ± 1.7  HDL, mmol/L  1.3 ± 0.6  1.1 ± 1.2  LDL, mmol/L  2.5 ± 1.4  1.4 ± 3.0  BMI, kg/m2  30.8 ± 6.9  32.2 ± 6.0  HbA1c, mmol/mol  61.5 ± 17.2  56.2 ± 15.1  Lipid-lowering treatment, n (%)  3342 (71.4)  731 (65.2)  View Large Model derivation, performance measure, and validation In the derivation dataset, the absolute risks of cardiovascular hospitalization within 2 years and rehospitalization within 90 days postcardiovascular hospitalization were 12.5% and 6.7%, respectively. Univariable associations between cardiovascular hospitalization and cardiovascular rehospitalization are listed in Supplemental Table 1. Of the 10 candidate predictors (16 categories), nine predictors (15 categories) were statistically, significantly associated with cardiovascular hospitalization and rehospitalization in the final multivariable model (Table 2). Table 2 shows apparent and internal validation performance statistics of the risk-prediction model. After adjustment for optimism, the final risk-prediction model was able to discriminate patients with type 2 diabetes, with and without cardiovascular hospitalization, with a C statistic of 0.7094 (95% confidence interval 0.7067 to 0.7205), and discriminate patients with type 2 diabetes, with and without cardiovascular rehospitalization, with a C statistic 0.7118 (0.7077 to 0.7159). The agreement between the observed and predicted proportion of cardiovascular hospitalization and rehospitalization showed good apparent calibration (Fig. 1, upper left for cardiovascular hospitalization and lower left for cardiovascular rehospitalization). The optimism-adjusted calibration slope was 1.0301 (0.9856 to 1.0747) and 1.0001 (0.9711 to 1.0247) for cardiovascular hospitalization and rehospitalization, respectively (Table 3). Table 2. Final Multivariate Analysis for Cardiovascular Hospitalization and Rehospitalization Risk Among People With Type 2 Diabetes in the Derivation Cohort Predictors  Coefficient  95% Confidence Interval  Cardiovascular hospitalization   Age ≥ 70 y  0.815914  (0.793045–0.838784)   Male  0.228943  (0.206719–0.251168)   HbA1c ≥ 57 mmol/mol (7.4%)  −0.03967  (−0.06088 to −0.01846)   (BMI/10)−2  −1.85384  (−2.39533 to −1.31235)   (BMI/10)0.5  0.690585  (0.551284–0.829887)   (SBP/100)2  −0.40302  (−0.58492 to −0.22111)   (SBP/100)2 × ln(SBP/100)  0.966205  (0.758028–1.174381)   (DBP/100)−2  0.474014  (0.387498–0.56053)   (DBP/100)−2 × ln(DBP/100)  0.2724  (0.188226–0.356575)   ln(Total cholesterol/10)  0.514695  (0.27381–0.75558)   (Total cholesterol/10)0.5  −1.05803  (−1.86382 to −0.25223)   ln(HDL)  0.073489  (0.04377–0.103208)   (HDL)3  −0.02384  (−0.02699 to −0.02069)   (LDL/10)0.5  −0.55634  (−0.67239 to −0.44028)   ln(LDL/10) × (LDL/10)0.5  −0.83161  (−1.01001 to −0.65322)   Constant  −3.80246  (−4.67529 to −2.92963)  Cardiovascular rehospitalization   Age ≥ 70 y  0.90054  (0.86384–0.93724)   Male  0.22328  (0.188299–0.258261)   HbA1c ≥ 57 mmol/mol (7.4%)  0.004076  (−0.0294 to 0.037547)   (BMI/10)−2  −4.17347  (−4.62492 to −3.72202)   (BMI/10)3  0.001821  (0.001318–0.002324)   SBP/100)2  −1.16118  (−1.46728 to −0.85507)   SBP/100)3  0.773551  (0.637616–0.909486)   (DBP/100)−2  0.5875  (0.439237–0.735763)   (DBP/100)−2 × ln(DBP/100)  0.4095  (0.260667–0.558332)   (Total cholesterol/10)−2  −0.00798  (−0.01031 to −0.00565)   (Total cholesterol/10)2  −0.02734  (−0.23117 to 0.176482)   ln(HDL/10)  0.051443  (0.004285–0.0986)   (HDL/10)3  −0.02718  (−0.03277 to −0.02159)   LDL/10  −1.34491  (−1.56307 to −1.12675)   ln(LDL/10)  −0.88347  (−1.28497 to −0.48196)   Constant  −4.55873  (−4.8866 to −4.23086)  Predictors  Coefficient  95% Confidence Interval  Cardiovascular hospitalization   Age ≥ 70 y  0.815914  (0.793045–0.838784)   Male  0.228943  (0.206719–0.251168)   HbA1c ≥ 57 mmol/mol (7.4%)  −0.03967  (−0.06088 to −0.01846)   (BMI/10)−2  −1.85384  (−2.39533 to −1.31235)   (BMI/10)0.5  0.690585  (0.551284–0.829887)   (SBP/100)2  −0.40302  (−0.58492 to −0.22111)   (SBP/100)2 × ln(SBP/100)  0.966205  (0.758028–1.174381)   (DBP/100)−2  0.474014  (0.387498–0.56053)   (DBP/100)−2 × ln(DBP/100)  0.2724  (0.188226–0.356575)   ln(Total cholesterol/10)  0.514695  (0.27381–0.75558)   (Total cholesterol/10)0.5  −1.05803  (−1.86382 to −0.25223)   ln(HDL)  0.073489  (0.04377–0.103208)   (HDL)3  −0.02384  (−0.02699 to −0.02069)   (LDL/10)0.5  −0.55634  (−0.67239 to −0.44028)   ln(LDL/10) × (LDL/10)0.5  −0.83161  (−1.01001 to −0.65322)   Constant  −3.80246  (−4.67529 to −2.92963)  Cardiovascular rehospitalization   Age ≥ 70 y  0.90054  (0.86384–0.93724)   Male  0.22328  (0.188299–0.258261)   HbA1c ≥ 57 mmol/mol (7.4%)  0.004076  (−0.0294 to 0.037547)   (BMI/10)−2  −4.17347  (−4.62492 to −3.72202)   (BMI/10)3  0.001821  (0.001318–0.002324)   SBP/100)2  −1.16118  (−1.46728 to −0.85507)   SBP/100)3  0.773551  (0.637616–0.909486)   (DBP/100)−2  0.5875  (0.439237–0.735763)   (DBP/100)−2 × ln(DBP/100)  0.4095  (0.260667–0.558332)   (Total cholesterol/10)−2  −0.00798  (−0.01031 to −0.00565)   (Total cholesterol/10)2  −0.02734  (−0.23117 to 0.176482)   ln(HDL/10)  0.051443  (0.004285–0.0986)   (HDL/10)3  −0.02718  (−0.03277 to −0.02159)   LDL/10  −1.34491  (−1.56307 to −1.12675)   ln(LDL/10)  −0.88347  (−1.28497 to −0.48196)   Constant  −4.55873  (−4.8866 to −4.23086)  Abbreviation: ln, natural logarithm. View Large Figure 1. View largeDownload slide Assessing calibration in the (left) derivation and (right) validation cohorts for cardiovascular (upper) hospitalization and (lower) rehospitalization. Figure 1. View largeDownload slide Assessing calibration in the (left) derivation and (right) validation cohorts for cardiovascular (upper) hospitalization and (lower) rehospitalization. Table 3. Model Diagnostics (With 95% CI) Measure  Derivation  Validation  Apparent Performance  Test Performance  Average Optimism  Optimism Corrected  Cardiovascular hospitalization     C statistic  0.7163 (0.7136–0.7190)  0.7027 (0.6996–0.7058)  +0.0069  0.7094 (0.7067–0.7205)  0.7092 (0.7033–0.7151)   Calibration slope  1.0000 (0.9806–1.0194)  0.9933 (0.9899–0.9966)  +0.0067  0.9933 (0.9739–1.0127)  1.0001 (0.9807–1.0195)  Cardiovascular rehospitalization     C statistic  0.7154 (0.7113–0.7195)  0.7136 (0.7105–0.7167)  +0.0036  0.7118 (0.7077–0.7159)  0.7098 (0.7014–0.7182)   Calibration slope  1.0000 (0.9766–1.0234)  0.9976 (0.9949–1.0003)  +0.0024  0.9976 (0.9742–0.9796)  0.9981 (0.9948–1.0482)  Measure  Derivation  Validation  Apparent Performance  Test Performance  Average Optimism  Optimism Corrected  Cardiovascular hospitalization     C statistic  0.7163 (0.7136–0.7190)  0.7027 (0.6996–0.7058)  +0.0069  0.7094 (0.7067–0.7205)  0.7092 (0.7033–0.7151)   Calibration slope  1.0000 (0.9806–1.0194)  0.9933 (0.9899–0.9966)  +0.0067  0.9933 (0.9739–1.0127)  1.0001 (0.9807–1.0195)  Cardiovascular rehospitalization     C statistic  0.7154 (0.7113–0.7195)  0.7136 (0.7105–0.7167)  +0.0036  0.7118 (0.7077–0.7159)  0.7098 (0.7014–0.7182)   Calibration slope  1.0000 (0.9766–1.0234)  0.9976 (0.9949–1.0003)  +0.0024  0.9976 (0.9742–0.9796)  0.9981 (0.9948–1.0482)  Abbreviation: CI, confidence interval. View Large External validation In the external validation cohort, the absolute risks for cardiovascular hospitalization and rehospitalization were 16.3% and 7.0%, respectively. The application of our final risk-prediction model to the independent population gave a C statistic of 0.7092 (0.7033 to 0.7151) for cardiovascular hospitalization and 0.7098 (0.7014 to 0.7182) for cardiovascular rehospitalization and good calibration (Fig. 1, upper right for cardiovascular hospitalization and lower right for cardiovascular rehospitalization), with the calibration slope 1.0001 (0.9807 to 1.0195) and 0.9981 (0.9948 to 1.0482) for cardiovascular hospitalization and rehospitalization, respectively. Performance at the threshold for 10% and 20% of patients at highest risk Table 4 shows the sensitivity, specificity, and observed risk for the 5%, 10%, 15%, 20%, and 25% of patients at the highest predicted risk of each outcome in the validation cohort, shown for illustrative purposes. For example, when a risk threshold of 24.53% for cardiovascular hospitalization and 7.93% for cardiovascular rehospitalization is used to identify the 20% at the highest predicted risk, the sensitivity was 33.40% for cardiovascular hospitalization and 45.20% for cardiovascular rehospitalization, the specificity was 84.60% for cardiovascular hospitalization and 75.90% for cardiovascular rehospitalization, and the observed risk was 30.09% for cardiovascular hospitalization and 11.98% for cardiovascular rehospitalization, respectively. Table 4. Predicted Risk of Cardiovascular Hospitalization and Rehospitalization in the Validation Cohort Based on Various Cut-Offs   Cut-Off, %, for Risk  Mean Predicted Risk, %  Sensitivity, %  Specificity, %  Positive Predictive Value, %  Observed Risk, %  Cardiovascular hospitalization               Top 5%  38.17  51.96  10.30 (9.70–10.90)  97.40 (97.20–97.50)  43.50 (41.50–45.50)  43.48   Top 10%  31.73  43.35  17.50 (16.80–18.30)  94.60 (94.40–94.80)  38.60 (37.20–40.10)  38.62   Top 15%  27.54  37.71  24.70 (23.90–25.60)  90.10 (89.80–90.40)  32.80 (31.80–33.90)  32.83   Top 20%  24.53  33.77  34.00 (33.10–35.00)  84.60 (84.20–84.90)  30.10 (29.20–31.00)  30.09   Top 25%  22.22  31.05  42.80 (41.80–43.80)  78.40 (78.00–78.70)  27.90 (27.20–28.60)  27.89  Cardiovascular rehospitalization               Top 5%  11.34  15.86  26.20 (24.90–27.50)  91.20 (91.00–91.50)  18.30 (17.40–19.30)  18.33   Top 10%  9.67  13.63  34.50 (33.10–36.00)  84.30 (84.00–84.60)  14.20 (13.50–14.90)  14.22   Top 15%  8.69  12.59  40.50 (39.00–42.00)  79.10 (78.80–79.50)  12.70 (12.20–13.30)  12.73   Top 20%  7.93  12.02  45.20 (43.70–46.70)  75.90 (75.50–76.30)  12.40 (11.90–12.90)  12.37   Top 25%  7.16  11.46  50.00 (48.50–51.50)  72.40 (72.00–72.70)  12.00 (11.50–12.50)  11.98    Cut-Off, %, for Risk  Mean Predicted Risk, %  Sensitivity, %  Specificity, %  Positive Predictive Value, %  Observed Risk, %  Cardiovascular hospitalization               Top 5%  38.17  51.96  10.30 (9.70–10.90)  97.40 (97.20–97.50)  43.50 (41.50–45.50)  43.48   Top 10%  31.73  43.35  17.50 (16.80–18.30)  94.60 (94.40–94.80)  38.60 (37.20–40.10)  38.62   Top 15%  27.54  37.71  24.70 (23.90–25.60)  90.10 (89.80–90.40)  32.80 (31.80–33.90)  32.83   Top 20%  24.53  33.77  34.00 (33.10–35.00)  84.60 (84.20–84.90)  30.10 (29.20–31.00)  30.09   Top 25%  22.22  31.05  42.80 (41.80–43.80)  78.40 (78.00–78.70)  27.90 (27.20–28.60)  27.89  Cardiovascular rehospitalization               Top 5%  11.34  15.86  26.20 (24.90–27.50)  91.20 (91.00–91.50)  18.30 (17.40–19.30)  18.33   Top 10%  9.67  13.63  34.50 (33.10–36.00)  84.30 (84.00–84.60)  14.20 (13.50–14.90)  14.22   Top 15%  8.69  12.59  40.50 (39.00–42.00)  79.10 (78.80–79.50)  12.70 (12.20–13.30)  12.73   Top 20%  7.93  12.02  45.20 (43.70–46.70)  75.90 (75.50–76.30)  12.40 (11.90–12.90)  12.37   Top 25%  7.16  11.46  50.00 (48.50–51.50)  72.40 (72.00–72.70)  12.00 (11.50–12.50)  11.98  View Large Clinical examples Supplemental Chart 1 gives a clinical example of the application of prognostic score charts with graphical illustrations for cardiovascular hospitalization and rehospitalization risk-prediction models to predict a 2-year risk of cardiovascular hospitalization and risk of rehospitalization within 90 days of a prior cardiovascular hospitalization. Discussion We have developed two risk-prediction models to estimate the absolute risk of cardiovascular hospitalization within 2 years and cardiovascular rehospitalization after 90 days of prior cardiovascular hospitalization in a cohort of patients with type 2 diabetes in England. We then externally validated this model in another English cohort. The two prediction models had excellent calibration and useful discrimination, with C statistics of >0.70, both in the derivation cohort and external validation cohort. The two prediction models were built from clinical variables, usually recorded and accessible in primary care settings, implying that they can be readily applied in routine primary care. Strengths and limitations Our two risk algorithms have several advantages over those in use in many developed countries. Our models are based on absolute risks, determined and validated in two independent populations. The models are developed from routinely recorded demographic and clinical measurements in primary care settings, which suggests that they can be straightforwardly applied in general practice and are readily amenable for further external validations in countries that have routine recorded data accessible for such aims. Furthermore, the two risk algorithms can be easily integrated into online calculators for implementation in general practices. The methods used to derive and validate the model are similar to those for other risk-prediction algorithms derived from the Clinical Practice Research Datalink and QResearch databases (21, 22). The majority of predictors in our final model are accurate and reliable clinical measurements (23), routinely recorded in primary care settings and updated and reviewed for patients with type 2 diabetes, and are less varied than in other datasets. Moreover, the proportion of missing values was low, which would lead to little variation in external applications, although multiple imputation was still applied in our study. We acknowledge that our prediction models do not take into account diabetes duration, antidiabetes treatments, antihypertensive treatments, prior history of CVDs, other diabetes complications (e.g., renal failure), lifestyle risk factors (such as smoking), and other comorbidities as a result of limitations in the original data, but we feel that the clinical measurements included in our models could be proxies for missing predictors. Data limitations also prevented the extension of our model to all diabetes complications rather than those relating to cardiovascular hospitalization. The relatively low sensitivities of our models to identify individuals at high risk of cardiovascular hospitalization and rehospitalization is another limitation of the study. As a result of the similarity between the derivation and validation cohorts, further external validation (e.g., cohorts from other countries) is warranted. Comparison with other studies Nirantharakumar et al. (5) developed a prediction model among patients with diabetes to estimate adverse events (either excessive length of stay or inpatient mortality) over 4 years using a secondary care dataset in Birmingham, England. The predictors applied in this model covered demographic characteristics, clinical pathological test results, and use of insulin, recorded within 72 hours of hospitalization. That population represented the people with at least previous inpatient hospitalization and probably reflects a cohort with more severe conditions and likely higher prior probabilities of an event. The ranges of clinical measurements during a hospital admission would tend to be greater than in the community, as patients would be sicker and, e.g., blood glucose control could be the reason for hospitalization or exacerbated by acute illness, making the dataset difficult to use as a basis for a prediction tool in routine care. Most importantly, this prediction model has not been externally validated, and the model performance needs to be evaluated further in external populations before its application in clinical practices. Rubin et al. (6) developed a tool to predict the risk of all-cause readmission within 30 days among hospitalized patients with diabetes using hospitalized data. The short time gap between predictor measurements and outcome made the tool less useful for clinical practice. The reasons for hospitalization could be quite mixed, with different pathway and potential interventions. Therefore, the use of the all-cause hospitalization risk as the outcome provides different information and allows less-targeted interventions. As with the model of Nirantharakumar et al. (5), this model has also not been externally validated in any independent population. Previous studies have not focused on CVD as both a major cause and cost for hospital admission among patients with diabetes. To understand the potential risk of cardiovascular hospitalization in the next year and the risk of a new episode (within 90 days) of a cardiovascular event (rehospitalization), it could be helpful for clinicians to facilitate tailored, more intensive care to those with high-risk profiles and to reduce hospitalization inpatient cost. Conclusion and policy implication Our study developed prediction tools to estimate the 2-year risk of cardiovascular hospitalization and rehospitalization within 90 days of a previous hospitalization. Our two prediction models have two important implications for clinical practice. First, they can be used as tools to screen populations at high risk of cardiovascular hospitalization and rehospitalization. Both algorithms are based on readily accessible clinical data, routinely recorded in primary care and reviewed by diabetes management teams. They can be readily integrated into primary care computer systems or developed into an application for a handheld device for ease of use. Secondly, our risk-prediction models could be used to establish different treatment thresholds in clinical practice through consensus development of national guidelines. Abbreviations: BMI body mass index CVD cardiovascular disease DBP diastolic blood pressure HbA1c hemoglobin A1c HDL high-density lipoprotein ICD International Classification of Diseases LDL low-density lipoprotein NHS National Health Service SBP systolic blood pressure. Acknowledgments We thank Toby Prevost, Chris Bunn, Simon Cohn, Sarah Donald, Charlotte Paddison, Candice Ward, Peers for Progress, West Anglia CLRN, Cambridgeshire & Peterborough PCT, Primary Care Research Network–East of England, Eastern Diabetes Research Network, MRC Epidemiology Unit, participating general practices, Jackie Williams, Caroline Taylor, Kym Mercer, Kevin Baker, Ben Bowers, Kalsoom Akhter (CUH Wolfson Diabetes & Endocrinology Clinic), James Brimicombe (Cambridge University), Kim Birch of Trumpington St General Practice, CUH Wolfson Diabetes & Endocrinology Clinic Educators, The RAPSID Patient Committee (Phillip Jones, Liz Carvlin, and Roger Smith), and the peers and peer-support participants. The views expressed are those of the authors and not necessarily those of the National Health Service, National Institute for Health Research, or Department of Health. Financial Support: This work was supported by National Institute for Health Research under its Research for Patient Benefit Programme Grants PB-PG-0808-17303 and PB-PG-0610-22311. Disclosure Summary: The authors have nothing to disclose. References 1. American Diabetes Association. Standards of medical care in diabetes--2012 [published erratum appears in Diabetes Care. 35(3): 660–660]. Diabetes Care . 2012; 35( Suppl 1): S11– S63. CrossRef Search ADS PubMed  2. Sampson MJ, Dozio N, Ferguson B, Dhatariya K. Total and excess bed occupancy by age, specialty and insulin use for nearly one million diabetes patients discharged from all English acute hospitals. Diabetes Res Clin Pract . 2007; 77( 1): 92– 98. Google Scholar CrossRef Search ADS PubMed  3. Simmons D, English P, Robins P, Craig A, Addicott R. Should diabetes be commissioned through multidisciplinary networks, rather than practice based commissioning? Prim Care Diabetes . 2011; 5( 1): 39– 44. Google Scholar CrossRef Search ADS PubMed  4. Vamos EP, Millett C, Parsons C, Aylin P, Majeed A, Bottle A. Nationwide study on trends in hospital admissions for major cardiovascular events and procedures among people with and without diabetes in England, 2004–2009. Diabetes Care . 2012; 35( 2): 265– 272. Google Scholar CrossRef Search ADS PubMed  5. Nirantharakumar K, Hemming K, Narendran P, Marshall T, Coleman JJ. A prediction model for adverse outcome in hospitalized patients with diabetes. Diabetes Care . 2013; 36( 11): 3566– 3572. Google Scholar CrossRef Search ADS PubMed  6. Rubin DJ, Handorf EA, Golden SH, Nelson DB, McDonnell ME, Zhao H. Development and validation of a novel tool to predict hospital readmission risk among patients with diabetes. Endocr Pract . 2016; 22( 10): 1204– 1215. Google Scholar CrossRef Search ADS PubMed  7. Khalid JM, Raluy-Callado M, Curtis BH, Boye KS, Maguire A, Reaney M. Rates and risk of hospitalisation among patients with type 2 diabetes: retrospective cohort study using the UK General Practice Research Database linked to English Hospital Episode Statistics. Int J Clin Pract . 2014; 68( 1): 40– 48. Google Scholar CrossRef Search ADS PubMed  8. Yu D, Simmons D. Association between blood pressure and risk of cardiovascular hospital admissions among people with type 2 diabetes. Heart . 2014; 100( 18): 1444– 1449. Google Scholar CrossRef Search ADS PubMed  9. Yu D, Simmons D. Relationship between HbA1c and risk of all-cause hospital admissions among people with Type 2 diabetes. Diabet Med . 2013; 30( 12): 1407– 1411. Google Scholar CrossRef Search ADS PubMed  10. Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S, Riley RD, Hemingway H, Altman DG, PROGRESS Group. Prognosis research strategy (PROGRESS) 3: prognostic model research. PLoS Med . 2013; 10( 2): e1001381. Google Scholar CrossRef Search ADS PubMed  11. Hemingway H, Croft P, Perel P, Hayden JA, Abrams K, Timmis A, Briggs A, Udumyan R, Moons KG, Steyerberg EW, Roberts I, Schroter S, Altman DG, Riley RD, PROGRESS Group. Prognosis research strategy (PROGRESS) 1: a framework for researching clinical outcomes. BMJ . 2013; 346: e5595. Google Scholar CrossRef Search ADS PubMed  12. Simmons D, Prevost AT, Bunn C, Holman D, Parker RA, Cohn S, Donald S, Paddison CA, Ward C, Robins P, Graffy J. Impact of community based peer support in type 2 diabetes: a cluster randomised controlled trial of individual and/or group approaches. PLoS One . 2015; 10( 3): e0120277. Google Scholar CrossRef Search ADS PubMed  13. Simmons D, Cohn S, Bunn C, Birch K, Donald S, Paddison C, Ward C, Robins P, Prevost AT, Graffy J. Testing a peer support intervention for people with type 2 diabetes: A pilot for a randomised controlled trial. BMC Fam Pract . 2013; 14: 5. Google Scholar CrossRef Search ADS PubMed  14. Fisher EB, Ayala GX, Ibarra L, Cherrington AL, Elder JP, Tang TS, Heisler M, Safford MM, Simmons D; Peers for Progress Investigator Group. Contributions of peer support to health, health care, and prevention: Papers from peers for progress. Ann Fam Med . 2015; 13( Suppl 1): S2– S8. Google Scholar CrossRef Search ADS PubMed  15. Dale JR, Williams SM, Bowyer V. What is the effect of peer support on diabetes outcomes in adults? A systematic review. Diabet Med . 2012; 29( 11): 1361– 1377. Google Scholar CrossRef Search ADS PubMed  16. Simmons D, Yu D, Wenzel H. Changes in hospital admissions and inpatient tariff associated with a Diabetes Integrated Care Initiative: preliminary findings. J Diabetes . 2014; 6( 1): 81– 89. Google Scholar CrossRef Search ADS PubMed  17. Rubin DB. Multiple Imputation for Nonresponse in Surveys. Hoboken, NJ: John Wiley & Sons, Inc.; 1987. 18. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol . 1996; 49( 12): 1373– 1379. Google Scholar CrossRef Search ADS PubMed  19. Yu D, Simmons D. Association between pulse pressure and risk of hospital admissions for cardiovascular events among people with type 2 diabetes: a population-based case-control study. Diabet Med . 2015; 32( 9): 1201– 1206. Google Scholar CrossRef Search ADS PubMed  20. Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, Vickers AJ, Ransohoff DF, Collins GS. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med . 2015; 162( 1): W1– W73. Google Scholar CrossRef Search ADS PubMed  21. Sultan AA, West J, Grainge MJ, Riley RD, Tata LJ, Stephansson O, Fleming KM, Nelson-Piercy C, Ludvigsson JF. Development and validation of risk prediction model for venous thromboembolism in postpartum women: multinational cohort study. BMJ . 2016; 355: i6253. Google Scholar CrossRef Search ADS PubMed  22. Hippisley-Cox J, Coupland C. Derivation and validation of updated QFracture algorithm to predict risk of osteoporotic fracture in primary care in the United Kingdom: prospective open cohort study. BMJ . 2012; 344: e3427. Google Scholar CrossRef Search ADS PubMed  23. Hemke AC, Heemskerk MB, van Diepen M, Dekker FW, Hoitsma AJ. Improved mortality prediction in dialysis patients using specific clinical and laboratory data. Am J Nephrol . 2015; 42( 2): 158– 167. Google Scholar CrossRef Search ADS PubMed  Copyright © 2018 Endocrine Society

Journal

Journal of Clinical Endocrinology and MetabolismOxford University Press

Published: Mar 1, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off