Development and Validation of the Nursing Home Minimum Data Set 3.0 Mortality Risk Score (MRS3)

Development and Validation of the Nursing Home Minimum Data Set 3.0 Mortality Risk Score (MRS3) Abstract Background To develop a score to predict mortality using the Minimum Data Set 3.0 (MDS 3.0) that can be readily calculated from items collected during nursing home (NH) residents’ admission assessments. Participants We developed a training cohort of Medicare beneficiaries newly admitted to United States NHs during 2012 (N = 1,426,815) and a testing cohort from 2013 (N = 1,160,964). Methods Data came from the MDS 3.0 assessments linked to the Medicare Beneficiary Summary File. Using the training dataset, we developed a composite MDS 3.0 Mortality Risk Score (MRS3) consisting of 17 clinical items and patients’ age groups based on their relation to 30-day mortality. We assessed the calibration and discrimination of the MRS3 in predicting 30- and 60-day mortality and compared its performance to the Charlson Comorbidity Index and the clinician’s assessment of 6-month prognosis measured at admission. Results The 30- and 60-day mortality rates for the testing population were 2.8% and 5.6%, respectively. Results from logistic regression models suggest that the MRS3 performed well in predicting death within 30 and 60 days (C-Statistics of 0.744 [95% confidence limit (CL) = 0.741, 0.747] and 0.709 [95% CL = 0.706, 0.711], respectively). The MRS3 was a superior predictor of mortality compared to the Charlson Comorbidity Index (C-statistics of 0.611 [95% CL = 0.607, 0.615] and 0.608 [95% CL = 0.605, 0.610]) and the clinicians’ assessments of patients’ 6-month prognoses (C-statistics of 0.543 [95% CL = 0.542, 0.545] and 0.528 [95% CL = 0.527, 0.529]). Conclusions The MRS3 is a good predictor of mortality and can be useful in guiding decision-making, informing plans of care, and adjusting for patients’ risk of mortality. Prognosis, Nursing home, Mortality, Risk adjustment Nursing homes (NHs) provide care to patients with a wide spectrum of clinical care needs, many of whom are in their last year of life (1,2). Understanding the clinical profile and risk of mortality upon NH admission can stimulate discussions designed to establish patients’ goals of care, as well as guide end-of-life decision-making and advance care planning with patients and family members (3,4). However, predicting a patient’s survival is fraught with difficulty and notoriously imprecise. Recognizing this difficulty, many mortality risk scores and models have been developed to assist physicians outside of the NH setting (eg, GRACE (5), TIMI (6), Ranson’s criteria for pancreatic mortality (7), Surgical Apgar Scale (8–10), Veterans Aging Cohort Study (VACS) index (11,12), and POMPE-C Tool) (13,14). In addition to their utility for clinical decision-making, mortality risk scores and prediction models are also used to adjust for a patient’s risk of adverse outcomes and as inclusion criteria in randomized control trials (15–17). Furthermore, mortality risk scores can be useful in calibrating facility case mix, thereby allowing for more valid comparisons of outcomes across providers (18). As such, they are often used as adjusters in outcomes research and publicly reported quality measures (19). While it is important to have valid scoring systems that predict mortality and signal a referral for advanced care planning or hospice, it is critically important that such systems are constructed from readily available data routinely obtained in clinical practice. The mandated NH resident assessment process, utilizing the Minimum Data Set version 3.0 (MDS 3.0) instrument (20), captures rich clinical and functional information about patients to inform clinical decision-making and care planning. Previous efforts to develop mortality risk scores for the NH setting have relied on sub-samples of patients (eg, those with advanced dementia) (21,22) and items derived from a previous version of the MDS resident assessment data (MDS 2.0) (23–30). However, many of the items in the MDS 2.0 used in previous mortality risk scores were eliminated in the new version of the MDS implemented in October 2010. As such, we have not had a mortality risk score for the NH population since the introduction of the MDS 3.0. The purpose of this study was to develop and evaluate a mortality risk score based upon a national population of newly admitted NH patients, which can be readily calculated from items collected during the admission assessment process using the MDS 3.0 in order to guide decision-making, better inform plans of care, and adjust for facility case-mix severity. Method Data Data for this study come from the MDS 3.0 and the Medicare Master Beneficiary Summary File. The MDS instrument is a standardized, comprehensive assessment that must be completed for all persons who receive care in a Medicare and/or Medicaid-certified nursing facility. The MDS captures information about residents’ physical, cognitive, psychological, and psychosocial functioning, in addition to medical diagnoses, clinical symptoms, and any treatments or therapies prescribed and received. Assessments are conducted on all residents at admission and discharge, in addition to other time intervals (eg, quarterly and annually). While initially designed for care planning, the MDS is also used for payment, quality monitoring, and research. The MDS 3.0, the current version of the assessment instrument, has been extensively validated (31–38). From the MDS 3.0, we obtained the first full assessment after its introduction (October 1, 2010) for each individual. We linked individual’s first MDS assessment from 2012–2013 to the 2012–2014 Medicare Master Summary Beneficiary File to identify NH residents who were Medicare beneficiaries and to determine beneficiaries’ dates of death. Importantly, the Medicare Master Beneficiary Summary File allows us to identify NH residents who die, regardless of where or when. Study Samples With the MDS 3.0, we identified a cohort of Medicare beneficiaries who were newly admitted to the NH in 2012 and 2013 with no NH stay since the introduction of the MDS 3.0 in October 2010. We split the sample to derive “training” and “testing” populations. The “training” population used to develop the prediction model consisted of all Medicare beneficiaries newly admitted to NHs during 2012 (N = 1,426,815). The population used to “test” the sensitivity and specificity of the prediction model included all Medicare beneficiaries newly admitted to NHs during 2013 (N = 1,160,964). Study Variables Measures of mortality were created to examine whether the new resident admitted to the NH died within 30 days and within 60 days of the assessment. Thirty-day mortality was used to develop the score and we tested the performance of the score using the 30- and 60-day measures of mortality. To develop the score, the research team, including a practicing geriatrician and palliative care physician, identified clinical indicators that were reported across assessments in the MDS 3.0 believed to be related to the likelihood of death. A total of 46 candidate measures was selected and grouped into three categories: 1) Cognitive and Physical Function, 2) Diagnoses, and 3) Health Conditions (see Supplementary Table 1, for a complete list of measures). For ease of calculation, we dichotomized ordinal variables measuring cognitive and physical function. The cognitive and physical function variables were dichotomized into categories representing severe functional limitation (a 28-point Activities of Daily Living [ADL] (39) score greater than or equal to 23) and severe cognitive impairment (a Cognitive Function Scale score of 4) (37). We collapsed variables that measured a similar construct. For example, three shortness of breath measures (lying flat, sitting, and with exertion) were collapsed into one measure to capture whether or not patients experienced any of these symptoms. We also created a composite measure to indicate if a patient experienced any swallowing disorder, and a composite measure to indicate if a patient had at least one stage 3, stage 4, or unstageable pressure ulcer. Analyses Approach to develop a mortality risk score With the training population, we conducted bivariate analyses, with facility fixed effects to remove any influence of the facility on mortality, predicting death at 30 days for each of the measures that were candidates for inclusion in the score. Only statistically significant variables (p < .0001) with odds ratios greater than 1.2 were retained for the multivariate models. A total of 21 clinical indicators were entered into a backwards, stepwise regression model predicting death within 30 days. Variables with a p-value greater than .2 were dropped from the model. The remaining 17 measures were tested in a logistic regression model with facility fixed effects predicting death within 30 days. We calculated the adjusted odds ratio for each variable to estimate the risk of death for a given variable controlling for coexistent cognitive and physical function, diagnoses, and health conditions. We created a weighted score, the MDS 3.0 Mortality Risk Score (MRS3), to represent the number and seriousness of a patient’s clinical indicators in addition to a patient’s age. Specifically, the adjusted odds ratios were employed as weights for the different clinical indicators. Indicators with an odds ratio of ≥1.2 and <1.5 were assigned a weight of 1 (n = 5); indicators with an odds ratio of ≥1.5 and <2.5 a weight of 2 (n = 7); indicators with an odds ratio of ≥2.5 a weight of 3 (n = 5). Age groups were assigned the following weighted values: Ages <65 = 1, 65–74 = 2, 75–84 = 3, 85–94 = 4, and 95+ = 5. (See Table 1, for a list of indicators and weighted values). Weighted values were summed for each individual to create a score that could range from 1 to 39 with 1 being least and 39 being the highest risk of mortality. However, given the very small number of individuals with a score greater than 20 (less than 0.001%), we censored the higher values of the score so that any above 20 were set to 20. Table 1. Scoring the MRS3 Component  MRS3 Value  Age <65 years  1  Age 65–74 years  2  Age 75–84 years  3  Age 85–94 years  4  Age 95+ years  5  Severe physical impairmenta  3  Acute change in mental status  3  Shortness of breath, any  3  Dehydrated  3  Severe cognitive impairmentb  3  Internal bleeding  2  Swallowing disorder, any  2  Vomiting  2  Fever  2  Unplanned weight loss  2  Heart Failure  2  Worst pressure ulcer, slough, or necrotic  2  Pneumonia  1  Any falls since admission or prior assessment  1  Viral hepatitis  1  Hyponatremia  1  At least one stage 3, 4, or unstageable pressure ulcer  1  Component  MRS3 Value  Age <65 years  1  Age 65–74 years  2  Age 75–84 years  3  Age 85–94 years  4  Age 95+ years  5  Severe physical impairmenta  3  Acute change in mental status  3  Shortness of breath, any  3  Dehydrated  3  Severe cognitive impairmentb  3  Internal bleeding  2  Swallowing disorder, any  2  Vomiting  2  Fever  2  Unplanned weight loss  2  Heart Failure  2  Worst pressure ulcer, slough, or necrotic  2  Pneumonia  1  Any falls since admission or prior assessment  1  Viral hepatitis  1  Hyponatremia  1  At least one stage 3, 4, or unstageable pressure ulcer  1  Notes: Weighted values are summed to create the MDS 3.0 Mortality Risk Score (MRS3) that could range from 1 to 39 with 1 being least and 39 being the highest risk of mortality. A MRS3 score of 8 or higher is associated with a 5.4 times increase in the odds of 30-day mortality compared to a score of 7 or lower (OR = 5.408, CI = 5.726, 5.543) and a 4.2 times increase in the odds of 60-day mortality (OR = 4.215, CI = 4.140, 4.291). aActivities of Daily Living Score greater than or equal to 23 (range 0–28, 0 = total independence, 28 = total dependence). bCognitive Function Scale score of 4 (range 1–4; 1 = intact cognition, 4 = severe cognitive impairment). View Large Table 1. Scoring the MRS3 Component  MRS3 Value  Age <65 years  1  Age 65–74 years  2  Age 75–84 years  3  Age 85–94 years  4  Age 95+ years  5  Severe physical impairmenta  3  Acute change in mental status  3  Shortness of breath, any  3  Dehydrated  3  Severe cognitive impairmentb  3  Internal bleeding  2  Swallowing disorder, any  2  Vomiting  2  Fever  2  Unplanned weight loss  2  Heart Failure  2  Worst pressure ulcer, slough, or necrotic  2  Pneumonia  1  Any falls since admission or prior assessment  1  Viral hepatitis  1  Hyponatremia  1  At least one stage 3, 4, or unstageable pressure ulcer  1  Component  MRS3 Value  Age <65 years  1  Age 65–74 years  2  Age 75–84 years  3  Age 85–94 years  4  Age 95+ years  5  Severe physical impairmenta  3  Acute change in mental status  3  Shortness of breath, any  3  Dehydrated  3  Severe cognitive impairmentb  3  Internal bleeding  2  Swallowing disorder, any  2  Vomiting  2  Fever  2  Unplanned weight loss  2  Heart Failure  2  Worst pressure ulcer, slough, or necrotic  2  Pneumonia  1  Any falls since admission or prior assessment  1  Viral hepatitis  1  Hyponatremia  1  At least one stage 3, 4, or unstageable pressure ulcer  1  Notes: Weighted values are summed to create the MDS 3.0 Mortality Risk Score (MRS3) that could range from 1 to 39 with 1 being least and 39 being the highest risk of mortality. A MRS3 score of 8 or higher is associated with a 5.4 times increase in the odds of 30-day mortality compared to a score of 7 or lower (OR = 5.408, CI = 5.726, 5.543) and a 4.2 times increase in the odds of 60-day mortality (OR = 4.215, CI = 4.140, 4.291). aActivities of Daily Living Score greater than or equal to 23 (range 0–28, 0 = total independence, 28 = total dependence). bCognitive Function Scale score of 4 (range 1–4; 1 = intact cognition, 4 = severe cognitive impairment). View Large Approach to External Validation We assessed the accuracy of the score with measures of discrimination and calibration. To do this, we conducted logistic regression analyses with the MRS3 to predict the odds of death at 30 and 60 days. We tested the discrimination of the score using the area under the receiver operating characteristic (ROC) curve (or concordance statistic, C). An area under the ROC curve of 0.5 indicates no discrimination, whereas a C-Statistic of 1 represents perfect discrimination. For comparison of the score’s discrimination to other established measures, we conducted separate logistic regression analyses using the Charlson Comorbidity Index and the 6-month prognosis indicator in the MDS assessment to predict death within 30 and 60 days. The Charlson Comorbidity Index is derived using the list of active diagnoses that are reported in the MDS 3.0 assessment. The 6-month prognosis variable is an indicator that is completed by a clinician and reported in the MDS 3.0 assessment. This variable is used to identify whether or not the patient has a life limiting illness or condition and is not expected to survive longer than 6 months. We also tested performance of the MRS3 in terms of calibration. Calibration refers to how closely the predicted mortality risk agrees with observed mortality. We assessed calibration of the MRS3 by plotting the observed proportions versus predicted probabilities of death at 30 and 60 days and by tabulating the 30-day mortality ratio (predicted to actual) for each value of the MRS3 score. We used the Youden Index (J) (40) to identify a cut point that discriminates between low versus high risk of mortality and examined its performance in logistic regression models predicting 30- and 60-day mortality. Sensitivity Analyses In sensitivity analyses, we tested a logistic regression model predicting mortality using the detailed coefficients for each indicator. Results Table 2 describes the training sample of newly admitted NH patients in terms of their demographic, clinical, and functional characteristics as well as the observed 30-, 60-day, and 1-year mortality rates. The mortality rate for the testing population was 2.9% within 30 days and 5.7% within 60 days. The mean MRS3 score was 4.74 (SD = 2.49) with a median of 4. Over one-third of the testing population was 85 years or older and the most frequently reported indicator was shortness of breath (18.34%) followed by heart failure (18.14%). The bivariate relationship with mortality for each variable included in the MRS3 can be found in the Supplementary Table 1. Table 2. Descriptive Characteristics of Sample   Training Population: New Admissions, 2012  Testing Population: New Admissions, 2013  Number of observations (N)  1,426,815  1,160,964   30-day mortality (%)  5.72  2.88   60-day mortality (%)  9.76  5.69   365-day mortality (%)  25.55  21.94  Age groups   Age: <65 (%)  9.08  9.17   Age: 65–74 (%)  21.84  22.3   Age: 75–84 (%)  35.1  34.78   Age: 85–94 (%)  30.46  30.26   Age: 95+ (%)  3.53  3.49  Demographics   Male  36.88  31.10   American Indian/Alaska  Native  0.34  0.30   Asian  1.37  1.54   Black  9.04  9.62   Hispanic or Latino  3.76  4.09   Native Hawaiian or Pacific  Islander  0.25  0.28   White  82.81  80.88  Cognitive and physical function   Severe cognitive impairmenta (%)  3.62  3.01   Acute change in mental status (%)  2.14  1.59   Severe Physical Impairmentb (%)  7.95  6.47   Any falls since admission or  prior assessment (%)  6.99  7.2  Diagnoses   Heart failure (%)  18.20  18.14   Pneumonia (%)  9.1  8.74   Hyponatremia (%)  2.24  2.3  Health conditions       Dehydrated (%)  0.42  0.33   Fever (%)  2.79  2.34   Internal bleeding (%)  0.93  0.82   Vomiting (%)  2.59  2.29   Shortness of breath, any (%)  19.38  18.34   Shortness of breath when lying  flat (%)  9.04  8.53   Shortness of breath when  sitting (%)  5.64  5.01   Shortness of breath with  exertion (%)  16.08  15.23  Swallowing disorder, any (%)  5.46  4.76   Choking (%)  2.35  2.01   Painful swallowing (%)  2.76  2.48   Holding food (%)  1.19  1.01   Lost liquid (%)  0.53  0.43   Unplanned weight loss (%)  4.47  4.05   At least one stage 3, 4, or  unstageable pressure ulcer (%)  3.34  3.44   Worst pressure ulcer, Slough or  Necrotic (%)  3.15  2.99   Worst pressure ulcer:   Slough (3) (%)  1.51  1.5   Worst pressure ulcer:   Necrotic (4) (%)  1.63  1.48    Training Population: New Admissions, 2012  Testing Population: New Admissions, 2013  Number of observations (N)  1,426,815  1,160,964   30-day mortality (%)  5.72  2.88   60-day mortality (%)  9.76  5.69   365-day mortality (%)  25.55  21.94  Age groups   Age: <65 (%)  9.08  9.17   Age: 65–74 (%)  21.84  22.3   Age: 75–84 (%)  35.1  34.78   Age: 85–94 (%)  30.46  30.26   Age: 95+ (%)  3.53  3.49  Demographics   Male  36.88  31.10   American Indian/Alaska  Native  0.34  0.30   Asian  1.37  1.54   Black  9.04  9.62   Hispanic or Latino  3.76  4.09   Native Hawaiian or Pacific  Islander  0.25  0.28   White  82.81  80.88  Cognitive and physical function   Severe cognitive impairmenta (%)  3.62  3.01   Acute change in mental status (%)  2.14  1.59   Severe Physical Impairmentb (%)  7.95  6.47   Any falls since admission or  prior assessment (%)  6.99  7.2  Diagnoses   Heart failure (%)  18.20  18.14   Pneumonia (%)  9.1  8.74   Hyponatremia (%)  2.24  2.3  Health conditions       Dehydrated (%)  0.42  0.33   Fever (%)  2.79  2.34   Internal bleeding (%)  0.93  0.82   Vomiting (%)  2.59  2.29   Shortness of breath, any (%)  19.38  18.34   Shortness of breath when lying  flat (%)  9.04  8.53   Shortness of breath when  sitting (%)  5.64  5.01   Shortness of breath with  exertion (%)  16.08  15.23  Swallowing disorder, any (%)  5.46  4.76   Choking (%)  2.35  2.01   Painful swallowing (%)  2.76  2.48   Holding food (%)  1.19  1.01   Lost liquid (%)  0.53  0.43   Unplanned weight loss (%)  4.47  4.05   At least one stage 3, 4, or  unstageable pressure ulcer (%)  3.34  3.44   Worst pressure ulcer, Slough or  Necrotic (%)  3.15  2.99   Worst pressure ulcer:   Slough (3) (%)  1.51  1.5   Worst pressure ulcer:   Necrotic (4) (%)  1.63  1.48  Notes:aCognitive Function Scale score of 4 (Range 1–4; 1 = intact cognition, 4 = severe cognitive impairment). bActivities of Daily Living Score greater than or equal to 23 (Range 0–28, 0 = total independence, 28 = total dependence). View Large Table 2. Descriptive Characteristics of Sample   Training Population: New Admissions, 2012  Testing Population: New Admissions, 2013  Number of observations (N)  1,426,815  1,160,964   30-day mortality (%)  5.72  2.88   60-day mortality (%)  9.76  5.69   365-day mortality (%)  25.55  21.94  Age groups   Age: <65 (%)  9.08  9.17   Age: 65–74 (%)  21.84  22.3   Age: 75–84 (%)  35.1  34.78   Age: 85–94 (%)  30.46  30.26   Age: 95+ (%)  3.53  3.49  Demographics   Male  36.88  31.10   American Indian/Alaska  Native  0.34  0.30   Asian  1.37  1.54   Black  9.04  9.62   Hispanic or Latino  3.76  4.09   Native Hawaiian or Pacific  Islander  0.25  0.28   White  82.81  80.88  Cognitive and physical function   Severe cognitive impairmenta (%)  3.62  3.01   Acute change in mental status (%)  2.14  1.59   Severe Physical Impairmentb (%)  7.95  6.47   Any falls since admission or  prior assessment (%)  6.99  7.2  Diagnoses   Heart failure (%)  18.20  18.14   Pneumonia (%)  9.1  8.74   Hyponatremia (%)  2.24  2.3  Health conditions       Dehydrated (%)  0.42  0.33   Fever (%)  2.79  2.34   Internal bleeding (%)  0.93  0.82   Vomiting (%)  2.59  2.29   Shortness of breath, any (%)  19.38  18.34   Shortness of breath when lying  flat (%)  9.04  8.53   Shortness of breath when  sitting (%)  5.64  5.01   Shortness of breath with  exertion (%)  16.08  15.23  Swallowing disorder, any (%)  5.46  4.76   Choking (%)  2.35  2.01   Painful swallowing (%)  2.76  2.48   Holding food (%)  1.19  1.01   Lost liquid (%)  0.53  0.43   Unplanned weight loss (%)  4.47  4.05   At least one stage 3, 4, or  unstageable pressure ulcer (%)  3.34  3.44   Worst pressure ulcer, Slough or  Necrotic (%)  3.15  2.99   Worst pressure ulcer:   Slough (3) (%)  1.51  1.5   Worst pressure ulcer:   Necrotic (4) (%)  1.63  1.48    Training Population: New Admissions, 2012  Testing Population: New Admissions, 2013  Number of observations (N)  1,426,815  1,160,964   30-day mortality (%)  5.72  2.88   60-day mortality (%)  9.76  5.69   365-day mortality (%)  25.55  21.94  Age groups   Age: <65 (%)  9.08  9.17   Age: 65–74 (%)  21.84  22.3   Age: 75–84 (%)  35.1  34.78   Age: 85–94 (%)  30.46  30.26   Age: 95+ (%)  3.53  3.49  Demographics   Male  36.88  31.10   American Indian/Alaska  Native  0.34  0.30   Asian  1.37  1.54   Black  9.04  9.62   Hispanic or Latino  3.76  4.09   Native Hawaiian or Pacific  Islander  0.25  0.28   White  82.81  80.88  Cognitive and physical function   Severe cognitive impairmenta (%)  3.62  3.01   Acute change in mental status (%)  2.14  1.59   Severe Physical Impairmentb (%)  7.95  6.47   Any falls since admission or  prior assessment (%)  6.99  7.2  Diagnoses   Heart failure (%)  18.20  18.14   Pneumonia (%)  9.1  8.74   Hyponatremia (%)  2.24  2.3  Health conditions       Dehydrated (%)  0.42  0.33   Fever (%)  2.79  2.34   Internal bleeding (%)  0.93  0.82   Vomiting (%)  2.59  2.29   Shortness of breath, any (%)  19.38  18.34   Shortness of breath when lying  flat (%)  9.04  8.53   Shortness of breath when  sitting (%)  5.64  5.01   Shortness of breath with  exertion (%)  16.08  15.23  Swallowing disorder, any (%)  5.46  4.76   Choking (%)  2.35  2.01   Painful swallowing (%)  2.76  2.48   Holding food (%)  1.19  1.01   Lost liquid (%)  0.53  0.43   Unplanned weight loss (%)  4.47  4.05   At least one stage 3, 4, or  unstageable pressure ulcer (%)  3.34  3.44   Worst pressure ulcer, Slough or  Necrotic (%)  3.15  2.99   Worst pressure ulcer:   Slough (3) (%)  1.51  1.5   Worst pressure ulcer:   Necrotic (4) (%)  1.63  1.48  Notes:aCognitive Function Scale score of 4 (Range 1–4; 1 = intact cognition, 4 = severe cognitive impairment). bActivities of Daily Living Score greater than or equal to 23 (Range 0–28, 0 = total independence, 28 = total dependence). View Large Results from our logistic regression models suggest that the MRS3 performed well in predicting death within 30 and 60 days (with C-statistics of 0.744 [95% CL = 0.741, 0.747] and 0.709 [95% CL = 0.706, 0.711]), respectively (see Table 3). When comparing the performance of the MRS3 models in predicting mortality to those using either the Charlson Comorbidity Index or the clinicians’ assessment of 6-month prognosis, we found that the MRS3 was superior (see Table 3). Specifically, the C-statistic for the model using the Charlson Comorbidity Index to predict 30-day mortality was 0.611 (95% CL = 0.607, 0.615), and it was even lower for the model using a clinician’s assessment of 6-month prognosis (C = 0.543, 95% CL = 0.542, 0.545). Table 3. Performance of MRS3 in Predicting Death among the Testing Population Compared to the Charlson Comorbidity Index and the 6-Month Prognosis Indicator   30-Day Mortality  60-Day Mortality  MRS3  0.744 (0.741, 0.747)  0.709 (0.706, 0.711)  Charlson Comorbidity Index  0.611 (0.607, 0.615)  0.608 (0.605, 0.610)  6-Month Prognosis  0.543 (0.542, 0.545)  0.528 (0.527, 0.529)    30-Day Mortality  60-Day Mortality  MRS3  0.744 (0.741, 0.747)  0.709 (0.706, 0.711)  Charlson Comorbidity Index  0.611 (0.607, 0.615)  0.608 (0.605, 0.610)  6-Month Prognosis  0.543 (0.542, 0.545)  0.528 (0.527, 0.529)  Notes: Values represent the C-statistics and 95% Confidence Intervals. View Large Table 3. Performance of MRS3 in Predicting Death among the Testing Population Compared to the Charlson Comorbidity Index and the 6-Month Prognosis Indicator   30-Day Mortality  60-Day Mortality  MRS3  0.744 (0.741, 0.747)  0.709 (0.706, 0.711)  Charlson Comorbidity Index  0.611 (0.607, 0.615)  0.608 (0.605, 0.610)  6-Month Prognosis  0.543 (0.542, 0.545)  0.528 (0.527, 0.529)    30-Day Mortality  60-Day Mortality  MRS3  0.744 (0.741, 0.747)  0.709 (0.706, 0.711)  Charlson Comorbidity Index  0.611 (0.607, 0.615)  0.608 (0.605, 0.610)  6-Month Prognosis  0.543 (0.542, 0.545)  0.528 (0.527, 0.529)  Notes: Values represent the C-statistics and 95% Confidence Intervals. View Large Figure 1 plots the observed versus predicted probability of death for each value of the MRS3 score in the testing sample. The smooth, 45-degree line represents a perfect prediction model where the observed value and predicted value are the same, while the shapes represent the actual probability of death at each time period at each value of the score. Because the models calibrate to the data well, the shapes and the line lie close to each other. As can be seen in Figure 1, the MRS3 performs well in predicting mortality with the greatest deviations seen among the highest values, which have fewer observations. Figure 1. View largeDownload slide Calibration curve for observed versus predicted probability of death at 30 and 60 days. Notes: Values plotted are the average predicted and average observed probability for each level of the MRS3 (1–26). Figure 1. View largeDownload slide Calibration curve for observed versus predicted probability of death at 30 and 60 days. Notes: Values plotted are the average predicted and average observed probability for each level of the MRS3 (1–26). We present the observed and predicted 30-day mortality rates, by MRS3 value, in Table 4. Results suggest that individuals with a low MRS3 have a very low observed rate of death in the short term. For example, of the 16% of individuals with an MRS3 score of 1 or 2, less than 1% had died within 30 days. In contrast, 34% of individuals with an MRS3 score of 15 or higher had died within 30 days. Table 4. Observed and Predicted 30-Day Mortality, by MRS3 Score MRS3 Score  N  Predicted Risk of 30-Day Mortality  Observed 30-Day Mortality Rate  Predicted 30-Day Mortality  Observed 30-Day Mortality  Ratio of Observed to Expected Mortality  1  46,255  0.59  0.58  271  267  1.01  2  124,129  0.79  0.79  986  976  1.01  3  199,554  1.08  0.93  2,150  1,864  1.15  4  207,875  1.46  1.28  3,035  2,658  1.14  5  124,279  1.98  2.12  2,456  2,628  0.93  6  113,835  2.67  2.89  3,039  3,287  0.92  7  80,065  3.60  3.90  2,880  3,125  0.92  8  51,509  4.83  5.05  2,488  2,600  0.96  9  38,639  6.46  6.72  2,496  2,598  0.96  10  21,656  8.59  8.83  1,860  1,912  0.97  11  11,687  11.33  11.25  1,325  1,315  1.01  12  7,157  14.81  15.15  1,061  1,084  0.98  13  4,072  19.13  17.68  780  720  1.08  14  2,134  24.35  21.46  520  458  1.13  15  1,361  30.46  27.55  415  375  1.11  16  785  37.34  34.65  294  272  1.08  17  439  44.77  35.54  197  156  1.26  18  209  52.44  43.54  110  91  1.20  19  136  60.01  43.38  82  59  1.38  20  137  67.12  59.85  92  82  1.12  MRS3 Score  N  Predicted Risk of 30-Day Mortality  Observed 30-Day Mortality Rate  Predicted 30-Day Mortality  Observed 30-Day Mortality  Ratio of Observed to Expected Mortality  1  46,255  0.59  0.58  271  267  1.01  2  124,129  0.79  0.79  986  976  1.01  3  199,554  1.08  0.93  2,150  1,864  1.15  4  207,875  1.46  1.28  3,035  2,658  1.14  5  124,279  1.98  2.12  2,456  2,628  0.93  6  113,835  2.67  2.89  3,039  3,287  0.92  7  80,065  3.60  3.90  2,880  3,125  0.92  8  51,509  4.83  5.05  2,488  2,600  0.96  9  38,639  6.46  6.72  2,496  2,598  0.96  10  21,656  8.59  8.83  1,860  1,912  0.97  11  11,687  11.33  11.25  1,325  1,315  1.01  12  7,157  14.81  15.15  1,061  1,084  0.98  13  4,072  19.13  17.68  780  720  1.08  14  2,134  24.35  21.46  520  458  1.13  15  1,361  30.46  27.55  415  375  1.11  16  785  37.34  34.65  294  272  1.08  17  439  44.77  35.54  197  156  1.26  18  209  52.44  43.54  110  91  1.20  19  136  60.01  43.38  82  59  1.38  20  137  67.12  59.85  92  82  1.12  View Large Table 4. Observed and Predicted 30-Day Mortality, by MRS3 Score MRS3 Score  N  Predicted Risk of 30-Day Mortality  Observed 30-Day Mortality Rate  Predicted 30-Day Mortality  Observed 30-Day Mortality  Ratio of Observed to Expected Mortality  1  46,255  0.59  0.58  271  267  1.01  2  124,129  0.79  0.79  986  976  1.01  3  199,554  1.08  0.93  2,150  1,864  1.15  4  207,875  1.46  1.28  3,035  2,658  1.14  5  124,279  1.98  2.12  2,456  2,628  0.93  6  113,835  2.67  2.89  3,039  3,287  0.92  7  80,065  3.60  3.90  2,880  3,125  0.92  8  51,509  4.83  5.05  2,488  2,600  0.96  9  38,639  6.46  6.72  2,496  2,598  0.96  10  21,656  8.59  8.83  1,860  1,912  0.97  11  11,687  11.33  11.25  1,325  1,315  1.01  12  7,157  14.81  15.15  1,061  1,084  0.98  13  4,072  19.13  17.68  780  720  1.08  14  2,134  24.35  21.46  520  458  1.13  15  1,361  30.46  27.55  415  375  1.11  16  785  37.34  34.65  294  272  1.08  17  439  44.77  35.54  197  156  1.26  18  209  52.44  43.54  110  91  1.20  19  136  60.01  43.38  82  59  1.38  20  137  67.12  59.85  92  82  1.12  MRS3 Score  N  Predicted Risk of 30-Day Mortality  Observed 30-Day Mortality Rate  Predicted 30-Day Mortality  Observed 30-Day Mortality  Ratio of Observed to Expected Mortality  1  46,255  0.59  0.58  271  267  1.01  2  124,129  0.79  0.79  986  976  1.01  3  199,554  1.08  0.93  2,150  1,864  1.15  4  207,875  1.46  1.28  3,035  2,658  1.14  5  124,279  1.98  2.12  2,456  2,628  0.93  6  113,835  2.67  2.89  3,039  3,287  0.92  7  80,065  3.60  3.90  2,880  3,125  0.92  8  51,509  4.83  5.05  2,488  2,600  0.96  9  38,639  6.46  6.72  2,496  2,598  0.96  10  21,656  8.59  8.83  1,860  1,912  0.97  11  11,687  11.33  11.25  1,325  1,315  1.01  12  7,157  14.81  15.15  1,061  1,084  0.98  13  4,072  19.13  17.68  780  720  1.08  14  2,134  24.35  21.46  520  458  1.13  15  1,361  30.46  27.55  415  375  1.11  16  785  37.34  34.65  294  272  1.08  17  439  44.77  35.54  197  156  1.26  18  209  52.44  43.54  110  91  1.20  19  136  60.01  43.38  82  59  1.38  20  137  67.12  59.85  92  82  1.12  View Large We identified a cut point of 7 to indicate low versus high risk of mortality. A MRS3 score of 8 or higher was associated with a 5.4 times increase in the odds of 30-day mortality compared to a score of 7 or lower (OR = 5.408, CI = 5.726, 5.543) and a 4.2 times increase in the odds of 60-day mortality (OR = 4.215, CI = 4.140, 4.291). In sensitivity analyses, we found a relatively small loss of precision by working with the MRS3 (a simpler model and scoring system) that we developed as opposed to the logistic regression model predicting mortality using the detailed coefficients for each indicator (results from the fully weighted model can be found in the Supplementary Table 3). Discussion As NHs increasingly care for more clinically complex and frail patients, a tool to enable clinicians to identify patients who may be close to the end of life could potentially serve as a trigger for advance care planning. Furthermore, with the variation in the types of patients admitted to NHs, a score, or set of items, that strongly predict mortality is needed to adequately adjust for patients’ risk and facility case mix. Results from our development and validation study suggest that the MRS3 is a useful new tool to predict 30-day mortality and adjust for patient risk using the MDS 3.0. This mortality risk score, developed for patients newly admitted to the NH, is of benefit to clinicians and health services researchers for multiple reasons. First, the MRS3 can be a helpful tool not only to provide prognosis information that can aide in guiding care, but it can also assist with planning for care. For example, a patient’s MRS3 score can serve as a signal for end of life and advanced care planning discussions with the patient and their family. Second, the MRS3 can inform goals of care, particularly for patients with high MRS3 scores and predicted to be near the end of life. Where to establish the cut point to identify patients who should be approached about advance care planning or for whom a hospice discussion might be necessary is an issue that will require further investigation. However, our results suggest that a score of 8 and above is associated with over five times the increased odds of mortality and might serve as this cut point. While this cut point maximizes the predictive value, the decision of the cut point must take into account the goals. For example, health providers may want to focus on sensitivity rather than specificity to ensure all that would benefit from advance care planning are offered the chance to state their wishes prior to a period of possible future mental incapacity. While the way the score was developed maximizes its clinical applicability, it is also useful to researchers who will benefit from a score that can be readily calculated and used with the MDS 3.0. This score has the potential to be used as adjustment for mortality in examining the impact of a new treatment or comparing quality across facilities. As with the development of any score, there are some limitations to note. First and foremost, the MRS3 was designed as a clinical aid and clinicians may have additional knowledge that better informs prognostication (eg, knowing that a patient is stopping dialysis). This score was not designed to take the place of a clinical opinion and training for appropriate prognostication and therefore should be approached and interpreted as such. Secondly, this score is not based solely on biological-based markers or objective physiological measures; the MRS3 relies on subjective clinical appraisals for at least some of the indicators. However, MDS coordinators and those who complete the assessments have been trained to obtain this information for care planning, payment, and public reporting; therefore, it can be argued that there is some standardization and objectivity in collecting this information and the MDS has been validated for its accuracy in previous studies (31–34,36,37). In addition, there are several items that comprise the MRS3 that are potentially resolvable or treatable. Therefore, future research should examine the use of the MRS3 as a time-dependent covariate to determine the impact of changes in the MRS3 score over time on improving the prediction of mortality. In addition, there were limitations of our approach to validation of the MRS3. First, the score was developed and tested on an admission cohort of NH patients. Therefore, the score may not be appropriate or as accurate in predicting mortality for long-stay NH residents as previous work suggests that long-stay residents tended to be older, male, white, unmarried, low-income subsidy recipients, have multiple comorbidities, and have higher rates of mortality than residents with shorter stays (41). Future work is needed to develop a separate mortality risk score for other cohorts of NH residents such as long-stay residents and those readmitted to the facility. In addition, the score relied only on items present on every assessment. There are a number of items present on the Federal OBRA-required full admission assessment that are not included on the 5-day prospective payment assessment that may have enhanced the performance of this score (ie, Cancer, Cirrhosis, and Renal Insufficiency, Renal Failure, or End-Stage Renal Disease). Despite these limitations, there are many strengths to the development and testing of this score that are worthy of mention. First, this score was developed to predict mortality with population-based administrative information for the national population of new admissions to NHs. Second, it was validated with an external, national population of new admissions to NHs in the following year. Third, this score is comprised of real-time data on health conditions and functioning in the MDS 3.0 that are routinely collected for every patient in every facility by trained staff, unlike other mortality scores or frailty indices requiring information from various data sources including surveys and lab results (42–44). Fourth, we were able to model a patient’s risk of death, regardless of location at time of death, for all Medicare beneficiaries admitted to NHs. Fifth, the transformation of a logistic regression model to an index provides greater clinical utility for decision-making and care planning. Finally, because the score is not a disease-specific measure, unlike many mortality risk scores with good performance characteristics, it allows for prognostication and risk adjustment across different diseases and risk profiles. In summary, the MRS3 is a potential useful tool for identifying mortality risk among patients admitted to NHs. The MRS3 provides valuable information that clinicians, service providers, researchers, and policy makers can use to inform decisions related to the care of patients admitted to NHs without imposing additional workload or assessment burden. Supplementary Material Supplementary data is available at The Journals of Gerontology, Series A: Biological Sciences and Medical Sciences online. Funding This work was supported by the National Institute of Aging at the National Institutes of Health (P01AG027296); and the U.S. Department of Veterans Affairs Health Services Research and Development Service (CDA14-422 to K.S.T.). The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs, the National Institutes of Health, or the U.S. government. Conflict of Interest Vince Mor is the Chair of the Independent Quality Committee at HCR ManorCare and am compensated for this service. HCR is a provider of short-term, post-hospital services, and long-term care. The company has a network of more than 500 nursing and rehabilitation centers, assisted living facilities, outpatient rehabilitation clinics, and hospice and home health care agencies. Vince Mor is a paid consultant to NaviHealth, Inc. and chair their Scientific Advisory Board. NaviHealth is wholly owned by Cardinal Health. The company offers post-acute care (PAC) management and services to more than 1.5 million beneficiaries in all regions of the country through its partnerships with health plans and health systems. Vince Mor is a former Director at PointRight, Inc. While he no longer provides any services or holds any positions at PointRight, he still has a small amount of equity in the company. PointRight is a private company based in Cambridge, MA. It provides predictive analytics solutions to thousands of post-acute providers, long-term care providers, hospitals, payers and insurance organizations. References 1. Liu K, Wissoker D, Swett A. Nursing home use by dual-eligible beneficiaries in the last year of life. Inquiry . 2007; 44: 88– 103. doi: 10.5034/inquiryjrnl_44.1.88 Google Scholar CrossRef Search ADS PubMed  2. Teno JM, Clarridge BR, Casey V, et al.   Family perspectives on end-of-life care at the last place of care. JAMA . 2004; 291: 88– 93. doi: 10.1001/jama.291.1.88 Google Scholar CrossRef Search ADS PubMed  3. Weeks JC, Cook EF, O’Day SJ, et al.   Relationship between cancer patients’ predictions of prognosis and their treatment preferences. JAMA . 1998; 279: 1709– 1714. doi: 10.1001/jama.279.21.1709 Google Scholar CrossRef Search ADS PubMed  4. Murphy DJ, Burrows D, Santilli S, et al.   The influence of the probability of survival on patients’ preferences regarding cardiopulmonary resuscitation. N Engl J Med . 1994; 330: 545– 549. doi: 10.1056/NEJM199402243300807 Google Scholar CrossRef Search ADS PubMed  5. Fox KA, Dabbous OH, Goldberg RJ, et al.   Prediction of risk of death and myocardial infarction in the six months after presentation with acute coronary syndrome: Prospective multinational observational study (GRACE). BMJ . 2006; 333: 1091. doi: 10.1136/bmj.38985.646481.55 Google Scholar CrossRef Search ADS PubMed  6. Antman EM, Cohen M, Bernink PJ, et al.   The TIMI risk score for unstable angina/non-ST elevation MI: A method for prognostication and therapeutic decision making. JAMA . 2000; 284: 835– 842. Google Scholar CrossRef Search ADS PubMed  7. Ranson JH, Rifkind KM, Roses DF, Fink SD, Eng K, Spencer FC. Prognostic signs and the role of operative management in acute pancreatitis. Surg Gynecol Obstet . 1974; 139: 69– 81. Google Scholar PubMed  8. Gawande AA, Kwaan MR, Regenbogen SE, Lipsitz SA, Zinner MJ. An Apgar score for surgery. J Am Coll Surg . 2007; 204: 201– 208. doi: 10.1016/j.jamcollsurg.2006.11.011 Google Scholar CrossRef Search ADS PubMed  9. Sobol JB, Gershengorn HB, Wunsch H, Li G. The surgical Apgar score is strongly associated with intensive care unit admission after high-risk intraabdominal surgery. Anesth Analg . 2013; 117: 438– 446. doi: 10.1213/ANE.0b013e31829180b7 Google Scholar CrossRef Search ADS PubMed  10. Reynolds PQ, Sanders NW, Schildcrout JS, Mercaldo ND, St Jacques PJ. Expansion of the surgical Apgar score across all surgical subspecialties as a means to predict postoperative mortality. Anesthesiology . 2011; 114: 1305– 1312. doi: 10.1097/ALN.0b013e318219d734. Google Scholar CrossRef Search ADS PubMed  11. Tate JP, Justice AC, Hughes MD, et al.   An internationally generalizable risk index for mortality after one year of antiretroviral therapy. AIDS . 2013; 27: 563– 572. doi: 10.1097/QAD.0b013e32835b8c7f Google Scholar CrossRef Search ADS PubMed  12. Justice AC, Modur SP, Tate JP, et al.  ; NA-ACCORD and VACS Project Teams. Predictive accuracy of the Veterans Aging Cohort Study index for mortality with HIV infection: A North American cross cohort analysis. J Acquir Immune Defic Syndr . 2013; 62: 149– 163. doi: 10.1097/QAI.0b013e31827df36c Google Scholar CrossRef Search ADS PubMed  13. Kline JA, Roy PM, Than MP, et al.   Derivation and validation of a multivariate model to predict mortality from pulmonary embolism with cancer: The POMPE-C tool. Thromb Res . 2012; 129: e194– e199. doi: 10.1016/j.thromres.2012.03.015 Google Scholar CrossRef Search ADS PubMed  14. van der Hulle T, den Exter PL, Kooiman J, van der Hoeven JJ, Huisman MV, Klok FA. Meta-analysis of the efficacy and safety of new oral anticoagulants in patients with cancer-associated acute venous thromboembolism. J Thromb Haemost . 2014; 12: 1116– 1120. doi: 10.1111/jth.12605 Google Scholar CrossRef Search ADS PubMed  15. Vickers AJ, Kramer BS, Baker SG. Selecting patients for randomized trials: A systematic approach based on risk group. Trials . 2006; 7: 30. doi: 10.1186/1745-6215-7-30 Google Scholar CrossRef Search ADS PubMed  16. Hernández AV, Steyerberg EW, Habbema JD. Covariate adjustment in randomized controlled trials with dichotomous outcomes increases statistical power and reduces sample size requirements. J Clin Epidemiol . 2004; 57: 454– 460. doi: 10.1016/j.jclinepi.2003.09.014 Google Scholar CrossRef Search ADS PubMed  17. Hernández AV, Eijkemans MJ, Steyerberg EW. Randomized controlled trials with time-to-event outcomes: How much does prespecified covariate adjustment increase power? Ann Epidemiol . 2006; 16: 41– 48. doi: 10.1016/j.annepidem.2005.09.007 Google Scholar CrossRef Search ADS PubMed  18. Iezzoni LI. Risk Adjustment for Measuring Health Care Outcomes . 3rd ed. Chicago, IL: Health Administration Press; 2003. 19. Iezzoni L. Risk Adjustment for Measuring Health Care Outcomes . 4th ed. Chicago, IL: Health Administration Press; 2012. 20. Centers for Medicare & Medicaid Services. MDS 3.0 RAI Manual . 2017. 21. Mitchell SL, Kiely DK, Hamel MB, Park PS, Morris JN, Fries BE. Estimating prognosis for nursing home residents with advanced dementia. JAMA . 2004; 291: 2734– 2740. doi: 10.1001/jama.291.22.2734 Google Scholar CrossRef Search ADS PubMed  22. Mitchell SL, Miller SC, Teno JM, Davis RB, Shaffer ML. The advanced dementia prognostic tool: A risk score to estimate survival in nursing home residents with advanced dementia. J Pain Symptom Manage . 2010; 40: 639– 651. doi: 10.1016/j.jpainsymman.2010.02.014 Google Scholar CrossRef Search ADS PubMed  23. Abicht-Swensen LM, Debner LK. The minimum data set 2.0: A functional assessment to predict mortality in nursing home residents. Am J Hosp Palliat Care . 1999; 16: 527– 532. doi: 10.1177/104990919901600308 Google Scholar CrossRef Search ADS PubMed  24. Flacker JM, Kiely DK. Mortality-related factors and 1-year survival in nursing home residents. J Am Geriatr Soc . 2003; 51: 213– 221.doi: 10.1046/j.1532-5415.2003.51060.x Google Scholar CrossRef Search ADS PubMed  25. Porock D, Oliver DP, Zweig S, et al.   Predicting death in the nursing home: Development and validation of the 6-month Minimum Data Set mortality risk index. J Gerontol A Biol Sci Med Sci . 2005; 60: 491– 498. doi: 10.1093/gerona/60.4.491 Google Scholar CrossRef Search ADS PubMed  26. Brink P, Kelley ML. Death in long-term care: A brief report examining factors associated with death within 31 days of assessment. Palliat Care . 2015; 9: 1– 5. doi: 10.4137/PCRT.S20347 Google Scholar PubMed  27. van Dijk PT, Mehr DR, Ooms ME, et al.   Comorbidity and 1-year mortality risks in nursing home residents. J Am Geriatr Soc . 2005; 53: 660– 665. doi: 10.1111/j.1532-5415.2005.53216.x Google Scholar CrossRef Search ADS PubMed  28. Levy C, Kheirbek R, Alemi F, et al.   Predictors of six-month mortality among nursing home residents: Diagnoses may be more predictive than functional disability. J Palliat Med . 2015; 18: 100– 106. doi: 10.1089/jpm.2014.0130 Google Scholar CrossRef Search ADS PubMed  29. Porock D, Parker-Oliver D, Petroski GF, Rantz M. The MDS mortality risk index: The evolution of a method for predicting 6-month mortality in nursing home residents. BMC Res Notes . 2010; 3: 200. doi: 10.1186/1756-0500-3-200 Google Scholar CrossRef Search ADS PubMed  30. Hirdes JP, Frijters DH, Teare GF. The MDS-CHESS scale: A new measure to predict mortality in institutionalized older people. J Am Geriatr Soc . 2003; 51: 96– 100. doi: 10.1034/j.1601-5215.2002.51017.x Google Scholar CrossRef Search ADS PubMed  31. Saliba D, Buchanan J. Making the investment count: Revision of the Minimum Data Set for nursing homes, MDS 3.0. J Am Med Dir Assoc . 2012; 13( 7): 602– 610. doi: 10.1016/j.jamda.2012.06.002 Google Scholar CrossRef Search ADS PubMed  32. Saliba D, Buchanan J, Edelen MO, et al.   MDS 3.0: Brief interview for mental status. J Am Med Dir Assoc . 2012; 13( 7): 611– 617. doi: 10.1016/j.jamda.2012.06.004 Google Scholar CrossRef Search ADS PubMed  33. Saliba D, DiFilippo S, Edelen MO, Kroenke K, Buchanan J, Streim J. Testing the PHQ-9 interview and observational versions (PHQ-9 OV) for MDS 3.0. J Am Med Dir Assoc . 2012; 13: 618– 625. doi: 10.1016/j.jamda.2012.06.003 Google Scholar CrossRef Search ADS PubMed  34. Saliba D, Jones M, Streim J, Ouslander J, Berlowitz D, Buchanan J. Overview of significant changes in the Minimum Data Set for nursing homes version 3.0. J Am Med Dir Assoc . 2012; 13: 595– 601. doi: 10.1016/j.jamda.2012.06.001 Google Scholar CrossRef Search ADS PubMed  35. Kosar CM, Thomas KS, Inouye SK, Mor V. Delirium during postacute nursing home admission and risk for adverse outcomes. J Am Geriatr Soc . 2017; 65: 1470– 1475. doi: 10.1111/jgs.14823 Google Scholar CrossRef Search ADS PubMed  36. Rahman M, Tyler D, Acquah JK, Lima J, Mor V. Sensitivity and specificity of the Minimum Data Set 3.0 discharge data relative to Medicare claims. J Am Med Dir Assoc . 2014; 15: 819– 824. doi: 10.1016/j.jamda.2014.06.017 Google Scholar CrossRef Search ADS PubMed  37. Thomas KS, Dosa D, Wysocki A, Mor V. The Minimum Data Set 3.0 cognitive function scale. Med Care . 2017; 55( 9): e68– e72. doi: 10.1097/MLR.0000000000000334 Google Scholar CrossRef Search ADS PubMed  38. Thomas KS, Wysocki A, Intrator O, Mor V. Finding gertrude: The resident’s voice in Minimum Data Set 3.0. J Am Med Dir Assoc . 2014; 15( 11): 802– 806. doi: 10.1016/j.jamda.2014.01.012 Google Scholar CrossRef Search ADS PubMed  39. Wysocki A, Thomas KS, Mor V. Functional improvement among short-stay nursing home residents in the MDS 3.0. J Am Med Dir Assoc . 2015; 16: 470– 474. doi: 10.1016/j.jamda.2014.11.018 Google Scholar CrossRef Search ADS PubMed  40. Ruopp MD, Perkins NJ, Whitcomb BW, Schisterman EF. Youden index and optimal cut-point estimated from observations affected by a lower limit of detection. Biom J . 2008; 50: 419– 430. doi: 10.1002/bimj.200710415 Google Scholar CrossRef Search ADS PubMed  41. Wei YJ, Simoni-Wastila L, Zuckerman IH, Brandt N, Lucas JA. Algorithm for identifying nursing home days using medicare claims and minimum data set assessment data. Med Care . 2016; 54: e73– e77. doi: 10.1097/MLR.0000000000000109 Google Scholar CrossRef Search ADS PubMed  42. Ritt M, Jäger J, Ritt JI, Sieber CC, Gaßmann KG. Operationalizing a frailty index using routine blood and urine tests. Clin Interv Aging . 2017; 12: 1029– 1040. doi: 10.2147/CIA.S131987 Google Scholar CrossRef Search ADS PubMed  43. Rockwood K, Mitnitski AB, MacKnight C. Some mathematical models of frailty and their clinical implications. Rev Clin Gerontol . 2002; 12: 109– 117. doi: 10.1017/S0959259802012236 Google Scholar CrossRef Search ADS   44. Rockwood K, Song X, MacKnight C, et al.   A global clinical measure of fitness and frailty in elderly people. CMAJ . 2005; 173: 489– 495. doi: 10.1503/cmaj.050051 Google Scholar CrossRef Search ADS PubMed  Published by Oxford University Press on behalf of The Gerontological Society of America 2018. This work is written by (a) US Government employee(s) and is in the public domain in the US. This work is written by (a) US Government employee(s) and is in the public domain in the US. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png The Journals of Gerontology Series A: Biomedical Sciences and Medical Sciences Oxford University Press

Development and Validation of the Nursing Home Minimum Data Set 3.0 Mortality Risk Score (MRS3)

Loading next page...
 
/lp/ou_press/development-and-validation-of-the-nursing-home-minimum-data-set-3-0-t2WN5ARhEZ
Publisher
Oxford University Press
Copyright
Published by Oxford University Press on behalf of The Gerontological Society of America 2018. This work is written by (a) US Government employee(s) and is in the public domain in the US.
ISSN
1079-5006
eISSN
1758-535X
D.O.I.
10.1093/gerona/gly044
Publisher site
See Article on Publisher Site

Abstract

Abstract Background To develop a score to predict mortality using the Minimum Data Set 3.0 (MDS 3.0) that can be readily calculated from items collected during nursing home (NH) residents’ admission assessments. Participants We developed a training cohort of Medicare beneficiaries newly admitted to United States NHs during 2012 (N = 1,426,815) and a testing cohort from 2013 (N = 1,160,964). Methods Data came from the MDS 3.0 assessments linked to the Medicare Beneficiary Summary File. Using the training dataset, we developed a composite MDS 3.0 Mortality Risk Score (MRS3) consisting of 17 clinical items and patients’ age groups based on their relation to 30-day mortality. We assessed the calibration and discrimination of the MRS3 in predicting 30- and 60-day mortality and compared its performance to the Charlson Comorbidity Index and the clinician’s assessment of 6-month prognosis measured at admission. Results The 30- and 60-day mortality rates for the testing population were 2.8% and 5.6%, respectively. Results from logistic regression models suggest that the MRS3 performed well in predicting death within 30 and 60 days (C-Statistics of 0.744 [95% confidence limit (CL) = 0.741, 0.747] and 0.709 [95% CL = 0.706, 0.711], respectively). The MRS3 was a superior predictor of mortality compared to the Charlson Comorbidity Index (C-statistics of 0.611 [95% CL = 0.607, 0.615] and 0.608 [95% CL = 0.605, 0.610]) and the clinicians’ assessments of patients’ 6-month prognoses (C-statistics of 0.543 [95% CL = 0.542, 0.545] and 0.528 [95% CL = 0.527, 0.529]). Conclusions The MRS3 is a good predictor of mortality and can be useful in guiding decision-making, informing plans of care, and adjusting for patients’ risk of mortality. Prognosis, Nursing home, Mortality, Risk adjustment Nursing homes (NHs) provide care to patients with a wide spectrum of clinical care needs, many of whom are in their last year of life (1,2). Understanding the clinical profile and risk of mortality upon NH admission can stimulate discussions designed to establish patients’ goals of care, as well as guide end-of-life decision-making and advance care planning with patients and family members (3,4). However, predicting a patient’s survival is fraught with difficulty and notoriously imprecise. Recognizing this difficulty, many mortality risk scores and models have been developed to assist physicians outside of the NH setting (eg, GRACE (5), TIMI (6), Ranson’s criteria for pancreatic mortality (7), Surgical Apgar Scale (8–10), Veterans Aging Cohort Study (VACS) index (11,12), and POMPE-C Tool) (13,14). In addition to their utility for clinical decision-making, mortality risk scores and prediction models are also used to adjust for a patient’s risk of adverse outcomes and as inclusion criteria in randomized control trials (15–17). Furthermore, mortality risk scores can be useful in calibrating facility case mix, thereby allowing for more valid comparisons of outcomes across providers (18). As such, they are often used as adjusters in outcomes research and publicly reported quality measures (19). While it is important to have valid scoring systems that predict mortality and signal a referral for advanced care planning or hospice, it is critically important that such systems are constructed from readily available data routinely obtained in clinical practice. The mandated NH resident assessment process, utilizing the Minimum Data Set version 3.0 (MDS 3.0) instrument (20), captures rich clinical and functional information about patients to inform clinical decision-making and care planning. Previous efforts to develop mortality risk scores for the NH setting have relied on sub-samples of patients (eg, those with advanced dementia) (21,22) and items derived from a previous version of the MDS resident assessment data (MDS 2.0) (23–30). However, many of the items in the MDS 2.0 used in previous mortality risk scores were eliminated in the new version of the MDS implemented in October 2010. As such, we have not had a mortality risk score for the NH population since the introduction of the MDS 3.0. The purpose of this study was to develop and evaluate a mortality risk score based upon a national population of newly admitted NH patients, which can be readily calculated from items collected during the admission assessment process using the MDS 3.0 in order to guide decision-making, better inform plans of care, and adjust for facility case-mix severity. Method Data Data for this study come from the MDS 3.0 and the Medicare Master Beneficiary Summary File. The MDS instrument is a standardized, comprehensive assessment that must be completed for all persons who receive care in a Medicare and/or Medicaid-certified nursing facility. The MDS captures information about residents’ physical, cognitive, psychological, and psychosocial functioning, in addition to medical diagnoses, clinical symptoms, and any treatments or therapies prescribed and received. Assessments are conducted on all residents at admission and discharge, in addition to other time intervals (eg, quarterly and annually). While initially designed for care planning, the MDS is also used for payment, quality monitoring, and research. The MDS 3.0, the current version of the assessment instrument, has been extensively validated (31–38). From the MDS 3.0, we obtained the first full assessment after its introduction (October 1, 2010) for each individual. We linked individual’s first MDS assessment from 2012–2013 to the 2012–2014 Medicare Master Summary Beneficiary File to identify NH residents who were Medicare beneficiaries and to determine beneficiaries’ dates of death. Importantly, the Medicare Master Beneficiary Summary File allows us to identify NH residents who die, regardless of where or when. Study Samples With the MDS 3.0, we identified a cohort of Medicare beneficiaries who were newly admitted to the NH in 2012 and 2013 with no NH stay since the introduction of the MDS 3.0 in October 2010. We split the sample to derive “training” and “testing” populations. The “training” population used to develop the prediction model consisted of all Medicare beneficiaries newly admitted to NHs during 2012 (N = 1,426,815). The population used to “test” the sensitivity and specificity of the prediction model included all Medicare beneficiaries newly admitted to NHs during 2013 (N = 1,160,964). Study Variables Measures of mortality were created to examine whether the new resident admitted to the NH died within 30 days and within 60 days of the assessment. Thirty-day mortality was used to develop the score and we tested the performance of the score using the 30- and 60-day measures of mortality. To develop the score, the research team, including a practicing geriatrician and palliative care physician, identified clinical indicators that were reported across assessments in the MDS 3.0 believed to be related to the likelihood of death. A total of 46 candidate measures was selected and grouped into three categories: 1) Cognitive and Physical Function, 2) Diagnoses, and 3) Health Conditions (see Supplementary Table 1, for a complete list of measures). For ease of calculation, we dichotomized ordinal variables measuring cognitive and physical function. The cognitive and physical function variables were dichotomized into categories representing severe functional limitation (a 28-point Activities of Daily Living [ADL] (39) score greater than or equal to 23) and severe cognitive impairment (a Cognitive Function Scale score of 4) (37). We collapsed variables that measured a similar construct. For example, three shortness of breath measures (lying flat, sitting, and with exertion) were collapsed into one measure to capture whether or not patients experienced any of these symptoms. We also created a composite measure to indicate if a patient experienced any swallowing disorder, and a composite measure to indicate if a patient had at least one stage 3, stage 4, or unstageable pressure ulcer. Analyses Approach to develop a mortality risk score With the training population, we conducted bivariate analyses, with facility fixed effects to remove any influence of the facility on mortality, predicting death at 30 days for each of the measures that were candidates for inclusion in the score. Only statistically significant variables (p < .0001) with odds ratios greater than 1.2 were retained for the multivariate models. A total of 21 clinical indicators were entered into a backwards, stepwise regression model predicting death within 30 days. Variables with a p-value greater than .2 were dropped from the model. The remaining 17 measures were tested in a logistic regression model with facility fixed effects predicting death within 30 days. We calculated the adjusted odds ratio for each variable to estimate the risk of death for a given variable controlling for coexistent cognitive and physical function, diagnoses, and health conditions. We created a weighted score, the MDS 3.0 Mortality Risk Score (MRS3), to represent the number and seriousness of a patient’s clinical indicators in addition to a patient’s age. Specifically, the adjusted odds ratios were employed as weights for the different clinical indicators. Indicators with an odds ratio of ≥1.2 and <1.5 were assigned a weight of 1 (n = 5); indicators with an odds ratio of ≥1.5 and <2.5 a weight of 2 (n = 7); indicators with an odds ratio of ≥2.5 a weight of 3 (n = 5). Age groups were assigned the following weighted values: Ages <65 = 1, 65–74 = 2, 75–84 = 3, 85–94 = 4, and 95+ = 5. (See Table 1, for a list of indicators and weighted values). Weighted values were summed for each individual to create a score that could range from 1 to 39 with 1 being least and 39 being the highest risk of mortality. However, given the very small number of individuals with a score greater than 20 (less than 0.001%), we censored the higher values of the score so that any above 20 were set to 20. Table 1. Scoring the MRS3 Component  MRS3 Value  Age <65 years  1  Age 65–74 years  2  Age 75–84 years  3  Age 85–94 years  4  Age 95+ years  5  Severe physical impairmenta  3  Acute change in mental status  3  Shortness of breath, any  3  Dehydrated  3  Severe cognitive impairmentb  3  Internal bleeding  2  Swallowing disorder, any  2  Vomiting  2  Fever  2  Unplanned weight loss  2  Heart Failure  2  Worst pressure ulcer, slough, or necrotic  2  Pneumonia  1  Any falls since admission or prior assessment  1  Viral hepatitis  1  Hyponatremia  1  At least one stage 3, 4, or unstageable pressure ulcer  1  Component  MRS3 Value  Age <65 years  1  Age 65–74 years  2  Age 75–84 years  3  Age 85–94 years  4  Age 95+ years  5  Severe physical impairmenta  3  Acute change in mental status  3  Shortness of breath, any  3  Dehydrated  3  Severe cognitive impairmentb  3  Internal bleeding  2  Swallowing disorder, any  2  Vomiting  2  Fever  2  Unplanned weight loss  2  Heart Failure  2  Worst pressure ulcer, slough, or necrotic  2  Pneumonia  1  Any falls since admission or prior assessment  1  Viral hepatitis  1  Hyponatremia  1  At least one stage 3, 4, or unstageable pressure ulcer  1  Notes: Weighted values are summed to create the MDS 3.0 Mortality Risk Score (MRS3) that could range from 1 to 39 with 1 being least and 39 being the highest risk of mortality. A MRS3 score of 8 or higher is associated with a 5.4 times increase in the odds of 30-day mortality compared to a score of 7 or lower (OR = 5.408, CI = 5.726, 5.543) and a 4.2 times increase in the odds of 60-day mortality (OR = 4.215, CI = 4.140, 4.291). aActivities of Daily Living Score greater than or equal to 23 (range 0–28, 0 = total independence, 28 = total dependence). bCognitive Function Scale score of 4 (range 1–4; 1 = intact cognition, 4 = severe cognitive impairment). View Large Table 1. Scoring the MRS3 Component  MRS3 Value  Age <65 years  1  Age 65–74 years  2  Age 75–84 years  3  Age 85–94 years  4  Age 95+ years  5  Severe physical impairmenta  3  Acute change in mental status  3  Shortness of breath, any  3  Dehydrated  3  Severe cognitive impairmentb  3  Internal bleeding  2  Swallowing disorder, any  2  Vomiting  2  Fever  2  Unplanned weight loss  2  Heart Failure  2  Worst pressure ulcer, slough, or necrotic  2  Pneumonia  1  Any falls since admission or prior assessment  1  Viral hepatitis  1  Hyponatremia  1  At least one stage 3, 4, or unstageable pressure ulcer  1  Component  MRS3 Value  Age <65 years  1  Age 65–74 years  2  Age 75–84 years  3  Age 85–94 years  4  Age 95+ years  5  Severe physical impairmenta  3  Acute change in mental status  3  Shortness of breath, any  3  Dehydrated  3  Severe cognitive impairmentb  3  Internal bleeding  2  Swallowing disorder, any  2  Vomiting  2  Fever  2  Unplanned weight loss  2  Heart Failure  2  Worst pressure ulcer, slough, or necrotic  2  Pneumonia  1  Any falls since admission or prior assessment  1  Viral hepatitis  1  Hyponatremia  1  At least one stage 3, 4, or unstageable pressure ulcer  1  Notes: Weighted values are summed to create the MDS 3.0 Mortality Risk Score (MRS3) that could range from 1 to 39 with 1 being least and 39 being the highest risk of mortality. A MRS3 score of 8 or higher is associated with a 5.4 times increase in the odds of 30-day mortality compared to a score of 7 or lower (OR = 5.408, CI = 5.726, 5.543) and a 4.2 times increase in the odds of 60-day mortality (OR = 4.215, CI = 4.140, 4.291). aActivities of Daily Living Score greater than or equal to 23 (range 0–28, 0 = total independence, 28 = total dependence). bCognitive Function Scale score of 4 (range 1–4; 1 = intact cognition, 4 = severe cognitive impairment). View Large Approach to External Validation We assessed the accuracy of the score with measures of discrimination and calibration. To do this, we conducted logistic regression analyses with the MRS3 to predict the odds of death at 30 and 60 days. We tested the discrimination of the score using the area under the receiver operating characteristic (ROC) curve (or concordance statistic, C). An area under the ROC curve of 0.5 indicates no discrimination, whereas a C-Statistic of 1 represents perfect discrimination. For comparison of the score’s discrimination to other established measures, we conducted separate logistic regression analyses using the Charlson Comorbidity Index and the 6-month prognosis indicator in the MDS assessment to predict death within 30 and 60 days. The Charlson Comorbidity Index is derived using the list of active diagnoses that are reported in the MDS 3.0 assessment. The 6-month prognosis variable is an indicator that is completed by a clinician and reported in the MDS 3.0 assessment. This variable is used to identify whether or not the patient has a life limiting illness or condition and is not expected to survive longer than 6 months. We also tested performance of the MRS3 in terms of calibration. Calibration refers to how closely the predicted mortality risk agrees with observed mortality. We assessed calibration of the MRS3 by plotting the observed proportions versus predicted probabilities of death at 30 and 60 days and by tabulating the 30-day mortality ratio (predicted to actual) for each value of the MRS3 score. We used the Youden Index (J) (40) to identify a cut point that discriminates between low versus high risk of mortality and examined its performance in logistic regression models predicting 30- and 60-day mortality. Sensitivity Analyses In sensitivity analyses, we tested a logistic regression model predicting mortality using the detailed coefficients for each indicator. Results Table 2 describes the training sample of newly admitted NH patients in terms of their demographic, clinical, and functional characteristics as well as the observed 30-, 60-day, and 1-year mortality rates. The mortality rate for the testing population was 2.9% within 30 days and 5.7% within 60 days. The mean MRS3 score was 4.74 (SD = 2.49) with a median of 4. Over one-third of the testing population was 85 years or older and the most frequently reported indicator was shortness of breath (18.34%) followed by heart failure (18.14%). The bivariate relationship with mortality for each variable included in the MRS3 can be found in the Supplementary Table 1. Table 2. Descriptive Characteristics of Sample   Training Population: New Admissions, 2012  Testing Population: New Admissions, 2013  Number of observations (N)  1,426,815  1,160,964   30-day mortality (%)  5.72  2.88   60-day mortality (%)  9.76  5.69   365-day mortality (%)  25.55  21.94  Age groups   Age: <65 (%)  9.08  9.17   Age: 65–74 (%)  21.84  22.3   Age: 75–84 (%)  35.1  34.78   Age: 85–94 (%)  30.46  30.26   Age: 95+ (%)  3.53  3.49  Demographics   Male  36.88  31.10   American Indian/Alaska  Native  0.34  0.30   Asian  1.37  1.54   Black  9.04  9.62   Hispanic or Latino  3.76  4.09   Native Hawaiian or Pacific  Islander  0.25  0.28   White  82.81  80.88  Cognitive and physical function   Severe cognitive impairmenta (%)  3.62  3.01   Acute change in mental status (%)  2.14  1.59   Severe Physical Impairmentb (%)  7.95  6.47   Any falls since admission or  prior assessment (%)  6.99  7.2  Diagnoses   Heart failure (%)  18.20  18.14   Pneumonia (%)  9.1  8.74   Hyponatremia (%)  2.24  2.3  Health conditions       Dehydrated (%)  0.42  0.33   Fever (%)  2.79  2.34   Internal bleeding (%)  0.93  0.82   Vomiting (%)  2.59  2.29   Shortness of breath, any (%)  19.38  18.34   Shortness of breath when lying  flat (%)  9.04  8.53   Shortness of breath when  sitting (%)  5.64  5.01   Shortness of breath with  exertion (%)  16.08  15.23  Swallowing disorder, any (%)  5.46  4.76   Choking (%)  2.35  2.01   Painful swallowing (%)  2.76  2.48   Holding food (%)  1.19  1.01   Lost liquid (%)  0.53  0.43   Unplanned weight loss (%)  4.47  4.05   At least one stage 3, 4, or  unstageable pressure ulcer (%)  3.34  3.44   Worst pressure ulcer, Slough or  Necrotic (%)  3.15  2.99   Worst pressure ulcer:   Slough (3) (%)  1.51  1.5   Worst pressure ulcer:   Necrotic (4) (%)  1.63  1.48    Training Population: New Admissions, 2012  Testing Population: New Admissions, 2013  Number of observations (N)  1,426,815  1,160,964   30-day mortality (%)  5.72  2.88   60-day mortality (%)  9.76  5.69   365-day mortality (%)  25.55  21.94  Age groups   Age: <65 (%)  9.08  9.17   Age: 65–74 (%)  21.84  22.3   Age: 75–84 (%)  35.1  34.78   Age: 85–94 (%)  30.46  30.26   Age: 95+ (%)  3.53  3.49  Demographics   Male  36.88  31.10   American Indian/Alaska  Native  0.34  0.30   Asian  1.37  1.54   Black  9.04  9.62   Hispanic or Latino  3.76  4.09   Native Hawaiian or Pacific  Islander  0.25  0.28   White  82.81  80.88  Cognitive and physical function   Severe cognitive impairmenta (%)  3.62  3.01   Acute change in mental status (%)  2.14  1.59   Severe Physical Impairmentb (%)  7.95  6.47   Any falls since admission or  prior assessment (%)  6.99  7.2  Diagnoses   Heart failure (%)  18.20  18.14   Pneumonia (%)  9.1  8.74   Hyponatremia (%)  2.24  2.3  Health conditions       Dehydrated (%)  0.42  0.33   Fever (%)  2.79  2.34   Internal bleeding (%)  0.93  0.82   Vomiting (%)  2.59  2.29   Shortness of breath, any (%)  19.38  18.34   Shortness of breath when lying  flat (%)  9.04  8.53   Shortness of breath when  sitting (%)  5.64  5.01   Shortness of breath with  exertion (%)  16.08  15.23  Swallowing disorder, any (%)  5.46  4.76   Choking (%)  2.35  2.01   Painful swallowing (%)  2.76  2.48   Holding food (%)  1.19  1.01   Lost liquid (%)  0.53  0.43   Unplanned weight loss (%)  4.47  4.05   At least one stage 3, 4, or  unstageable pressure ulcer (%)  3.34  3.44   Worst pressure ulcer, Slough or  Necrotic (%)  3.15  2.99   Worst pressure ulcer:   Slough (3) (%)  1.51  1.5   Worst pressure ulcer:   Necrotic (4) (%)  1.63  1.48  Notes:aCognitive Function Scale score of 4 (Range 1–4; 1 = intact cognition, 4 = severe cognitive impairment). bActivities of Daily Living Score greater than or equal to 23 (Range 0–28, 0 = total independence, 28 = total dependence). View Large Table 2. Descriptive Characteristics of Sample   Training Population: New Admissions, 2012  Testing Population: New Admissions, 2013  Number of observations (N)  1,426,815  1,160,964   30-day mortality (%)  5.72  2.88   60-day mortality (%)  9.76  5.69   365-day mortality (%)  25.55  21.94  Age groups   Age: <65 (%)  9.08  9.17   Age: 65–74 (%)  21.84  22.3   Age: 75–84 (%)  35.1  34.78   Age: 85–94 (%)  30.46  30.26   Age: 95+ (%)  3.53  3.49  Demographics   Male  36.88  31.10   American Indian/Alaska  Native  0.34  0.30   Asian  1.37  1.54   Black  9.04  9.62   Hispanic or Latino  3.76  4.09   Native Hawaiian or Pacific  Islander  0.25  0.28   White  82.81  80.88  Cognitive and physical function   Severe cognitive impairmenta (%)  3.62  3.01   Acute change in mental status (%)  2.14  1.59   Severe Physical Impairmentb (%)  7.95  6.47   Any falls since admission or  prior assessment (%)  6.99  7.2  Diagnoses   Heart failure (%)  18.20  18.14   Pneumonia (%)  9.1  8.74   Hyponatremia (%)  2.24  2.3  Health conditions       Dehydrated (%)  0.42  0.33   Fever (%)  2.79  2.34   Internal bleeding (%)  0.93  0.82   Vomiting (%)  2.59  2.29   Shortness of breath, any (%)  19.38  18.34   Shortness of breath when lying  flat (%)  9.04  8.53   Shortness of breath when  sitting (%)  5.64  5.01   Shortness of breath with  exertion (%)  16.08  15.23  Swallowing disorder, any (%)  5.46  4.76   Choking (%)  2.35  2.01   Painful swallowing (%)  2.76  2.48   Holding food (%)  1.19  1.01   Lost liquid (%)  0.53  0.43   Unplanned weight loss (%)  4.47  4.05   At least one stage 3, 4, or  unstageable pressure ulcer (%)  3.34  3.44   Worst pressure ulcer, Slough or  Necrotic (%)  3.15  2.99   Worst pressure ulcer:   Slough (3) (%)  1.51  1.5   Worst pressure ulcer:   Necrotic (4) (%)  1.63  1.48    Training Population: New Admissions, 2012  Testing Population: New Admissions, 2013  Number of observations (N)  1,426,815  1,160,964   30-day mortality (%)  5.72  2.88   60-day mortality (%)  9.76  5.69   365-day mortality (%)  25.55  21.94  Age groups   Age: <65 (%)  9.08  9.17   Age: 65–74 (%)  21.84  22.3   Age: 75–84 (%)  35.1  34.78   Age: 85–94 (%)  30.46  30.26   Age: 95+ (%)  3.53  3.49  Demographics   Male  36.88  31.10   American Indian/Alaska  Native  0.34  0.30   Asian  1.37  1.54   Black  9.04  9.62   Hispanic or Latino  3.76  4.09   Native Hawaiian or Pacific  Islander  0.25  0.28   White  82.81  80.88  Cognitive and physical function   Severe cognitive impairmenta (%)  3.62  3.01   Acute change in mental status (%)  2.14  1.59   Severe Physical Impairmentb (%)  7.95  6.47   Any falls since admission or  prior assessment (%)  6.99  7.2  Diagnoses   Heart failure (%)  18.20  18.14   Pneumonia (%)  9.1  8.74   Hyponatremia (%)  2.24  2.3  Health conditions       Dehydrated (%)  0.42  0.33   Fever (%)  2.79  2.34   Internal bleeding (%)  0.93  0.82   Vomiting (%)  2.59  2.29   Shortness of breath, any (%)  19.38  18.34   Shortness of breath when lying  flat (%)  9.04  8.53   Shortness of breath when  sitting (%)  5.64  5.01   Shortness of breath with  exertion (%)  16.08  15.23  Swallowing disorder, any (%)  5.46  4.76   Choking (%)  2.35  2.01   Painful swallowing (%)  2.76  2.48   Holding food (%)  1.19  1.01   Lost liquid (%)  0.53  0.43   Unplanned weight loss (%)  4.47  4.05   At least one stage 3, 4, or  unstageable pressure ulcer (%)  3.34  3.44   Worst pressure ulcer, Slough or  Necrotic (%)  3.15  2.99   Worst pressure ulcer:   Slough (3) (%)  1.51  1.5   Worst pressure ulcer:   Necrotic (4) (%)  1.63  1.48  Notes:aCognitive Function Scale score of 4 (Range 1–4; 1 = intact cognition, 4 = severe cognitive impairment). bActivities of Daily Living Score greater than or equal to 23 (Range 0–28, 0 = total independence, 28 = total dependence). View Large Results from our logistic regression models suggest that the MRS3 performed well in predicting death within 30 and 60 days (with C-statistics of 0.744 [95% CL = 0.741, 0.747] and 0.709 [95% CL = 0.706, 0.711]), respectively (see Table 3). When comparing the performance of the MRS3 models in predicting mortality to those using either the Charlson Comorbidity Index or the clinicians’ assessment of 6-month prognosis, we found that the MRS3 was superior (see Table 3). Specifically, the C-statistic for the model using the Charlson Comorbidity Index to predict 30-day mortality was 0.611 (95% CL = 0.607, 0.615), and it was even lower for the model using a clinician’s assessment of 6-month prognosis (C = 0.543, 95% CL = 0.542, 0.545). Table 3. Performance of MRS3 in Predicting Death among the Testing Population Compared to the Charlson Comorbidity Index and the 6-Month Prognosis Indicator   30-Day Mortality  60-Day Mortality  MRS3  0.744 (0.741, 0.747)  0.709 (0.706, 0.711)  Charlson Comorbidity Index  0.611 (0.607, 0.615)  0.608 (0.605, 0.610)  6-Month Prognosis  0.543 (0.542, 0.545)  0.528 (0.527, 0.529)    30-Day Mortality  60-Day Mortality  MRS3  0.744 (0.741, 0.747)  0.709 (0.706, 0.711)  Charlson Comorbidity Index  0.611 (0.607, 0.615)  0.608 (0.605, 0.610)  6-Month Prognosis  0.543 (0.542, 0.545)  0.528 (0.527, 0.529)  Notes: Values represent the C-statistics and 95% Confidence Intervals. View Large Table 3. Performance of MRS3 in Predicting Death among the Testing Population Compared to the Charlson Comorbidity Index and the 6-Month Prognosis Indicator   30-Day Mortality  60-Day Mortality  MRS3  0.744 (0.741, 0.747)  0.709 (0.706, 0.711)  Charlson Comorbidity Index  0.611 (0.607, 0.615)  0.608 (0.605, 0.610)  6-Month Prognosis  0.543 (0.542, 0.545)  0.528 (0.527, 0.529)    30-Day Mortality  60-Day Mortality  MRS3  0.744 (0.741, 0.747)  0.709 (0.706, 0.711)  Charlson Comorbidity Index  0.611 (0.607, 0.615)  0.608 (0.605, 0.610)  6-Month Prognosis  0.543 (0.542, 0.545)  0.528 (0.527, 0.529)  Notes: Values represent the C-statistics and 95% Confidence Intervals. View Large Figure 1 plots the observed versus predicted probability of death for each value of the MRS3 score in the testing sample. The smooth, 45-degree line represents a perfect prediction model where the observed value and predicted value are the same, while the shapes represent the actual probability of death at each time period at each value of the score. Because the models calibrate to the data well, the shapes and the line lie close to each other. As can be seen in Figure 1, the MRS3 performs well in predicting mortality with the greatest deviations seen among the highest values, which have fewer observations. Figure 1. View largeDownload slide Calibration curve for observed versus predicted probability of death at 30 and 60 days. Notes: Values plotted are the average predicted and average observed probability for each level of the MRS3 (1–26). Figure 1. View largeDownload slide Calibration curve for observed versus predicted probability of death at 30 and 60 days. Notes: Values plotted are the average predicted and average observed probability for each level of the MRS3 (1–26). We present the observed and predicted 30-day mortality rates, by MRS3 value, in Table 4. Results suggest that individuals with a low MRS3 have a very low observed rate of death in the short term. For example, of the 16% of individuals with an MRS3 score of 1 or 2, less than 1% had died within 30 days. In contrast, 34% of individuals with an MRS3 score of 15 or higher had died within 30 days. Table 4. Observed and Predicted 30-Day Mortality, by MRS3 Score MRS3 Score  N  Predicted Risk of 30-Day Mortality  Observed 30-Day Mortality Rate  Predicted 30-Day Mortality  Observed 30-Day Mortality  Ratio of Observed to Expected Mortality  1  46,255  0.59  0.58  271  267  1.01  2  124,129  0.79  0.79  986  976  1.01  3  199,554  1.08  0.93  2,150  1,864  1.15  4  207,875  1.46  1.28  3,035  2,658  1.14  5  124,279  1.98  2.12  2,456  2,628  0.93  6  113,835  2.67  2.89  3,039  3,287  0.92  7  80,065  3.60  3.90  2,880  3,125  0.92  8  51,509  4.83  5.05  2,488  2,600  0.96  9  38,639  6.46  6.72  2,496  2,598  0.96  10  21,656  8.59  8.83  1,860  1,912  0.97  11  11,687  11.33  11.25  1,325  1,315  1.01  12  7,157  14.81  15.15  1,061  1,084  0.98  13  4,072  19.13  17.68  780  720  1.08  14  2,134  24.35  21.46  520  458  1.13  15  1,361  30.46  27.55  415  375  1.11  16  785  37.34  34.65  294  272  1.08  17  439  44.77  35.54  197  156  1.26  18  209  52.44  43.54  110  91  1.20  19  136  60.01  43.38  82  59  1.38  20  137  67.12  59.85  92  82  1.12  MRS3 Score  N  Predicted Risk of 30-Day Mortality  Observed 30-Day Mortality Rate  Predicted 30-Day Mortality  Observed 30-Day Mortality  Ratio of Observed to Expected Mortality  1  46,255  0.59  0.58  271  267  1.01  2  124,129  0.79  0.79  986  976  1.01  3  199,554  1.08  0.93  2,150  1,864  1.15  4  207,875  1.46  1.28  3,035  2,658  1.14  5  124,279  1.98  2.12  2,456  2,628  0.93  6  113,835  2.67  2.89  3,039  3,287  0.92  7  80,065  3.60  3.90  2,880  3,125  0.92  8  51,509  4.83  5.05  2,488  2,600  0.96  9  38,639  6.46  6.72  2,496  2,598  0.96  10  21,656  8.59  8.83  1,860  1,912  0.97  11  11,687  11.33  11.25  1,325  1,315  1.01  12  7,157  14.81  15.15  1,061  1,084  0.98  13  4,072  19.13  17.68  780  720  1.08  14  2,134  24.35  21.46  520  458  1.13  15  1,361  30.46  27.55  415  375  1.11  16  785  37.34  34.65  294  272  1.08  17  439  44.77  35.54  197  156  1.26  18  209  52.44  43.54  110  91  1.20  19  136  60.01  43.38  82  59  1.38  20  137  67.12  59.85  92  82  1.12  View Large Table 4. Observed and Predicted 30-Day Mortality, by MRS3 Score MRS3 Score  N  Predicted Risk of 30-Day Mortality  Observed 30-Day Mortality Rate  Predicted 30-Day Mortality  Observed 30-Day Mortality  Ratio of Observed to Expected Mortality  1  46,255  0.59  0.58  271  267  1.01  2  124,129  0.79  0.79  986  976  1.01  3  199,554  1.08  0.93  2,150  1,864  1.15  4  207,875  1.46  1.28  3,035  2,658  1.14  5  124,279  1.98  2.12  2,456  2,628  0.93  6  113,835  2.67  2.89  3,039  3,287  0.92  7  80,065  3.60  3.90  2,880  3,125  0.92  8  51,509  4.83  5.05  2,488  2,600  0.96  9  38,639  6.46  6.72  2,496  2,598  0.96  10  21,656  8.59  8.83  1,860  1,912  0.97  11  11,687  11.33  11.25  1,325  1,315  1.01  12  7,157  14.81  15.15  1,061  1,084  0.98  13  4,072  19.13  17.68  780  720  1.08  14  2,134  24.35  21.46  520  458  1.13  15  1,361  30.46  27.55  415  375  1.11  16  785  37.34  34.65  294  272  1.08  17  439  44.77  35.54  197  156  1.26  18  209  52.44  43.54  110  91  1.20  19  136  60.01  43.38  82  59  1.38  20  137  67.12  59.85  92  82  1.12  MRS3 Score  N  Predicted Risk of 30-Day Mortality  Observed 30-Day Mortality Rate  Predicted 30-Day Mortality  Observed 30-Day Mortality  Ratio of Observed to Expected Mortality  1  46,255  0.59  0.58  271  267  1.01  2  124,129  0.79  0.79  986  976  1.01  3  199,554  1.08  0.93  2,150  1,864  1.15  4  207,875  1.46  1.28  3,035  2,658  1.14  5  124,279  1.98  2.12  2,456  2,628  0.93  6  113,835  2.67  2.89  3,039  3,287  0.92  7  80,065  3.60  3.90  2,880  3,125  0.92  8  51,509  4.83  5.05  2,488  2,600  0.96  9  38,639  6.46  6.72  2,496  2,598  0.96  10  21,656  8.59  8.83  1,860  1,912  0.97  11  11,687  11.33  11.25  1,325  1,315  1.01  12  7,157  14.81  15.15  1,061  1,084  0.98  13  4,072  19.13  17.68  780  720  1.08  14  2,134  24.35  21.46  520  458  1.13  15  1,361  30.46  27.55  415  375  1.11  16  785  37.34  34.65  294  272  1.08  17  439  44.77  35.54  197  156  1.26  18  209  52.44  43.54  110  91  1.20  19  136  60.01  43.38  82  59  1.38  20  137  67.12  59.85  92  82  1.12  View Large We identified a cut point of 7 to indicate low versus high risk of mortality. A MRS3 score of 8 or higher was associated with a 5.4 times increase in the odds of 30-day mortality compared to a score of 7 or lower (OR = 5.408, CI = 5.726, 5.543) and a 4.2 times increase in the odds of 60-day mortality (OR = 4.215, CI = 4.140, 4.291). In sensitivity analyses, we found a relatively small loss of precision by working with the MRS3 (a simpler model and scoring system) that we developed as opposed to the logistic regression model predicting mortality using the detailed coefficients for each indicator (results from the fully weighted model can be found in the Supplementary Table 3). Discussion As NHs increasingly care for more clinically complex and frail patients, a tool to enable clinicians to identify patients who may be close to the end of life could potentially serve as a trigger for advance care planning. Furthermore, with the variation in the types of patients admitted to NHs, a score, or set of items, that strongly predict mortality is needed to adequately adjust for patients’ risk and facility case mix. Results from our development and validation study suggest that the MRS3 is a useful new tool to predict 30-day mortality and adjust for patient risk using the MDS 3.0. This mortality risk score, developed for patients newly admitted to the NH, is of benefit to clinicians and health services researchers for multiple reasons. First, the MRS3 can be a helpful tool not only to provide prognosis information that can aide in guiding care, but it can also assist with planning for care. For example, a patient’s MRS3 score can serve as a signal for end of life and advanced care planning discussions with the patient and their family. Second, the MRS3 can inform goals of care, particularly for patients with high MRS3 scores and predicted to be near the end of life. Where to establish the cut point to identify patients who should be approached about advance care planning or for whom a hospice discussion might be necessary is an issue that will require further investigation. However, our results suggest that a score of 8 and above is associated with over five times the increased odds of mortality and might serve as this cut point. While this cut point maximizes the predictive value, the decision of the cut point must take into account the goals. For example, health providers may want to focus on sensitivity rather than specificity to ensure all that would benefit from advance care planning are offered the chance to state their wishes prior to a period of possible future mental incapacity. While the way the score was developed maximizes its clinical applicability, it is also useful to researchers who will benefit from a score that can be readily calculated and used with the MDS 3.0. This score has the potential to be used as adjustment for mortality in examining the impact of a new treatment or comparing quality across facilities. As with the development of any score, there are some limitations to note. First and foremost, the MRS3 was designed as a clinical aid and clinicians may have additional knowledge that better informs prognostication (eg, knowing that a patient is stopping dialysis). This score was not designed to take the place of a clinical opinion and training for appropriate prognostication and therefore should be approached and interpreted as such. Secondly, this score is not based solely on biological-based markers or objective physiological measures; the MRS3 relies on subjective clinical appraisals for at least some of the indicators. However, MDS coordinators and those who complete the assessments have been trained to obtain this information for care planning, payment, and public reporting; therefore, it can be argued that there is some standardization and objectivity in collecting this information and the MDS has been validated for its accuracy in previous studies (31–34,36,37). In addition, there are several items that comprise the MRS3 that are potentially resolvable or treatable. Therefore, future research should examine the use of the MRS3 as a time-dependent covariate to determine the impact of changes in the MRS3 score over time on improving the prediction of mortality. In addition, there were limitations of our approach to validation of the MRS3. First, the score was developed and tested on an admission cohort of NH patients. Therefore, the score may not be appropriate or as accurate in predicting mortality for long-stay NH residents as previous work suggests that long-stay residents tended to be older, male, white, unmarried, low-income subsidy recipients, have multiple comorbidities, and have higher rates of mortality than residents with shorter stays (41). Future work is needed to develop a separate mortality risk score for other cohorts of NH residents such as long-stay residents and those readmitted to the facility. In addition, the score relied only on items present on every assessment. There are a number of items present on the Federal OBRA-required full admission assessment that are not included on the 5-day prospective payment assessment that may have enhanced the performance of this score (ie, Cancer, Cirrhosis, and Renal Insufficiency, Renal Failure, or End-Stage Renal Disease). Despite these limitations, there are many strengths to the development and testing of this score that are worthy of mention. First, this score was developed to predict mortality with population-based administrative information for the national population of new admissions to NHs. Second, it was validated with an external, national population of new admissions to NHs in the following year. Third, this score is comprised of real-time data on health conditions and functioning in the MDS 3.0 that are routinely collected for every patient in every facility by trained staff, unlike other mortality scores or frailty indices requiring information from various data sources including surveys and lab results (42–44). Fourth, we were able to model a patient’s risk of death, regardless of location at time of death, for all Medicare beneficiaries admitted to NHs. Fifth, the transformation of a logistic regression model to an index provides greater clinical utility for decision-making and care planning. Finally, because the score is not a disease-specific measure, unlike many mortality risk scores with good performance characteristics, it allows for prognostication and risk adjustment across different diseases and risk profiles. In summary, the MRS3 is a potential useful tool for identifying mortality risk among patients admitted to NHs. The MRS3 provides valuable information that clinicians, service providers, researchers, and policy makers can use to inform decisions related to the care of patients admitted to NHs without imposing additional workload or assessment burden. Supplementary Material Supplementary data is available at The Journals of Gerontology, Series A: Biological Sciences and Medical Sciences online. Funding This work was supported by the National Institute of Aging at the National Institutes of Health (P01AG027296); and the U.S. Department of Veterans Affairs Health Services Research and Development Service (CDA14-422 to K.S.T.). The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs, the National Institutes of Health, or the U.S. government. Conflict of Interest Vince Mor is the Chair of the Independent Quality Committee at HCR ManorCare and am compensated for this service. HCR is a provider of short-term, post-hospital services, and long-term care. The company has a network of more than 500 nursing and rehabilitation centers, assisted living facilities, outpatient rehabilitation clinics, and hospice and home health care agencies. Vince Mor is a paid consultant to NaviHealth, Inc. and chair their Scientific Advisory Board. NaviHealth is wholly owned by Cardinal Health. The company offers post-acute care (PAC) management and services to more than 1.5 million beneficiaries in all regions of the country through its partnerships with health plans and health systems. Vince Mor is a former Director at PointRight, Inc. While he no longer provides any services or holds any positions at PointRight, he still has a small amount of equity in the company. PointRight is a private company based in Cambridge, MA. It provides predictive analytics solutions to thousands of post-acute providers, long-term care providers, hospitals, payers and insurance organizations. References 1. Liu K, Wissoker D, Swett A. Nursing home use by dual-eligible beneficiaries in the last year of life. Inquiry . 2007; 44: 88– 103. doi: 10.5034/inquiryjrnl_44.1.88 Google Scholar CrossRef Search ADS PubMed  2. Teno JM, Clarridge BR, Casey V, et al.   Family perspectives on end-of-life care at the last place of care. JAMA . 2004; 291: 88– 93. doi: 10.1001/jama.291.1.88 Google Scholar CrossRef Search ADS PubMed  3. Weeks JC, Cook EF, O’Day SJ, et al.   Relationship between cancer patients’ predictions of prognosis and their treatment preferences. JAMA . 1998; 279: 1709– 1714. doi: 10.1001/jama.279.21.1709 Google Scholar CrossRef Search ADS PubMed  4. Murphy DJ, Burrows D, Santilli S, et al.   The influence of the probability of survival on patients’ preferences regarding cardiopulmonary resuscitation. N Engl J Med . 1994; 330: 545– 549. doi: 10.1056/NEJM199402243300807 Google Scholar CrossRef Search ADS PubMed  5. Fox KA, Dabbous OH, Goldberg RJ, et al.   Prediction of risk of death and myocardial infarction in the six months after presentation with acute coronary syndrome: Prospective multinational observational study (GRACE). BMJ . 2006; 333: 1091. doi: 10.1136/bmj.38985.646481.55 Google Scholar CrossRef Search ADS PubMed  6. Antman EM, Cohen M, Bernink PJ, et al.   The TIMI risk score for unstable angina/non-ST elevation MI: A method for prognostication and therapeutic decision making. JAMA . 2000; 284: 835– 842. Google Scholar CrossRef Search ADS PubMed  7. Ranson JH, Rifkind KM, Roses DF, Fink SD, Eng K, Spencer FC. Prognostic signs and the role of operative management in acute pancreatitis. Surg Gynecol Obstet . 1974; 139: 69– 81. Google Scholar PubMed  8. Gawande AA, Kwaan MR, Regenbogen SE, Lipsitz SA, Zinner MJ. An Apgar score for surgery. J Am Coll Surg . 2007; 204: 201– 208. doi: 10.1016/j.jamcollsurg.2006.11.011 Google Scholar CrossRef Search ADS PubMed  9. Sobol JB, Gershengorn HB, Wunsch H, Li G. The surgical Apgar score is strongly associated with intensive care unit admission after high-risk intraabdominal surgery. Anesth Analg . 2013; 117: 438– 446. doi: 10.1213/ANE.0b013e31829180b7 Google Scholar CrossRef Search ADS PubMed  10. Reynolds PQ, Sanders NW, Schildcrout JS, Mercaldo ND, St Jacques PJ. Expansion of the surgical Apgar score across all surgical subspecialties as a means to predict postoperative mortality. Anesthesiology . 2011; 114: 1305– 1312. doi: 10.1097/ALN.0b013e318219d734. Google Scholar CrossRef Search ADS PubMed  11. Tate JP, Justice AC, Hughes MD, et al.   An internationally generalizable risk index for mortality after one year of antiretroviral therapy. AIDS . 2013; 27: 563– 572. doi: 10.1097/QAD.0b013e32835b8c7f Google Scholar CrossRef Search ADS PubMed  12. Justice AC, Modur SP, Tate JP, et al.  ; NA-ACCORD and VACS Project Teams. Predictive accuracy of the Veterans Aging Cohort Study index for mortality with HIV infection: A North American cross cohort analysis. J Acquir Immune Defic Syndr . 2013; 62: 149– 163. doi: 10.1097/QAI.0b013e31827df36c Google Scholar CrossRef Search ADS PubMed  13. Kline JA, Roy PM, Than MP, et al.   Derivation and validation of a multivariate model to predict mortality from pulmonary embolism with cancer: The POMPE-C tool. Thromb Res . 2012; 129: e194– e199. doi: 10.1016/j.thromres.2012.03.015 Google Scholar CrossRef Search ADS PubMed  14. van der Hulle T, den Exter PL, Kooiman J, van der Hoeven JJ, Huisman MV, Klok FA. Meta-analysis of the efficacy and safety of new oral anticoagulants in patients with cancer-associated acute venous thromboembolism. J Thromb Haemost . 2014; 12: 1116– 1120. doi: 10.1111/jth.12605 Google Scholar CrossRef Search ADS PubMed  15. Vickers AJ, Kramer BS, Baker SG. Selecting patients for randomized trials: A systematic approach based on risk group. Trials . 2006; 7: 30. doi: 10.1186/1745-6215-7-30 Google Scholar CrossRef Search ADS PubMed  16. Hernández AV, Steyerberg EW, Habbema JD. Covariate adjustment in randomized controlled trials with dichotomous outcomes increases statistical power and reduces sample size requirements. J Clin Epidemiol . 2004; 57: 454– 460. doi: 10.1016/j.jclinepi.2003.09.014 Google Scholar CrossRef Search ADS PubMed  17. Hernández AV, Eijkemans MJ, Steyerberg EW. Randomized controlled trials with time-to-event outcomes: How much does prespecified covariate adjustment increase power? Ann Epidemiol . 2006; 16: 41– 48. doi: 10.1016/j.annepidem.2005.09.007 Google Scholar CrossRef Search ADS PubMed  18. Iezzoni LI. Risk Adjustment for Measuring Health Care Outcomes . 3rd ed. Chicago, IL: Health Administration Press; 2003. 19. Iezzoni L. Risk Adjustment for Measuring Health Care Outcomes . 4th ed. Chicago, IL: Health Administration Press; 2012. 20. Centers for Medicare & Medicaid Services. MDS 3.0 RAI Manual . 2017. 21. Mitchell SL, Kiely DK, Hamel MB, Park PS, Morris JN, Fries BE. Estimating prognosis for nursing home residents with advanced dementia. JAMA . 2004; 291: 2734– 2740. doi: 10.1001/jama.291.22.2734 Google Scholar CrossRef Search ADS PubMed  22. Mitchell SL, Miller SC, Teno JM, Davis RB, Shaffer ML. The advanced dementia prognostic tool: A risk score to estimate survival in nursing home residents with advanced dementia. J Pain Symptom Manage . 2010; 40: 639– 651. doi: 10.1016/j.jpainsymman.2010.02.014 Google Scholar CrossRef Search ADS PubMed  23. Abicht-Swensen LM, Debner LK. The minimum data set 2.0: A functional assessment to predict mortality in nursing home residents. Am J Hosp Palliat Care . 1999; 16: 527– 532. doi: 10.1177/104990919901600308 Google Scholar CrossRef Search ADS PubMed  24. Flacker JM, Kiely DK. Mortality-related factors and 1-year survival in nursing home residents. J Am Geriatr Soc . 2003; 51: 213– 221.doi: 10.1046/j.1532-5415.2003.51060.x Google Scholar CrossRef Search ADS PubMed  25. Porock D, Oliver DP, Zweig S, et al.   Predicting death in the nursing home: Development and validation of the 6-month Minimum Data Set mortality risk index. J Gerontol A Biol Sci Med Sci . 2005; 60: 491– 498. doi: 10.1093/gerona/60.4.491 Google Scholar CrossRef Search ADS PubMed  26. Brink P, Kelley ML. Death in long-term care: A brief report examining factors associated with death within 31 days of assessment. Palliat Care . 2015; 9: 1– 5. doi: 10.4137/PCRT.S20347 Google Scholar PubMed  27. van Dijk PT, Mehr DR, Ooms ME, et al.   Comorbidity and 1-year mortality risks in nursing home residents. J Am Geriatr Soc . 2005; 53: 660– 665. doi: 10.1111/j.1532-5415.2005.53216.x Google Scholar CrossRef Search ADS PubMed  28. Levy C, Kheirbek R, Alemi F, et al.   Predictors of six-month mortality among nursing home residents: Diagnoses may be more predictive than functional disability. J Palliat Med . 2015; 18: 100– 106. doi: 10.1089/jpm.2014.0130 Google Scholar CrossRef Search ADS PubMed  29. Porock D, Parker-Oliver D, Petroski GF, Rantz M. The MDS mortality risk index: The evolution of a method for predicting 6-month mortality in nursing home residents. BMC Res Notes . 2010; 3: 200. doi: 10.1186/1756-0500-3-200 Google Scholar CrossRef Search ADS PubMed  30. Hirdes JP, Frijters DH, Teare GF. The MDS-CHESS scale: A new measure to predict mortality in institutionalized older people. J Am Geriatr Soc . 2003; 51: 96– 100. doi: 10.1034/j.1601-5215.2002.51017.x Google Scholar CrossRef Search ADS PubMed  31. Saliba D, Buchanan J. Making the investment count: Revision of the Minimum Data Set for nursing homes, MDS 3.0. J Am Med Dir Assoc . 2012; 13( 7): 602– 610. doi: 10.1016/j.jamda.2012.06.002 Google Scholar CrossRef Search ADS PubMed  32. Saliba D, Buchanan J, Edelen MO, et al.   MDS 3.0: Brief interview for mental status. J Am Med Dir Assoc . 2012; 13( 7): 611– 617. doi: 10.1016/j.jamda.2012.06.004 Google Scholar CrossRef Search ADS PubMed  33. Saliba D, DiFilippo S, Edelen MO, Kroenke K, Buchanan J, Streim J. Testing the PHQ-9 interview and observational versions (PHQ-9 OV) for MDS 3.0. J Am Med Dir Assoc . 2012; 13: 618– 625. doi: 10.1016/j.jamda.2012.06.003 Google Scholar CrossRef Search ADS PubMed  34. Saliba D, Jones M, Streim J, Ouslander J, Berlowitz D, Buchanan J. Overview of significant changes in the Minimum Data Set for nursing homes version 3.0. J Am Med Dir Assoc . 2012; 13: 595– 601. doi: 10.1016/j.jamda.2012.06.001 Google Scholar CrossRef Search ADS PubMed  35. Kosar CM, Thomas KS, Inouye SK, Mor V. Delirium during postacute nursing home admission and risk for adverse outcomes. J Am Geriatr Soc . 2017; 65: 1470– 1475. doi: 10.1111/jgs.14823 Google Scholar CrossRef Search ADS PubMed  36. Rahman M, Tyler D, Acquah JK, Lima J, Mor V. Sensitivity and specificity of the Minimum Data Set 3.0 discharge data relative to Medicare claims. J Am Med Dir Assoc . 2014; 15: 819– 824. doi: 10.1016/j.jamda.2014.06.017 Google Scholar CrossRef Search ADS PubMed  37. Thomas KS, Dosa D, Wysocki A, Mor V. The Minimum Data Set 3.0 cognitive function scale. Med Care . 2017; 55( 9): e68– e72. doi: 10.1097/MLR.0000000000000334 Google Scholar CrossRef Search ADS PubMed  38. Thomas KS, Wysocki A, Intrator O, Mor V. Finding gertrude: The resident’s voice in Minimum Data Set 3.0. J Am Med Dir Assoc . 2014; 15( 11): 802– 806. doi: 10.1016/j.jamda.2014.01.012 Google Scholar CrossRef Search ADS PubMed  39. Wysocki A, Thomas KS, Mor V. Functional improvement among short-stay nursing home residents in the MDS 3.0. J Am Med Dir Assoc . 2015; 16: 470– 474. doi: 10.1016/j.jamda.2014.11.018 Google Scholar CrossRef Search ADS PubMed  40. Ruopp MD, Perkins NJ, Whitcomb BW, Schisterman EF. Youden index and optimal cut-point estimated from observations affected by a lower limit of detection. Biom J . 2008; 50: 419– 430. doi: 10.1002/bimj.200710415 Google Scholar CrossRef Search ADS PubMed  41. Wei YJ, Simoni-Wastila L, Zuckerman IH, Brandt N, Lucas JA. Algorithm for identifying nursing home days using medicare claims and minimum data set assessment data. Med Care . 2016; 54: e73– e77. doi: 10.1097/MLR.0000000000000109 Google Scholar CrossRef Search ADS PubMed  42. Ritt M, Jäger J, Ritt JI, Sieber CC, Gaßmann KG. Operationalizing a frailty index using routine blood and urine tests. Clin Interv Aging . 2017; 12: 1029– 1040. doi: 10.2147/CIA.S131987 Google Scholar CrossRef Search ADS PubMed  43. Rockwood K, Mitnitski AB, MacKnight C. Some mathematical models of frailty and their clinical implications. Rev Clin Gerontol . 2002; 12: 109– 117. doi: 10.1017/S0959259802012236 Google Scholar CrossRef Search ADS   44. Rockwood K, Song X, MacKnight C, et al.   A global clinical measure of fitness and frailty in elderly people. CMAJ . 2005; 173: 489– 495. doi: 10.1503/cmaj.050051 Google Scholar CrossRef Search ADS PubMed  Published by Oxford University Press on behalf of The Gerontological Society of America 2018. This work is written by (a) US Government employee(s) and is in the public domain in the US. This work is written by (a) US Government employee(s) and is in the public domain in the US.

Journal

The Journals of Gerontology Series A: Biomedical Sciences and Medical SciencesOxford University Press

Published: Mar 5, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off