A Machine Learning Recommender System to Tailor Preference Assessments to Enhance Person-Centered Care Among Nursing Home Residents

A Machine Learning Recommender System to Tailor Preference Assessments to Enhance Person-Centered... Abstract Background and Objectives Nursing homes (NHs) using the Preferences for Everyday Living Inventory (PELI-NH) to assess important preferences and provide person-centered care find the number of items (72) to be a barrier to using the assessment. Research Design and Methods Using a sample of n = 255 NH resident responses to the PELI-NH, we used the 16 preference items from the MDS 3.0 Section F to develop a machine learning recommender system to identify additional PELI-NH items that may be important to specific residents. Much like the Netflix recommender system, our system is based on the concept of collaborative filtering whereby insights and predictions (e.g., filters) are created using the interests and preferences of many users. The algorithm identifies multiple sets of “you might also like” patterns called association rules, based upon responses to the 16 MDS preferences that recommends an additional set of preferences with a high likelihood of being important to a specific resident. Results In the evaluation of the combined apriori and logistic regression approach, we obtained a high recall performance (i.e., the ratio of correctly predicted preferences compared with all predicted preferences and nonpreferences) and high precision (i.e., the ratio of correctly predicted rules with respect to the rules predicted to be true) of 80.2% and 79.2%, respectively. Discussion and Implications The recommender system successfully provides guidance on how to best tailor the preference items asked of residents and can support preference capture in busy clinical environments, contributing to the feasibility of delivering person-centered care. CMS datasets (OSCAR MDS), Long-term care, Nursing homes, Quality of Care, Technology Current trends in long-term care, in concert with the Centers for Medicare and Medicaid Services (CMS) Quality of Life guidelines, call for a shift in focus to person-centered care. Evolving conceptualizations of optimal care in the NH further emphasize the need to “know the person” to deliver individualized, holistic care. In the United States, preference assessment has been incorporated into the Minimum Data Set (MDS 3.0) through Section F Preferences for Customary Routine and Activities upon admission and annually thereafter for NH residents. Section F includes eight preferences for personal care and eight items for activity preferences (Saliba & Buchanan, 2008). These 16 preference questions were informed by the PELI-NH (Van Haitsma et al., 2013), which includes 72 preference questions across five domains. The state of Ohio has recently mandated that the PELI-NH be used for pay for performance initiatives in NH settings (http://codes.ohio.gov/oac/5160-3-58). One major barrier reported by 61% of Ohio providers to meeting this mandate included the time required for staff to complete the interview (unpublished data). Although there is a desire by NH providers to use the PELI-NH to personalize care, they seek ways to more efficiently identify important preferences. Knowledge of a resident’s everyday care preferences provides the foundation for ongoing individualized care planning. However, obtaining idiographic information requires asking an individual a multitude of questions, which creates an enhanced burden of paperwork and assessments in addition to respondent fatigue. For example, nursing staff perceive that 40%–90% of their workday is spent on paperwork and documentation (Ashcraft, Cherry, & Owen, 2007). This time spent completing paperwork not only impedes a nursing staff’s ability to provide direct care (Ashcraft et al., 2007) but has also been found to be a source of job dissatisfaction for nurses across various care settings (Trossman, 2002). Therefore, there has been a movement toward standardization to address the burdens of paperwork and to streamline data collection so it can be used across transitions in care. Concern regarding the continuity of an individual’s care arises during the transition across different systems of care as there is often a lack of care coordination between long-term and postacute care providers (LTPAC; Clark, Elswick, Gabriel, Gurupur, & Wisniewski, 2016). As a result the Improving Medicare Post-Acute Care Transformation Act of 2014 (IMPACT Act) was enacted (Clark et al., 2016) requiring that all LTPAC providers, which includes long-term care hospitals, skilled nursing facilities, home health agencies, and inpatient rehabilitation facilities, standardize assessment data and reporting to allow for seamless sharing and interoperability between providers (CMS, 2015). The standardization of paperwork and documentation not only helps to achieve person-centered outcomes for the individual transitioning across care settings but also improves assessment and care coordination as providers are able to be more efficient in sharing relevant information (Edelen et al., 2017). To assist in the reduction of fatigue (Hess et al., 2012) while also providing personalization of resident experience we have explored the application of “recommender systems” to preference assessment in the NH setting. Recommender systems (Aggarwal, 2016) have gained in popularity as a practical application of machine learning. Consumer websites have long seen demonstration of this technology in areas such as e-commerce (Amazon), movies (Netflix), and music (Pandora) in the form of ratings systems that are used to help users identify potential likes and dislikes. Different strategies for implementing recommender systems have been suggested, including the use of association rules (Bendakir & Aimeur, 2006; Cakir & Aras, 2012; Lin, 2000). In the health care domain, the idea of using data mining is not a new one (Patel et al., 2009; Simovici, 2012). Applications have included treatment effectiveness and condition identification (Koh & Tan, 2005). Hu, for instance, has suggested using the Apriori algorithm for mining medical data as a means for diagnosis of conditions (Hu, 2010). In the area of long-term care, we are unaware of work being done to understand resident preferences using recommendations. Therefore, we developed an approach that personalizes preferences tailored to each individual resident respondent through combining the Apriori algorithm (Agrawal & Srikant, 1994) and logistic regression configured with a generalized linear regression model (Nelder & Wedderburn, 1972). The idea behind the preference recommender is to develop a software algorithm that suggests additional PELI-NH preference items to ask using responses from a residents’ 16 MDS items. The preference recommender can provide guidance or decision support to provider communities as they explore the next best set of preference questions to ask each NH resident. Our initial investigations have focused on using a rule-based collaborative filtering approach, with responses to preference questions used in a manner analogous to selection of products in a market basket. In collaborative filtering, insights and predictions (e.g., filters) are created using the interests and preferences of many users (i.e., by collaborating). Therefore, this study seeks to apply data mining and machine learning techniques to explore their utility in providing insight into a reduced set of more targeted and meaningful potential preferences specific to each individual. The ultimate goal is to tailor preference assessments based upon the knowledge gained from resident responses to the 16 MDS 3.0 Section F preference items thereby reducing the need to ask an additional 56 PELI-NH items. Design and Methods Procedures and Participants The PELI-NH for NH residents was administered via face-to-face interviews by research assistants with n = 255 NH residents with an Mini-Mental State Examination (MMSE) ≥ 13 (MMSE; Folstein, Folstein, & McHugh, 1975) at baseline (T1) and 3 months later (T2). The 72 items in the PELI-NH include the 16 MDS 3.0 Section F Preferences for Customary Routine and Activities. NH residents were recruited from 28 locations in the suburbs of a major metropolitan East Coast area of the United States. The facility contact person from each NH identified residents who would enjoy participating in an interview about their likes and dislikes, who were up to moderately cognitively capable (MMSE ≥ 13), English speaking, and had a length of stay of at least 1 week and an expected stay of 3 months. The final sample consisted of 255 NH residents completing both the T1 and T2 interviews of 342 who completed T1. Our attrition rate from T1 to T2 was 25.4% which was due to death, transfer, change in cognitive ability, withdrawal, or change in medical stability over the 3 months. Additional recruitment details can be found here (Abbott et al., 2018; Bangerter, Abbott, Heid, Eshraghi, & Van Haitsma, 2017). Informed consent for participation in the study was established in-person and was repeated before the follow-up interview 3 months later. The PELI (Van Haitsma et al., 2013, 2014) is a comprehensive, reliable assessment instrument that examines the content, meaning, and importance of 72 psychosocial preferences for social contact, growth activities, leisure activities, self-dominion, and enlisting others in care (Carpenter, Van Haitsma, Ruckdeschel, & Lawton, 2000). The PELI asks respondents to rate these items using the stem “How important is it to you to. . . [Insert preference]” with response options on a 4-point Likert scale from 1 (very important) to 4 (not important at all). The content and structure were developed via concept mapping with n = 550 older adults receiving home health services (Van Haitsma et al., 2013). The second iteration of the PELI was its modification for use in a NH population (PELI-NH) based on results from cognitive interviewing techniques with n = 70 residents (Curyto, Van Haitsma, & Towsley, 2016). Cognitive interviews resulted in the 72-item PELI-NH, which assesses NH resident preferences grouped into the five originally derived concept mapping domains (Carpenter et al., 2000). Additional studies have reported that the PELI-NH has high levels of face validity as well as convergent and discriminant validity (Van Haitsma, et al., 2013). The NH resident test–retest reliability over 1 week is 87.3% (Van Haitsma et al., 2014) and 73.35% with proxy respondents (Heid, Bangerter, Abbott, & Van Haitsma, 2017). For the analysis in this current study, response options were collapsed into important (very and somewhat) or not important (not very and not important at all). Developing the Recommender for Daily Preferences We have developed an approach that personalizes preferences tailored to each individual resident through combining the Apriori algorithm (Agrawal & Srikant, 1994) and a logistic regression configured with a generalized linear regression model (Nelder & Wedderburn, 1972). The Apriori algorithm is used to identify frequent itemsets of preferences in a dataset, while logistic regression is used to assess the quality of the mined rules. In our context, itemsets describe the preferences that most often co-occur in a dataset. Specifically, two preferences A and B co-occur when a resident indicates “very important” or “somewhat important” for each preference A and B. Itemsets that include multiple preferences {A, B, C} can be used to build rulesets that infer association rules such as {A, B} → {C}, where knowledge of the co-occurrence (or co-preference) of the set {A, B} indicates a statistically likely occurrence or preference for an item C. The Apriori approach is often used to identify rulesets for market baskets to make recommendations for product upsells. For example, when a shopper chooses a set of products {A, B}, a recommendation is made for a shopper to consider product {C} based on the behavior of shoppers that also chose the set of products A and B. For instance, a grocery store recommendation system might suggest that if a market basket contains eggs and peanut butter then milk should be added to the basket. Using the shopping analogy, we consider preferences of residents as analogous to the selection of products for a market basket by an individual customer. As such, we configure the Apriori algorithm to consider each of the 72 preferences of the PELI-NH as if it were a product to be purchased by a resident. Expressing a positive preference (e.g., very important or somewhat important) is held as analogous to purchasing of a product. Conversely, expressing a negative preference (e.g., not very important or not important at all) is held as analogous to not purchasing a product. Another key aspect of the Apriori algorithm is the use of transactions to build the frequent itemsets and association rules, the collection of which constitutes a trained learner. Trained learners are machine learning algorithms that are established by using patterns present in a dataset. They are said to be trained since the patterns must be discovered over time through analysis of data rather than by creation of a strict algorithm. In the case of our approach, the recommender is trained with transactions that provide the basis of evidence for how often various items (or in our case, preferences) co-occur. A transaction, in this context, corresponds to a single visit by a shopper to a store to purchase products collected in a market basket. Each visit constitutes a separate transaction, with separate visits by a single shopper being considered different transactions. In the case of our learner, the n = 255 different residents, each with two sets of responses to the PELI-NH, results in n = 510 transactions available for training the learner. Individual transactions are considered independently of previous responses for a given resident and are merely used in establishing the patterns for the learner. While a larger dataset will potentially improve the quality of the learner over time, the Apriori algorithm identifies frequent itemsets that are present in the data as provided. As such, patterns are identified regardless of the size of the inputs. Recommendations using Apriori is based on the idea that when a shopper starts to fill a basket with products, the learner can make suggestions according to the items that frequently co-occur. When applied to the full collection of PELI-NH items, our approach is to interview a resident to explicitly determine responses to the 16 MDS 3.0 Section F questions. Using that as a market basket, the approach uses the trained learner to indicate which preferences from the remaining 56 PELI-NH questions co-occur with those MDS preferences that have been identified by the resident. In all cases, the 56 PELI-NH questions are only drawn from the training set for training purposes. Subsequent predictions only require a resident to respond to the 16 MDS 3.0 questions. Using the shopping analogy, this would correspond to identifying which products commonly appear in market baskets that look similar to the one assembled by the shopper. Supplementary Appendix B lists each question from the MDS Section F along with the percentages of our sample who endorsed the preference as very or somewhat important. Analyses To generate a list of recommended questions to ask an individual resident, the recommendation system is first given as input the set of 16 MDS preference responses. The recommendation engine compares these preference inputs with the antecedents (left-hand side) of all of its rules, searching for matches. For all matches that are found, the consequents (right-hand sides) of the rules are aggregated and converged into a single list. Each aggregated consequent is given a set of summary metrics of all of the association rules that predicted it. These consequents and their summary metrics are then passed through a generalized logistic regression model, which classifies whether each consequent is likely to be a good recommendation or not. The consequents of all predictions labeled as good are then returned as the final list of recommended questions. The Apriori algorithm is typically configured by setting thresholds for two parameters: minimum support and minimum confidence. In this context, support indicates how often an itemset X appears in a dataset, while the confidence indicates how often a rule is found to be true. In particular, the support for an association rule A → B is defined as  Support(A → B)=|A ∪​B|Total no. of transactions   where “Total no. of transactions” correspond to the number of responses to the PELI survey used in developing the learner (e.g., the evidence for the rule). For our dataset, antecedents are drawn from the 16 MDS 3.0 Section F set of questions, the consequents are drawn from the 56 PELI-NH questions, and the total transactions corresponds to the 510 total responses collected. The confidence parameter for an association rule A → B is defined as  Confidence(A → B)= |A ∪​B||A| and indicates how often a consequent B occurs in rules involving A. For our implementation, we found that a minimum support of 0.01 and a minimum confidence of 0.8 produced the best results in that they expose the largest number of rules (support) and provide the strongest association between antecedent and consequent (confidence). In addition to the Apriori algorithm, we applied the use of logistic regression configured with a generalized linear regression model to help identify which rules had the highest potential utility. This model assigns the properties of the Apriori algorithm as the independent variables, such as support, confidence, lift and their corresponding average and maximum values to predict whether a consequent is likely to be important or not. In general, this allowed us to achieve a relatively high precision while keeping support relatively low to allow for inclusion of a larger set of rules. Finally, given that support, confidence, and lift all play a part in a rule’s overall usefulness, these measures are individually unsuitable for ranking or ordering of rules. Instead, a score metric is used, which is simply the product of support and confidence (Lin, 2000), and similarly liftscore which is the product of support, confidence, and lift. Results Demographic Characteristics NH residents were mostly widowed (44%) white (77%) females (67.8%) with a mean age of 81 years (standard deviation [SD] = 11.21) and a high school education (54%). Residents had an average length of stay of 924 days and an average MMSE score of 24.6 (SD = 3.9; Supplementary Appendix A). To illustrate our approach, we present results from two residents with different preference profiles. As with any instance of our approach, we begin with training a learner using the combined apriori/logistic regression approach to build the consequent rule sets, using a minimum confidence of 0.8 and a minimum support of 0.01 for the Apriori algorithm. We chose a relatively high confidence to strongly associate antecedent preferences with consequent preferences and chose a relatively low support in order to expose a wider variety of rulesets. In the approach, we started with responses to the 16 MDS 3.0 Section F questions (Table 1). Resident 1 is a 95-year old white female with a high school education. Resident 2 is 90-year old white male with some college education. For example, among the many preferences indicated, Resident 1 finds snack availability to be not important at all (as indicated by the assignment of a value of 4), while Resident 2 finds snack availability to be very important (as indicated by the assignment of a value of 1). We ran each individual resident’s preferences against learned recommendations that were built from 80% of the 510 total responses available in the dataset as generated by the combined apriori/logistic regression learner to arrive at individual sets of recommendations of questions to pursue from the remaining 56 non-MDS PELI-NH questions (Tables 2 and 3). The tables are sorted using the score metric, with the corresponding support, confidence, and lift scores shown for reference. In addition, the final two columns show the actual preference reported by the resident and the predicted or recommendation of the learner for comparison (see “pred” column in Tables 2 and 3). For example, in Table 2, the machine learner has recommended 39 of the 56 questions be considered in decreasing order based on the score. The actual column shows that the predictions for the first several questions are correct (as indicated by the “1”) with respect to the resident’s actual response. All of the recommended preferences were endorsed as important by the resident except for the row labeled “learning,” which corresponds to the question “How important is it to you to learn about topics that interest you?” Table 1. Actual Responses to MDS Section F Preferences Used to Inform Recommender of Two Residentsa MDS 3.0 Section F  Resident 1: white female, age 95, high school educated  Resident 2: white male, age 90, some college education  F0400 A  Choose clothes = 2  Choose clothes = 4  F0400 B  Care of personal things = 1  Care of personal things = 1  F0400 C  Choose bathing = 1  Choose bathing = 4  F0400 D  Snack availability = 4  Snack availability = 1  F0400 E  Bedtime = 2  Bedtime = 3  F0400 F  Who involved in care discussions = 1  Who involved in care discussions = 1  F0400 G  Private phone = 1  Private phone = 1  F0400 H  Lock things = 1  Lock things = 4  F0500 A  Reading material = 1  Reading material = 1  F0500 B  Music = 3  Music = 1  F0500 C  Pets = 4  Pets = 3  F0500 D  News = 3  News = 4  F0500 E  Group activities = 4  Group activities = 2  F0500 F  Activities = 1  Activities = 1  F0500 G  Outside time = 1  Outside time = 1  F0500 H  Religious services = 1  Religious services = 4  MDS 3.0 Section F  Resident 1: white female, age 95, high school educated  Resident 2: white male, age 90, some college education  F0400 A  Choose clothes = 2  Choose clothes = 4  F0400 B  Care of personal things = 1  Care of personal things = 1  F0400 C  Choose bathing = 1  Choose bathing = 4  F0400 D  Snack availability = 4  Snack availability = 1  F0400 E  Bedtime = 2  Bedtime = 3  F0400 F  Who involved in care discussions = 1  Who involved in care discussions = 1  F0400 G  Private phone = 1  Private phone = 1  F0400 H  Lock things = 1  Lock things = 4  F0500 A  Reading material = 1  Reading material = 1  F0500 B  Music = 3  Music = 1  F0500 C  Pets = 4  Pets = 3  F0500 D  News = 3  News = 4  F0500 E  Group activities = 4  Group activities = 2  F0500 F  Activities = 1  Activities = 1  F0500 G  Outside time = 1  Outside time = 1  F0500 H  Religious services = 1  Religious services = 4  Note:a1 = very important, 2 = somewhat important, 3 = not very important, 4 = not important at all. View Large Table 1. Actual Responses to MDS Section F Preferences Used to Inform Recommender of Two Residentsa MDS 3.0 Section F  Resident 1: white female, age 95, high school educated  Resident 2: white male, age 90, some college education  F0400 A  Choose clothes = 2  Choose clothes = 4  F0400 B  Care of personal things = 1  Care of personal things = 1  F0400 C  Choose bathing = 1  Choose bathing = 4  F0400 D  Snack availability = 4  Snack availability = 1  F0400 E  Bedtime = 2  Bedtime = 3  F0400 F  Who involved in care discussions = 1  Who involved in care discussions = 1  F0400 G  Private phone = 1  Private phone = 1  F0400 H  Lock things = 1  Lock things = 4  F0500 A  Reading material = 1  Reading material = 1  F0500 B  Music = 3  Music = 1  F0500 C  Pets = 4  Pets = 3  F0500 D  News = 3  News = 4  F0500 E  Group activities = 4  Group activities = 2  F0500 F  Activities = 1  Activities = 1  F0500 G  Outside time = 1  Outside time = 1  F0500 H  Religious services = 1  Religious services = 4  MDS 3.0 Section F  Resident 1: white female, age 95, high school educated  Resident 2: white male, age 90, some college education  F0400 A  Choose clothes = 2  Choose clothes = 4  F0400 B  Care of personal things = 1  Care of personal things = 1  F0400 C  Choose bathing = 1  Choose bathing = 4  F0400 D  Snack availability = 4  Snack availability = 1  F0400 E  Bedtime = 2  Bedtime = 3  F0400 F  Who involved in care discussions = 1  Who involved in care discussions = 1  F0400 G  Private phone = 1  Private phone = 1  F0400 H  Lock things = 1  Lock things = 4  F0500 A  Reading material = 1  Reading material = 1  F0500 B  Music = 3  Music = 1  F0500 C  Pets = 4  Pets = 3  F0500 D  News = 3  News = 4  F0500 E  Group activities = 4  Group activities = 2  F0500 F  Activities = 1  Activities = 1  F0500 G  Outside time = 1  Outside time = 1  F0500 H  Religious services = 1  Religious services = 4  Note:a1 = very important, 2 = somewhat important, 3 = not very important, 4 = not important at all. View Large Table 2. Recommender Predicted Resident Preferences—Resident 1 Consequent  score  liftscore  support  conf  lift  act  pred  Food choice  0.2526  0.3058  0.3053  0.8270  1.2094  1  1  Living place care  0.2275  0.2661  0.2714  0.8418  1.2826  1  1  Bed setup  0.2051  0.2537  0.2510  0.8166  1.3395  1  1  Hobbies  0.2020  0.2759  0.2343  0.8528  1.3761  1  1  Learning  0.1795  0.2281  0.2168  0.8449  1.4912  0  1  Staff care  0.1543  0.1913  0.1854  0.8505  1.3336  1  1  Family contact  0.1488  0.1701  0.1666  0.9089  1.2531  1  1  Choose mouth care  0.1478  0.1756  0.1751  0.8484  1.2293  1  1  Helps feel better  0.1443  0.1792  0.1740  0.8476  1.3171  1  1  Choose care professional  0.1415  0.1630  0.1645  0.8710  1.1652  0  1  Staff respect  0.1360  0.1745  0.1570  0.8704  1.3867  0  1  Bath frequency  0.1232  0.1510  0.1493  0.8420  1.2548  1  1  Privacy  0.1125  0.1565  0.1354  0.8574  1.4337  1  1  Choose hair care  0.1107  0.1377  0.1344  0.8398  1.2650  1  1  Bath time  0.1076  0.1333  0.1304  0.8416  1.2624  1  1  Choose nail care frequency  0.1076  0.1333  0.1304  0.8416  1.2624  1  1  Caregiver gender  0.0982  0.1368  0.1180  0.8603  1.4385  1  1  Choose used name  0.0982  0.1368  0.1180  0.8603  1.4385  1  1  Mental professional if sad  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Caregiver know bathroom needs  0.0758  0.0959  0.0924  0.8536  1.3795  1  1  Bedtime routine  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Choose wake time  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Follow morning routine  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Naps  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Room temp  0.0263  0.0428  0.0319  0.8213  1.8613  1  1  TV  0.0162  0.0246  0.0196  0.8343  1.5748  1  1  1-on-1 time  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Alone time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Children  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Friends contact  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Meet new people  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Room lighting  0.0146  0.0295  0.0180  0.8172  2.0758  1  1  Entertainment events  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Exercise  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Games  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Outdoor tasks  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Plant care  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Sports  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Watch movies with others  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Consequent  score  liftscore  support  conf  lift  act  pred  Food choice  0.2526  0.3058  0.3053  0.8270  1.2094  1  1  Living place care  0.2275  0.2661  0.2714  0.8418  1.2826  1  1  Bed setup  0.2051  0.2537  0.2510  0.8166  1.3395  1  1  Hobbies  0.2020  0.2759  0.2343  0.8528  1.3761  1  1  Learning  0.1795  0.2281  0.2168  0.8449  1.4912  0  1  Staff care  0.1543  0.1913  0.1854  0.8505  1.3336  1  1  Family contact  0.1488  0.1701  0.1666  0.9089  1.2531  1  1  Choose mouth care  0.1478  0.1756  0.1751  0.8484  1.2293  1  1  Helps feel better  0.1443  0.1792  0.1740  0.8476  1.3171  1  1  Choose care professional  0.1415  0.1630  0.1645  0.8710  1.1652  0  1  Staff respect  0.1360  0.1745  0.1570  0.8704  1.3867  0  1  Bath frequency  0.1232  0.1510  0.1493  0.8420  1.2548  1  1  Privacy  0.1125  0.1565  0.1354  0.8574  1.4337  1  1  Choose hair care  0.1107  0.1377  0.1344  0.8398  1.2650  1  1  Bath time  0.1076  0.1333  0.1304  0.8416  1.2624  1  1  Choose nail care frequency  0.1076  0.1333  0.1304  0.8416  1.2624  1  1  Caregiver gender  0.0982  0.1368  0.1180  0.8603  1.4385  1  1  Choose used name  0.0982  0.1368  0.1180  0.8603  1.4385  1  1  Mental professional if sad  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Caregiver know bathroom needs  0.0758  0.0959  0.0924  0.8536  1.3795  1  1  Bedtime routine  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Choose wake time  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Follow morning routine  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Naps  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Room temp  0.0263  0.0428  0.0319  0.8213  1.8613  1  1  TV  0.0162  0.0246  0.0196  0.8343  1.5748  1  1  1-on-1 time  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Alone time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Children  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Friends contact  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Meet new people  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Room lighting  0.0146  0.0295  0.0180  0.8172  2.0758  1  1  Entertainment events  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Exercise  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Games  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Outdoor tasks  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Plant care  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Sports  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Watch movies with others  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Note: act = actual; conf = confidence; pred = predicted. View Large Table 2. Recommender Predicted Resident Preferences—Resident 1 Consequent  score  liftscore  support  conf  lift  act  pred  Food choice  0.2526  0.3058  0.3053  0.8270  1.2094  1  1  Living place care  0.2275  0.2661  0.2714  0.8418  1.2826  1  1  Bed setup  0.2051  0.2537  0.2510  0.8166  1.3395  1  1  Hobbies  0.2020  0.2759  0.2343  0.8528  1.3761  1  1  Learning  0.1795  0.2281  0.2168  0.8449  1.4912  0  1  Staff care  0.1543  0.1913  0.1854  0.8505  1.3336  1  1  Family contact  0.1488  0.1701  0.1666  0.9089  1.2531  1  1  Choose mouth care  0.1478  0.1756  0.1751  0.8484  1.2293  1  1  Helps feel better  0.1443  0.1792  0.1740  0.8476  1.3171  1  1  Choose care professional  0.1415  0.1630  0.1645  0.8710  1.1652  0  1  Staff respect  0.1360  0.1745  0.1570  0.8704  1.3867  0  1  Bath frequency  0.1232  0.1510  0.1493  0.8420  1.2548  1  1  Privacy  0.1125  0.1565  0.1354  0.8574  1.4337  1  1  Choose hair care  0.1107  0.1377  0.1344  0.8398  1.2650  1  1  Bath time  0.1076  0.1333  0.1304  0.8416  1.2624  1  1  Choose nail care frequency  0.1076  0.1333  0.1304  0.8416  1.2624  1  1  Caregiver gender  0.0982  0.1368  0.1180  0.8603  1.4385  1  1  Choose used name  0.0982  0.1368  0.1180  0.8603  1.4385  1  1  Mental professional if sad  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Caregiver know bathroom needs  0.0758  0.0959  0.0924  0.8536  1.3795  1  1  Bedtime routine  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Choose wake time  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Follow morning routine  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Naps  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Room temp  0.0263  0.0428  0.0319  0.8213  1.8613  1  1  TV  0.0162  0.0246  0.0196  0.8343  1.5748  1  1  1-on-1 time  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Alone time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Children  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Friends contact  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Meet new people  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Room lighting  0.0146  0.0295  0.0180  0.8172  2.0758  1  1  Entertainment events  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Exercise  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Games  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Outdoor tasks  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Plant care  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Sports  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Watch movies with others  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Consequent  score  liftscore  support  conf  lift  act  pred  Food choice  0.2526  0.3058  0.3053  0.8270  1.2094  1  1  Living place care  0.2275  0.2661  0.2714  0.8418  1.2826  1  1  Bed setup  0.2051  0.2537  0.2510  0.8166  1.3395  1  1  Hobbies  0.2020  0.2759  0.2343  0.8528  1.3761  1  1  Learning  0.1795  0.2281  0.2168  0.8449  1.4912  0  1  Staff care  0.1543  0.1913  0.1854  0.8505  1.3336  1  1  Family contact  0.1488  0.1701  0.1666  0.9089  1.2531  1  1  Choose mouth care  0.1478  0.1756  0.1751  0.8484  1.2293  1  1  Helps feel better  0.1443  0.1792  0.1740  0.8476  1.3171  1  1  Choose care professional  0.1415  0.1630  0.1645  0.8710  1.1652  0  1  Staff respect  0.1360  0.1745  0.1570  0.8704  1.3867  0  1  Bath frequency  0.1232  0.1510  0.1493  0.8420  1.2548  1  1  Privacy  0.1125  0.1565  0.1354  0.8574  1.4337  1  1  Choose hair care  0.1107  0.1377  0.1344  0.8398  1.2650  1  1  Bath time  0.1076  0.1333  0.1304  0.8416  1.2624  1  1  Choose nail care frequency  0.1076  0.1333  0.1304  0.8416  1.2624  1  1  Caregiver gender  0.0982  0.1368  0.1180  0.8603  1.4385  1  1  Choose used name  0.0982  0.1368  0.1180  0.8603  1.4385  1  1  Mental professional if sad  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Caregiver know bathroom needs  0.0758  0.0959  0.0924  0.8536  1.3795  1  1  Bedtime routine  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Choose wake time  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Follow morning routine  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Naps  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Room temp  0.0263  0.0428  0.0319  0.8213  1.8613  1  1  TV  0.0162  0.0246  0.0196  0.8343  1.5748  1  1  1-on-1 time  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Alone time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Children  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Friends contact  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Meet new people  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Room lighting  0.0146  0.0295  0.0180  0.8172  2.0758  1  1  Entertainment events  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Exercise  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Games  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Outdoor tasks  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Plant care  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Sports  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Watch movies with others  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Note: act = actual; conf = confidence; pred = predicted. View Large Table 3. Recommender Predicted Resident Preferences—Resident 2 Consequent  score  liftscore  support  conf  lift  act  pred  Living place care  0.2280  0.2665  0.2727  0.8388  1.3200  0  1  Hobbies  0.2075  0.2789  0.2447  0.8434  1.3555  1  1  Food choice  0.1991  0.2440  0.2376  0.8409  1.2298  1  1  Learning  0.1969  0.2492  0.2382  0.8397  1.4698  1  1  Staff care  0.1384  0.1740  0.1666  0.8523  1.3451  1  1  Staff respect  0.1381  0.1736  0.1572  0.8842  1.3480  1  1  Helps feel better  0.1380  0.1758  0.1663  0.8528  1.3640  1  1  Choose mouth care  0.1371  0.1636  0.1624  0.8523  1.2325  1  1  Choose care professional  0.1337  0.1543  0.1552  0.8718  1.1662  1  1  Family contact  0.1202  0.1363  0.1360  0.9030  1.2558  1  1  Bed setup  0.1175  0.1454  0.1443  0.8294  1.3535  1  1  Choose nail care frequency  0.1076  0.1333  0.1304  0.8416  1.2624  0  1  Bath frequency  0.1061  0.1313  0.1286  0.8416  1.2609  0  1  Bath time  0.1058  0.1311  0.1282  0.8411  1.2705  1  1  Choose hair care  0.1049  0.1302  0.1273  0.8410  1.2650  0  1  Privacy  0.1049  0.1461  0.1261  0.8585  1.4354  1  1  Room temp  0.0994  0.1406  0.1206  0.8202  1.5735  1  1  Caregiver know bathroom needs  0.0993  0.1250  0.1201  0.8537  1.4079  1  1  Caregiver gender  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Choose used name  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Mental professional if sad  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Bedtime routine  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Choose wake time  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Follow morning routine  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Naps  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Children  0.0172  0.0299  0.0198  0.8750  1.7623  1  1  Room lighting  0.0162  0.0324  0.0196  0.8265  2.0936  1  1  1-on-1 time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Alone time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Meet new people  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Friends contact  0.0145  0.0292  0.0165  0.8816  2.0072  1  1  TV  0.0132  0.0199  0.0155  0.8499  1.5135  1  1  Exercise  0.0117  0.0248  0.0135  0.8661  2.1246  1  1  Outdoor tasks  0.0117  0.0268  0.0135  0.8661  2.2818  1  1  Entertainment events  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Games  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Plant care  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Sports  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Watch movies with others  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Reminisce  0.0102  0.0175  0.0123  0.8333  1.7172  1  1  Consequent  score  liftscore  support  conf  lift  act  pred  Living place care  0.2280  0.2665  0.2727  0.8388  1.3200  0  1  Hobbies  0.2075  0.2789  0.2447  0.8434  1.3555  1  1  Food choice  0.1991  0.2440  0.2376  0.8409  1.2298  1  1  Learning  0.1969  0.2492  0.2382  0.8397  1.4698  1  1  Staff care  0.1384  0.1740  0.1666  0.8523  1.3451  1  1  Staff respect  0.1381  0.1736  0.1572  0.8842  1.3480  1  1  Helps feel better  0.1380  0.1758  0.1663  0.8528  1.3640  1  1  Choose mouth care  0.1371  0.1636  0.1624  0.8523  1.2325  1  1  Choose care professional  0.1337  0.1543  0.1552  0.8718  1.1662  1  1  Family contact  0.1202  0.1363  0.1360  0.9030  1.2558  1  1  Bed setup  0.1175  0.1454  0.1443  0.8294  1.3535  1  1  Choose nail care frequency  0.1076  0.1333  0.1304  0.8416  1.2624  0  1  Bath frequency  0.1061  0.1313  0.1286  0.8416  1.2609  0  1  Bath time  0.1058  0.1311  0.1282  0.8411  1.2705  1  1  Choose hair care  0.1049  0.1302  0.1273  0.8410  1.2650  0  1  Privacy  0.1049  0.1461  0.1261  0.8585  1.4354  1  1  Room temp  0.0994  0.1406  0.1206  0.8202  1.5735  1  1  Caregiver know bathroom needs  0.0993  0.1250  0.1201  0.8537  1.4079  1  1  Caregiver gender  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Choose used name  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Mental professional if sad  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Bedtime routine  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Choose wake time  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Follow morning routine  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Naps  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Children  0.0172  0.0299  0.0198  0.8750  1.7623  1  1  Room lighting  0.0162  0.0324  0.0196  0.8265  2.0936  1  1  1-on-1 time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Alone time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Meet new people  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Friends contact  0.0145  0.0292  0.0165  0.8816  2.0072  1  1  TV  0.0132  0.0199  0.0155  0.8499  1.5135  1  1  Exercise  0.0117  0.0248  0.0135  0.8661  2.1246  1  1  Outdoor tasks  0.0117  0.0268  0.0135  0.8661  2.2818  1  1  Entertainment events  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Games  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Plant care  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Sports  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Watch movies with others  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Reminisce  0.0102  0.0175  0.0123  0.8333  1.7172  1  1  Note: act = actual; conf = confidence; pred = predicted. View Large Table 3. Recommender Predicted Resident Preferences—Resident 2 Consequent  score  liftscore  support  conf  lift  act  pred  Living place care  0.2280  0.2665  0.2727  0.8388  1.3200  0  1  Hobbies  0.2075  0.2789  0.2447  0.8434  1.3555  1  1  Food choice  0.1991  0.2440  0.2376  0.8409  1.2298  1  1  Learning  0.1969  0.2492  0.2382  0.8397  1.4698  1  1  Staff care  0.1384  0.1740  0.1666  0.8523  1.3451  1  1  Staff respect  0.1381  0.1736  0.1572  0.8842  1.3480  1  1  Helps feel better  0.1380  0.1758  0.1663  0.8528  1.3640  1  1  Choose mouth care  0.1371  0.1636  0.1624  0.8523  1.2325  1  1  Choose care professional  0.1337  0.1543  0.1552  0.8718  1.1662  1  1  Family contact  0.1202  0.1363  0.1360  0.9030  1.2558  1  1  Bed setup  0.1175  0.1454  0.1443  0.8294  1.3535  1  1  Choose nail care frequency  0.1076  0.1333  0.1304  0.8416  1.2624  0  1  Bath frequency  0.1061  0.1313  0.1286  0.8416  1.2609  0  1  Bath time  0.1058  0.1311  0.1282  0.8411  1.2705  1  1  Choose hair care  0.1049  0.1302  0.1273  0.8410  1.2650  0  1  Privacy  0.1049  0.1461  0.1261  0.8585  1.4354  1  1  Room temp  0.0994  0.1406  0.1206  0.8202  1.5735  1  1  Caregiver know bathroom needs  0.0993  0.1250  0.1201  0.8537  1.4079  1  1  Caregiver gender  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Choose used name  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Mental professional if sad  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Bedtime routine  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Choose wake time  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Follow morning routine  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Naps  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Children  0.0172  0.0299  0.0198  0.8750  1.7623  1  1  Room lighting  0.0162  0.0324  0.0196  0.8265  2.0936  1  1  1-on-1 time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Alone time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Meet new people  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Friends contact  0.0145  0.0292  0.0165  0.8816  2.0072  1  1  TV  0.0132  0.0199  0.0155  0.8499  1.5135  1  1  Exercise  0.0117  0.0248  0.0135  0.8661  2.1246  1  1  Outdoor tasks  0.0117  0.0268  0.0135  0.8661  2.2818  1  1  Entertainment events  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Games  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Plant care  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Sports  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Watch movies with others  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Reminisce  0.0102  0.0175  0.0123  0.8333  1.7172  1  1  Consequent  score  liftscore  support  conf  lift  act  pred  Living place care  0.2280  0.2665  0.2727  0.8388  1.3200  0  1  Hobbies  0.2075  0.2789  0.2447  0.8434  1.3555  1  1  Food choice  0.1991  0.2440  0.2376  0.8409  1.2298  1  1  Learning  0.1969  0.2492  0.2382  0.8397  1.4698  1  1  Staff care  0.1384  0.1740  0.1666  0.8523  1.3451  1  1  Staff respect  0.1381  0.1736  0.1572  0.8842  1.3480  1  1  Helps feel better  0.1380  0.1758  0.1663  0.8528  1.3640  1  1  Choose mouth care  0.1371  0.1636  0.1624  0.8523  1.2325  1  1  Choose care professional  0.1337  0.1543  0.1552  0.8718  1.1662  1  1  Family contact  0.1202  0.1363  0.1360  0.9030  1.2558  1  1  Bed setup  0.1175  0.1454  0.1443  0.8294  1.3535  1  1  Choose nail care frequency  0.1076  0.1333  0.1304  0.8416  1.2624  0  1  Bath frequency  0.1061  0.1313  0.1286  0.8416  1.2609  0  1  Bath time  0.1058  0.1311  0.1282  0.8411  1.2705  1  1  Choose hair care  0.1049  0.1302  0.1273  0.8410  1.2650  0  1  Privacy  0.1049  0.1461  0.1261  0.8585  1.4354  1  1  Room temp  0.0994  0.1406  0.1206  0.8202  1.5735  1  1  Caregiver know bathroom needs  0.0993  0.1250  0.1201  0.8537  1.4079  1  1  Caregiver gender  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Choose used name  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Mental professional if sad  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Bedtime routine  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Choose wake time  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Follow morning routine  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Naps  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Children  0.0172  0.0299  0.0198  0.8750  1.7623  1  1  Room lighting  0.0162  0.0324  0.0196  0.8265  2.0936  1  1  1-on-1 time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Alone time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Meet new people  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Friends contact  0.0145  0.0292  0.0165  0.8816  2.0072  1  1  TV  0.0132  0.0199  0.0155  0.8499  1.5135  1  1  Exercise  0.0117  0.0248  0.0135  0.8661  2.1246  1  1  Outdoor tasks  0.0117  0.0268  0.0135  0.8661  2.2818  1  1  Entertainment events  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Games  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Plant care  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Sports  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Watch movies with others  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Reminisce  0.0102  0.0175  0.0123  0.8333  1.7172  1  1  Note: act = actual; conf = confidence; pred = predicted. View Large The recommendations for Resident 2 in Table 3 demonstrate how different responses to the MDS items produce a different set of recommended preferences to ask. This is due in large part to the fact that the MDS preferences for Resident 2 are indeed different from the preferences of Resident 1. In this case, the machine learner has recommended 40 of the 56 questions be considered. The very first recommended preference, Living place care (i.e., “How important is it to you to take care of the place you live?”), was not endorsed as important by the resident’s actual data. However, the subsequent 10 recommendations were correctly predicted. Obviously, a recommendation of asking an additional 39 or 40 questions would still be time-consuming and burdensome for a resident respondent. Therefore, the decreasing score metric can be used to identify a threshold or cut-off by which to configure the recommender for suggesting which questions to pursue with a resident. A defined threshold would allow us to configure the recommender to tailor the preference interview further. For instance, if a threshold of 0.13 is used, then for Resident 1, the first 12 questions shown in the list would be pursued, while for that same threshold, the first 10 questions would be pursued for Resident 2. The precise selection of the threshold is currently part of our future investigations and will rely on a number of factors including analysis of the correctness of the predictions (i.e., did the resident agree with the recommendation). We used a fivefold cross-validation to evaluate the quality of the recommender and to identify the best configuration of the combined apriori/logistic regression approach described above. In the fivefold cross-validation, we grouped the data into five partitions, using four partitions, or 80% of the available data to train the learner and the remaining partition, or 20% as test data. This was repeated five times, rotating each partition as a test set against the remaining training partitions. In our evaluation of the approach, we examined a number of variations of the parameterization of the algorithms and models and received roughly the same performance across those variations. We were able to obtain recall = .8021, precision = .7919, accuracy = .6979, and F1-score = .7953. Recall measures the ratio of correctly predicted rules compared to all of the predicted rules (either positive or negative). Similarly, precision measures the ratio of correctly predicted rules with respect to the rules predicted to be true. In both instances, a correctly predicted rule corresponds to an association rule in which the consequent is predicted to be important (i.e., very important or somewhat important) that is indeed important to the resident. The overall accuracy of the approach (i.e., ratio of correctly predicted preferences and nonpreferences) is approximately 70%. The F1-score provides a harmonic average of both precision and recall, and in this case is nearly 0.80. While precision and recall are high, the F1-score of nearly 0.80 emphasizes a balance in the ability of the learner to predict actual preferences. To put these results in perspective, we note that the quality of a learner and recommender is always dependent on the domain in which it is being applied. From this standpoint, the results here indicate that the learner provides recommendations that are relevant to a resident about four times in five. Contrast this with a recommender that uses a coin-flip to make the binary choice of preference or nonpreference, which would result in identifying a preference only 50% of the time. When translated into a savings in time to both the interviewer and resident, the potential is noteworthy. Discussion The PELI-NH is increasingly being used as a guide for providing preference-based person-centered care in NH communities. The amount of time needed to complete the 72 item PELI-NH is a considerable barrier. Our machine learning mechanism has the potential to reduce the time needed for completing the PELI-NH interview while still incorporating important resident preferences. Our initial work has shown that we can achieve a reasonable rate of accuracy in providing recommendations on potential preferences for a resident with a high rate of precision, a fruitful potential for providers. We are unaware of other recommendation systems that have been created for addressing preferences in the context of person-centered care. Clearly, however, the idea of developing recommendation systems is not a new one (Aggarwal, 2016). The work reported on here uses an implementation that has been applied in many contexts using two well-known methodologies (Agrawal & Srikant, 1994; Nelder & Wedderburn, 1972). Furthermore, the results we have attained are consistent with the evaluation metrics used in recommendation tasks (Gunawardana & Shani, 2009), and could be applied in all long-term services and support settings (e.g., NH, assisted living, and home- and community-based settings). The novelty lies in assisting provider communities with a methodology for conducting PELI-NH interviews that reduces respondent fatigue and staff time to conduct interviews while also alleviating the burden of additional paperwork. Through using a recommender, questions are tailored for each resident. This approach could be seen as a parallel to precision health, where treatments are tailored to the unique genetic, environmental, and lifestyle habits of individual patients. Understanding the unique preference inventory of each NH resident can aid in bringing preferences into practice to improve the quality of care and quality of life to best meet the psychosocial needs of each person. In addition, the recommendations can be used as a suggested next best preference to ask. A provider community could decide to focus on the top-3, -5, or -10 recommendations for each resident. Alternatively recommendations can be based upon items with a score or liftscore that exceeds a particular value. In our example, we used 0.13 as a threshold for illustration but ultimately, the chosen threshold will be based on the experiences of the staff that are administering the PELI interviews. Regardless of the threshold, the approach allows provider communities to select preference questions according to the ranking provided by the recommender. In addition, the preference recommender can overcome barriers such as the amount of time needed for documenting important preferences through integration into existing electronic medical records. It would also remediate the lack of support within the staff workflow, which has also been found to be a major barrier to the adoption of new technology (Schulz et al., 2015). In developing the recommender, we have focused on positive results from precision and recall for crafting a recommendation set, as is consistent with the work of Lin (2000). As such, we chose not to optimize specificity, which is an indicator of whether a preference is unimportant. Overall, we expect that as we continue to develop the algorithm, we will be able to continue to improve the rule-based recommender. In addition, we plan on exploring other potential methods for suggesting preferences. We envision this recommender system is a novel method for providing guidance to provider communities on how to best tailor preference interviews to residents. Next Steps The recommender that we have built (and the results shown here) are based on a dataset that is indicative of a specific set of communities. The recommendations for other communities and for specific residents evolve over time based on the preferences of the residents in the community. For instance, the preferences of communities in the Pacific Northwest may be different from the preferences of communities in the Southern United States. As such, it is stressed that the recommended questions should always be validated against a resident’s actual preferences. Finally, the score and liftscore thresholds used to recommend questions to be pursued need to be selected based on the experiences of a specific community and person, again taking account the issues of fatigue and paperwork burden. Next steps include building in a component to the recommender that seeks provider input in determining the thresholds. For example, the provider could indicate that they only have the capacity to ask 3, 5, or 10 additional questions. As we work to refine the recommender, our goal is to use it as an active learning system, with the hope of gaining the benefits of collaborative filtering. In other words, the quality of the recommendations has the potential to improve with the increase in the number of resident interviews that are conducted. Each additional transaction allows us to refine the rule sets through the benefits of the wisdom of the crowd. In addition, we are interested in applying other recommendation strategies, including item-based collaborative filtering (Sarwar, Karypis, Konstan, & Reidl, 2001) which identifies similarity based on the ratings of products rather than the transactional co-occurrence approach described here. Limitations We evaluated the learning approach using a relatively small transaction dataset (n = 510) with an assumption that responses of individuals were independent when in fact the data contained PELI-NH responses of n = 255 NH residents asked at two points in time 3 months apart. This assumption is akin to assuming that shopping transactions are independent across separate shopping trips to a market. We also assumed that the responses of very important and somewhat important indicate “preference,” while other responses indicated “nonpreference.” This decision was based upon the findings of Van Haitsma and colleagues (2014) and emphasizes the fact that our approach is only concerned with recommending PELI-NH items to explore rather than trying to precisely predict a preference rating of very important, somewhat important, not very important, or not important at all for a resident. With respect to the data, despite a rigorous approach to sampling among participants and settings, all respondents had the cognitive capability to answer questions about their preferences. Therefore, preferences of individuals with moderate or severe cognitive impairment, or those who were unable to communicate are not represented in this study. In addition, the sample was drawn from large metropolitan areas on the East Coast of the United States and may not reflect the preferences of NH residents living in other areas of the country. Future work in this area will be improved as more data are added. For example, including the voices of people living with cognitive impairment and individuals living in different areas of the country will improve accuracy of the recommender. Conclusion At a time when NH providers are asked to improve the quality and satisfaction with care at a lower cost we must embrace technology to help improve the efficiency and effectiveness of care. The findings from this article highlight the applicability of machine learning and specifically recommender systems to the NH setting. We found that the recommender system was extremely accurate at predicting important preferences of NH residents. Because the recommender uses data from other residents to predict, the algorithm will become more accurate as larger datasets become available. Ultimately, the use of machine learning could assist NH providers in tailoring their preference assessments to maximize staff time while minimizing the burden of asking a resident dozens of questions. This work provides direction for future research and is critical in advancing the delivery of preference-based, person-centered care. Supplementary Material Supplementary data are available at The Gerontologist online. Funding This work was supported by generous funding from the National Institute of Nursing Research grant (R21NR011334 to K. Van Haitsma [PI]), the Patrick and Catherine Weldon Donaghue Medical Research Foundation, and the Ohio Department of Medicaid. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Nursing Research, the National Institutes of Health, the Donaghue Foundation, or the Ohio Department of Medicaid. Conflict of Interest None reported. Acknowledgments We would like to thank Karen Eshraghi and Christina Duntzee, the research team members who worked diligently to collect this data, and the older adults who participated in the project. References Abbott, K. M., Klumpp, R., Leser, K., Straker, J., Gannod, G., & Van Haitsma, K. ( 2018). Delivering person-centered care: Important preferences for recipients of long-term services and supports. Journal of the American Medical Directors Association , 19, 169– 173. doi: 10.1016/j.jamda.2017.10.005 Google Scholar CrossRef Search ADS PubMed  Aggarwal, C. C. ( 2016). Recommender systems: The textbook  ( 1st ed.). Springer International Publishing. Retrieved from https://doi.org/10.1007/978-3-319-29659-3 Google Scholar CrossRef Search ADS   Agrawal, R., & Srikant, R. ( 1994). Fast algorithms for mining association rules in large databases. In Proceedings of the 20th International Conference on Very Large Data Bases (pp. 487– 499). San Francisco: Morgan Kaufmann. Retrieved from http://dl.acm.org/citation.cfm?id=645920.672836 Ashcraft, A., Cherry, B., & Owen, D. ( 2007) Perceptions of job satisfaction and the regulatory environment among nurse aides and charge nurses in long-term care. Geriatric Nursing , 28, 183– 192. doi: 10.1016/j.gerinurse.2007.01.015 Google Scholar CrossRef Search ADS PubMed  Bangerter, L. R., Abbott, K., Heid, A., Eshraghi, K., & Van Haitsma, K. ( 2017). Using spontaneous commentary of nursing home residents to develop resident-centered measurement tools: A case study. Geriatric Nursing (New York, N.Y.) , 38, 548– 550. doi: 10.1016/j.gerinurse.2017.04.003 Google Scholar CrossRef Search ADS PubMed  Bendakir, N., & Aimeur, E. ( 2006). Using association rules for course recommendation. In Proceedings of the Association for the Advancement of Artificial Intelligence Workshop on Educational Data Mining (pp. 31– 40). Retrieved from https://www.aaai.org/Papers/Workshops/2006/WS-06-05/WS06-05-005.pdf Cakir, O., & Aras, M. E. ( 2012). A recommendation engine by using association rules. Procedia—Social and Behavioral Sciences , 62, 452– 456. doi: 10.1016/j.sbspro.2012.09.074 Google Scholar CrossRef Search ADS   Carpenter, B. D., Van Haitsma, K., Ruckdeschel, K., & Lawton, M. P. ( 2000). The psychosocial preferences of older adults: A pilot examination of content and structure. The Gerontologist , 40, 335– 348. doi: 10.1093/geront/40.3.335 Google Scholar CrossRef Search ADS PubMed  Centers for Medicare and Medicaid Services.( 2015). Impact act of 2014 and cross setting measures. Retrieved from https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Post-Acute-Care-Quality-Initiatives/IMPACT-Act-of-2014-and-Cross-Setting-Measures.html Clark, S., Elswick, S., Gabriel, M., Gurupur, V., & Wisniewski, P. ( 2016). Transitions of care: A patient-centered perspective of health information systems that support post-acute care. Journal of Integrated Design and Process Science , 20, 95– 110. doi: 10.3233/jid-2016-0008 Google Scholar CrossRef Search ADS   Curyto, K., Van Haitsma, K. S., & Towsley, G. L. ( 2016). Cognitive interviewing: Revising the preferences for everyday living inventory for use in the nursing home. Research in Gerontological Nursing , 9, 24– 34. doi: 10.3928/19404921- 20150522-04 Google Scholar CrossRef Search ADS PubMed  Edelen, M., Gage, B. J., Rose, A. J., Ahluwalia, S., Soo Jin DeSantis, A., Dunbar, M. S.,… Stucky, B. D. ( 2017). Development and maintenance of standardized cross setting patient assessment data for post-acute care: Summary report of findings from alpha 1 pilot testing . Santa Monica, CA: RAND Corporation. Retrieved from https://www.rand.org/pubs/research_reports/RR1895.html Google Scholar CrossRef Search ADS   Folstein, M. F., Folstein, S. E., & McHugh, P. R. ( 1975). “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research , 12, 189– 198.  doi: 10.1016/0022-3956(75) 90026-6 Google Scholar CrossRef Search ADS PubMed  Gunawardana, A., & Shani, G. ( 2009). A survey of accuracy evaluation metrics of recommendation tasks. The Journal of Machine Learning Research , 10, 2935– 2962. doi: 10.1145/ 1577069.1755883 Heid, A. R., Bangerter, M. A., Abbott, K. M., & Van Haitsma, K. ( 2017). Do family proxies get it right? Concordance in reports of nursing home residents’ everyday preferences. Journal of Applied Gerontology , 36, 667– 691. doi: 10.1177/ 0733464815581485 Google Scholar CrossRef Search ADS PubMed  Hess, S., Hensher, D., & Daly, A. ( 2012). Not bored yet—revisiting respondent fatigue in stated choice experiments. Transportation Research Part A: Policy and Practice , 46, 626– 644. doi: 10.1016/j.tra.2011.11.008 Google Scholar CrossRef Search ADS   Hu, R. ( 2010). Medical data mining based on association rules. Computer and Information Science , 3, 104– 108. doi: 10.5539/cis.v3n4p104 Google Scholar CrossRef Search ADS   Koh, H. C., & Tan, G. ( 2005). Data mining applications in healthcare. Journal of Healthcare Information Management , 19, 64– 72. doi: 10.4314/ijonas.v5i1.49926 Google Scholar PubMed  Lin, W. ( 2000). Association rule mining for collaborative recommender systems . Worchester Polytechnic Institute. Retrieved from https://web.wpi.edu/Pubs/ETD/Available/etd-0515100-145926/ unrestricted/wlin.pdf Nelder, J. A., & Wedderburn, R. W. M. ( 1972). Generalized linear models. Journal of the Royal Statistical Society , 135, 370– 384. Retrieved from https://docs.ufpr.br/~taconeli/CE225/Artigo.pdf Google Scholar CrossRef Search ADS   Patel, V. L., Shortliffe, E. H., Stefanelli, M., Szolovits, P., Michael, R., Bellazzi, R.,… Martin, C. K. ( 2009). The coming of age of artificial intelligence in medicine. Artificial Intelligence , 46, 152– 159. doi: 10.1016/j.artmed.2008.07.017 Saliba, D., & Buchanan, J. ( 2008). Development and validation of a revised nursing home assessment tool: MDS 3.0 . Santa Monica, CA: Rand Corporation Health. Sarwar, B., Karypis, G., Konstan, J., & Reidl, J. ( 2001). Item-based collaborative filtering recommendation algorithms. In Proceedings of the Tenth International Conference on World Wide Web—WWW ‘01 (pp. 285– 295). doi: 10.1145/371920.372071 Schulz, R., Wahl, H. W., Matthews, J. T., De Vito Dabbs, A., Beach, S. R., & Czaja, S. J. ( 2015). Advancing the aging and technology agenda in gerontology. The Gerontologist , 55, 724– 734. doi: 10.1093/geront/gnu071 Google Scholar CrossRef Search ADS PubMed  Simovici, D. A. ( 2012). Data mining of medical data: Opportunities and challenges in mining association rules . Potsdam: IALS. Retrieved from http://www.cs.umb.edu/~dsim/papersps/dmmd.pdf Trossman, S. ( 2002). The documentation dilemma. Nurses poised to address paperwork burden. Tar Heel Nurse , 64, 10– 11. Google Scholar PubMed  Van Haitsma, K., Abbott, K. M., Heid, A. R., Carpenter, B., Curyto, K., Kleban, M.,… Spector, A. ( 2014). The consistency of self-reported preferences for everyday living: Implications for person-centered care delivery. Journal of Gerontological Nursing , 40, 34– 46. doi: 10.3928/00989134-20140820-01 Google Scholar CrossRef Search ADS PubMed  Van Haitsma, K., Curyto, K., Spector, A., Towsley, G., Kleban, M., Carpenter, B.,… Koren, M. J. ( 2013). The preferences for everyday living inventory: Scale development and description of psychosocial preferences responses in community-dwelling elders. The Gerontologist , 53, 582– 595. doi: 10.1093/geront/gns102 Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press on behalf of The Gerontological Society of America. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png The Gerontologist Oxford University Press

A Machine Learning Recommender System to Tailor Preference Assessments to Enhance Person-Centered Care Among Nursing Home Residents

Loading next page...
 
/lp/ou_press/a-machine-learning-recommender-system-to-tailor-preference-assessments-FQbXloaGK6
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press on behalf of The Gerontological Society of America. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.
ISSN
0016-9013
eISSN
1758-5341
D.O.I.
10.1093/geront/gny056
Publisher site
See Article on Publisher Site

Abstract

Abstract Background and Objectives Nursing homes (NHs) using the Preferences for Everyday Living Inventory (PELI-NH) to assess important preferences and provide person-centered care find the number of items (72) to be a barrier to using the assessment. Research Design and Methods Using a sample of n = 255 NH resident responses to the PELI-NH, we used the 16 preference items from the MDS 3.0 Section F to develop a machine learning recommender system to identify additional PELI-NH items that may be important to specific residents. Much like the Netflix recommender system, our system is based on the concept of collaborative filtering whereby insights and predictions (e.g., filters) are created using the interests and preferences of many users. The algorithm identifies multiple sets of “you might also like” patterns called association rules, based upon responses to the 16 MDS preferences that recommends an additional set of preferences with a high likelihood of being important to a specific resident. Results In the evaluation of the combined apriori and logistic regression approach, we obtained a high recall performance (i.e., the ratio of correctly predicted preferences compared with all predicted preferences and nonpreferences) and high precision (i.e., the ratio of correctly predicted rules with respect to the rules predicted to be true) of 80.2% and 79.2%, respectively. Discussion and Implications The recommender system successfully provides guidance on how to best tailor the preference items asked of residents and can support preference capture in busy clinical environments, contributing to the feasibility of delivering person-centered care. CMS datasets (OSCAR MDS), Long-term care, Nursing homes, Quality of Care, Technology Current trends in long-term care, in concert with the Centers for Medicare and Medicaid Services (CMS) Quality of Life guidelines, call for a shift in focus to person-centered care. Evolving conceptualizations of optimal care in the NH further emphasize the need to “know the person” to deliver individualized, holistic care. In the United States, preference assessment has been incorporated into the Minimum Data Set (MDS 3.0) through Section F Preferences for Customary Routine and Activities upon admission and annually thereafter for NH residents. Section F includes eight preferences for personal care and eight items for activity preferences (Saliba & Buchanan, 2008). These 16 preference questions were informed by the PELI-NH (Van Haitsma et al., 2013), which includes 72 preference questions across five domains. The state of Ohio has recently mandated that the PELI-NH be used for pay for performance initiatives in NH settings (http://codes.ohio.gov/oac/5160-3-58). One major barrier reported by 61% of Ohio providers to meeting this mandate included the time required for staff to complete the interview (unpublished data). Although there is a desire by NH providers to use the PELI-NH to personalize care, they seek ways to more efficiently identify important preferences. Knowledge of a resident’s everyday care preferences provides the foundation for ongoing individualized care planning. However, obtaining idiographic information requires asking an individual a multitude of questions, which creates an enhanced burden of paperwork and assessments in addition to respondent fatigue. For example, nursing staff perceive that 40%–90% of their workday is spent on paperwork and documentation (Ashcraft, Cherry, & Owen, 2007). This time spent completing paperwork not only impedes a nursing staff’s ability to provide direct care (Ashcraft et al., 2007) but has also been found to be a source of job dissatisfaction for nurses across various care settings (Trossman, 2002). Therefore, there has been a movement toward standardization to address the burdens of paperwork and to streamline data collection so it can be used across transitions in care. Concern regarding the continuity of an individual’s care arises during the transition across different systems of care as there is often a lack of care coordination between long-term and postacute care providers (LTPAC; Clark, Elswick, Gabriel, Gurupur, & Wisniewski, 2016). As a result the Improving Medicare Post-Acute Care Transformation Act of 2014 (IMPACT Act) was enacted (Clark et al., 2016) requiring that all LTPAC providers, which includes long-term care hospitals, skilled nursing facilities, home health agencies, and inpatient rehabilitation facilities, standardize assessment data and reporting to allow for seamless sharing and interoperability between providers (CMS, 2015). The standardization of paperwork and documentation not only helps to achieve person-centered outcomes for the individual transitioning across care settings but also improves assessment and care coordination as providers are able to be more efficient in sharing relevant information (Edelen et al., 2017). To assist in the reduction of fatigue (Hess et al., 2012) while also providing personalization of resident experience we have explored the application of “recommender systems” to preference assessment in the NH setting. Recommender systems (Aggarwal, 2016) have gained in popularity as a practical application of machine learning. Consumer websites have long seen demonstration of this technology in areas such as e-commerce (Amazon), movies (Netflix), and music (Pandora) in the form of ratings systems that are used to help users identify potential likes and dislikes. Different strategies for implementing recommender systems have been suggested, including the use of association rules (Bendakir & Aimeur, 2006; Cakir & Aras, 2012; Lin, 2000). In the health care domain, the idea of using data mining is not a new one (Patel et al., 2009; Simovici, 2012). Applications have included treatment effectiveness and condition identification (Koh & Tan, 2005). Hu, for instance, has suggested using the Apriori algorithm for mining medical data as a means for diagnosis of conditions (Hu, 2010). In the area of long-term care, we are unaware of work being done to understand resident preferences using recommendations. Therefore, we developed an approach that personalizes preferences tailored to each individual resident respondent through combining the Apriori algorithm (Agrawal & Srikant, 1994) and logistic regression configured with a generalized linear regression model (Nelder & Wedderburn, 1972). The idea behind the preference recommender is to develop a software algorithm that suggests additional PELI-NH preference items to ask using responses from a residents’ 16 MDS items. The preference recommender can provide guidance or decision support to provider communities as they explore the next best set of preference questions to ask each NH resident. Our initial investigations have focused on using a rule-based collaborative filtering approach, with responses to preference questions used in a manner analogous to selection of products in a market basket. In collaborative filtering, insights and predictions (e.g., filters) are created using the interests and preferences of many users (i.e., by collaborating). Therefore, this study seeks to apply data mining and machine learning techniques to explore their utility in providing insight into a reduced set of more targeted and meaningful potential preferences specific to each individual. The ultimate goal is to tailor preference assessments based upon the knowledge gained from resident responses to the 16 MDS 3.0 Section F preference items thereby reducing the need to ask an additional 56 PELI-NH items. Design and Methods Procedures and Participants The PELI-NH for NH residents was administered via face-to-face interviews by research assistants with n = 255 NH residents with an Mini-Mental State Examination (MMSE) ≥ 13 (MMSE; Folstein, Folstein, & McHugh, 1975) at baseline (T1) and 3 months later (T2). The 72 items in the PELI-NH include the 16 MDS 3.0 Section F Preferences for Customary Routine and Activities. NH residents were recruited from 28 locations in the suburbs of a major metropolitan East Coast area of the United States. The facility contact person from each NH identified residents who would enjoy participating in an interview about their likes and dislikes, who were up to moderately cognitively capable (MMSE ≥ 13), English speaking, and had a length of stay of at least 1 week and an expected stay of 3 months. The final sample consisted of 255 NH residents completing both the T1 and T2 interviews of 342 who completed T1. Our attrition rate from T1 to T2 was 25.4% which was due to death, transfer, change in cognitive ability, withdrawal, or change in medical stability over the 3 months. Additional recruitment details can be found here (Abbott et al., 2018; Bangerter, Abbott, Heid, Eshraghi, & Van Haitsma, 2017). Informed consent for participation in the study was established in-person and was repeated before the follow-up interview 3 months later. The PELI (Van Haitsma et al., 2013, 2014) is a comprehensive, reliable assessment instrument that examines the content, meaning, and importance of 72 psychosocial preferences for social contact, growth activities, leisure activities, self-dominion, and enlisting others in care (Carpenter, Van Haitsma, Ruckdeschel, & Lawton, 2000). The PELI asks respondents to rate these items using the stem “How important is it to you to. . . [Insert preference]” with response options on a 4-point Likert scale from 1 (very important) to 4 (not important at all). The content and structure were developed via concept mapping with n = 550 older adults receiving home health services (Van Haitsma et al., 2013). The second iteration of the PELI was its modification for use in a NH population (PELI-NH) based on results from cognitive interviewing techniques with n = 70 residents (Curyto, Van Haitsma, & Towsley, 2016). Cognitive interviews resulted in the 72-item PELI-NH, which assesses NH resident preferences grouped into the five originally derived concept mapping domains (Carpenter et al., 2000). Additional studies have reported that the PELI-NH has high levels of face validity as well as convergent and discriminant validity (Van Haitsma, et al., 2013). The NH resident test–retest reliability over 1 week is 87.3% (Van Haitsma et al., 2014) and 73.35% with proxy respondents (Heid, Bangerter, Abbott, & Van Haitsma, 2017). For the analysis in this current study, response options were collapsed into important (very and somewhat) or not important (not very and not important at all). Developing the Recommender for Daily Preferences We have developed an approach that personalizes preferences tailored to each individual resident through combining the Apriori algorithm (Agrawal & Srikant, 1994) and a logistic regression configured with a generalized linear regression model (Nelder & Wedderburn, 1972). The Apriori algorithm is used to identify frequent itemsets of preferences in a dataset, while logistic regression is used to assess the quality of the mined rules. In our context, itemsets describe the preferences that most often co-occur in a dataset. Specifically, two preferences A and B co-occur when a resident indicates “very important” or “somewhat important” for each preference A and B. Itemsets that include multiple preferences {A, B, C} can be used to build rulesets that infer association rules such as {A, B} → {C}, where knowledge of the co-occurrence (or co-preference) of the set {A, B} indicates a statistically likely occurrence or preference for an item C. The Apriori approach is often used to identify rulesets for market baskets to make recommendations for product upsells. For example, when a shopper chooses a set of products {A, B}, a recommendation is made for a shopper to consider product {C} based on the behavior of shoppers that also chose the set of products A and B. For instance, a grocery store recommendation system might suggest that if a market basket contains eggs and peanut butter then milk should be added to the basket. Using the shopping analogy, we consider preferences of residents as analogous to the selection of products for a market basket by an individual customer. As such, we configure the Apriori algorithm to consider each of the 72 preferences of the PELI-NH as if it were a product to be purchased by a resident. Expressing a positive preference (e.g., very important or somewhat important) is held as analogous to purchasing of a product. Conversely, expressing a negative preference (e.g., not very important or not important at all) is held as analogous to not purchasing a product. Another key aspect of the Apriori algorithm is the use of transactions to build the frequent itemsets and association rules, the collection of which constitutes a trained learner. Trained learners are machine learning algorithms that are established by using patterns present in a dataset. They are said to be trained since the patterns must be discovered over time through analysis of data rather than by creation of a strict algorithm. In the case of our approach, the recommender is trained with transactions that provide the basis of evidence for how often various items (or in our case, preferences) co-occur. A transaction, in this context, corresponds to a single visit by a shopper to a store to purchase products collected in a market basket. Each visit constitutes a separate transaction, with separate visits by a single shopper being considered different transactions. In the case of our learner, the n = 255 different residents, each with two sets of responses to the PELI-NH, results in n = 510 transactions available for training the learner. Individual transactions are considered independently of previous responses for a given resident and are merely used in establishing the patterns for the learner. While a larger dataset will potentially improve the quality of the learner over time, the Apriori algorithm identifies frequent itemsets that are present in the data as provided. As such, patterns are identified regardless of the size of the inputs. Recommendations using Apriori is based on the idea that when a shopper starts to fill a basket with products, the learner can make suggestions according to the items that frequently co-occur. When applied to the full collection of PELI-NH items, our approach is to interview a resident to explicitly determine responses to the 16 MDS 3.0 Section F questions. Using that as a market basket, the approach uses the trained learner to indicate which preferences from the remaining 56 PELI-NH questions co-occur with those MDS preferences that have been identified by the resident. In all cases, the 56 PELI-NH questions are only drawn from the training set for training purposes. Subsequent predictions only require a resident to respond to the 16 MDS 3.0 questions. Using the shopping analogy, this would correspond to identifying which products commonly appear in market baskets that look similar to the one assembled by the shopper. Supplementary Appendix B lists each question from the MDS Section F along with the percentages of our sample who endorsed the preference as very or somewhat important. Analyses To generate a list of recommended questions to ask an individual resident, the recommendation system is first given as input the set of 16 MDS preference responses. The recommendation engine compares these preference inputs with the antecedents (left-hand side) of all of its rules, searching for matches. For all matches that are found, the consequents (right-hand sides) of the rules are aggregated and converged into a single list. Each aggregated consequent is given a set of summary metrics of all of the association rules that predicted it. These consequents and their summary metrics are then passed through a generalized logistic regression model, which classifies whether each consequent is likely to be a good recommendation or not. The consequents of all predictions labeled as good are then returned as the final list of recommended questions. The Apriori algorithm is typically configured by setting thresholds for two parameters: minimum support and minimum confidence. In this context, support indicates how often an itemset X appears in a dataset, while the confidence indicates how often a rule is found to be true. In particular, the support for an association rule A → B is defined as  Support(A → B)=|A ∪​B|Total no. of transactions   where “Total no. of transactions” correspond to the number of responses to the PELI survey used in developing the learner (e.g., the evidence for the rule). For our dataset, antecedents are drawn from the 16 MDS 3.0 Section F set of questions, the consequents are drawn from the 56 PELI-NH questions, and the total transactions corresponds to the 510 total responses collected. The confidence parameter for an association rule A → B is defined as  Confidence(A → B)= |A ∪​B||A| and indicates how often a consequent B occurs in rules involving A. For our implementation, we found that a minimum support of 0.01 and a minimum confidence of 0.8 produced the best results in that they expose the largest number of rules (support) and provide the strongest association between antecedent and consequent (confidence). In addition to the Apriori algorithm, we applied the use of logistic regression configured with a generalized linear regression model to help identify which rules had the highest potential utility. This model assigns the properties of the Apriori algorithm as the independent variables, such as support, confidence, lift and their corresponding average and maximum values to predict whether a consequent is likely to be important or not. In general, this allowed us to achieve a relatively high precision while keeping support relatively low to allow for inclusion of a larger set of rules. Finally, given that support, confidence, and lift all play a part in a rule’s overall usefulness, these measures are individually unsuitable for ranking or ordering of rules. Instead, a score metric is used, which is simply the product of support and confidence (Lin, 2000), and similarly liftscore which is the product of support, confidence, and lift. Results Demographic Characteristics NH residents were mostly widowed (44%) white (77%) females (67.8%) with a mean age of 81 years (standard deviation [SD] = 11.21) and a high school education (54%). Residents had an average length of stay of 924 days and an average MMSE score of 24.6 (SD = 3.9; Supplementary Appendix A). To illustrate our approach, we present results from two residents with different preference profiles. As with any instance of our approach, we begin with training a learner using the combined apriori/logistic regression approach to build the consequent rule sets, using a minimum confidence of 0.8 and a minimum support of 0.01 for the Apriori algorithm. We chose a relatively high confidence to strongly associate antecedent preferences with consequent preferences and chose a relatively low support in order to expose a wider variety of rulesets. In the approach, we started with responses to the 16 MDS 3.0 Section F questions (Table 1). Resident 1 is a 95-year old white female with a high school education. Resident 2 is 90-year old white male with some college education. For example, among the many preferences indicated, Resident 1 finds snack availability to be not important at all (as indicated by the assignment of a value of 4), while Resident 2 finds snack availability to be very important (as indicated by the assignment of a value of 1). We ran each individual resident’s preferences against learned recommendations that were built from 80% of the 510 total responses available in the dataset as generated by the combined apriori/logistic regression learner to arrive at individual sets of recommendations of questions to pursue from the remaining 56 non-MDS PELI-NH questions (Tables 2 and 3). The tables are sorted using the score metric, with the corresponding support, confidence, and lift scores shown for reference. In addition, the final two columns show the actual preference reported by the resident and the predicted or recommendation of the learner for comparison (see “pred” column in Tables 2 and 3). For example, in Table 2, the machine learner has recommended 39 of the 56 questions be considered in decreasing order based on the score. The actual column shows that the predictions for the first several questions are correct (as indicated by the “1”) with respect to the resident’s actual response. All of the recommended preferences were endorsed as important by the resident except for the row labeled “learning,” which corresponds to the question “How important is it to you to learn about topics that interest you?” Table 1. Actual Responses to MDS Section F Preferences Used to Inform Recommender of Two Residentsa MDS 3.0 Section F  Resident 1: white female, age 95, high school educated  Resident 2: white male, age 90, some college education  F0400 A  Choose clothes = 2  Choose clothes = 4  F0400 B  Care of personal things = 1  Care of personal things = 1  F0400 C  Choose bathing = 1  Choose bathing = 4  F0400 D  Snack availability = 4  Snack availability = 1  F0400 E  Bedtime = 2  Bedtime = 3  F0400 F  Who involved in care discussions = 1  Who involved in care discussions = 1  F0400 G  Private phone = 1  Private phone = 1  F0400 H  Lock things = 1  Lock things = 4  F0500 A  Reading material = 1  Reading material = 1  F0500 B  Music = 3  Music = 1  F0500 C  Pets = 4  Pets = 3  F0500 D  News = 3  News = 4  F0500 E  Group activities = 4  Group activities = 2  F0500 F  Activities = 1  Activities = 1  F0500 G  Outside time = 1  Outside time = 1  F0500 H  Religious services = 1  Religious services = 4  MDS 3.0 Section F  Resident 1: white female, age 95, high school educated  Resident 2: white male, age 90, some college education  F0400 A  Choose clothes = 2  Choose clothes = 4  F0400 B  Care of personal things = 1  Care of personal things = 1  F0400 C  Choose bathing = 1  Choose bathing = 4  F0400 D  Snack availability = 4  Snack availability = 1  F0400 E  Bedtime = 2  Bedtime = 3  F0400 F  Who involved in care discussions = 1  Who involved in care discussions = 1  F0400 G  Private phone = 1  Private phone = 1  F0400 H  Lock things = 1  Lock things = 4  F0500 A  Reading material = 1  Reading material = 1  F0500 B  Music = 3  Music = 1  F0500 C  Pets = 4  Pets = 3  F0500 D  News = 3  News = 4  F0500 E  Group activities = 4  Group activities = 2  F0500 F  Activities = 1  Activities = 1  F0500 G  Outside time = 1  Outside time = 1  F0500 H  Religious services = 1  Religious services = 4  Note:a1 = very important, 2 = somewhat important, 3 = not very important, 4 = not important at all. View Large Table 1. Actual Responses to MDS Section F Preferences Used to Inform Recommender of Two Residentsa MDS 3.0 Section F  Resident 1: white female, age 95, high school educated  Resident 2: white male, age 90, some college education  F0400 A  Choose clothes = 2  Choose clothes = 4  F0400 B  Care of personal things = 1  Care of personal things = 1  F0400 C  Choose bathing = 1  Choose bathing = 4  F0400 D  Snack availability = 4  Snack availability = 1  F0400 E  Bedtime = 2  Bedtime = 3  F0400 F  Who involved in care discussions = 1  Who involved in care discussions = 1  F0400 G  Private phone = 1  Private phone = 1  F0400 H  Lock things = 1  Lock things = 4  F0500 A  Reading material = 1  Reading material = 1  F0500 B  Music = 3  Music = 1  F0500 C  Pets = 4  Pets = 3  F0500 D  News = 3  News = 4  F0500 E  Group activities = 4  Group activities = 2  F0500 F  Activities = 1  Activities = 1  F0500 G  Outside time = 1  Outside time = 1  F0500 H  Religious services = 1  Religious services = 4  MDS 3.0 Section F  Resident 1: white female, age 95, high school educated  Resident 2: white male, age 90, some college education  F0400 A  Choose clothes = 2  Choose clothes = 4  F0400 B  Care of personal things = 1  Care of personal things = 1  F0400 C  Choose bathing = 1  Choose bathing = 4  F0400 D  Snack availability = 4  Snack availability = 1  F0400 E  Bedtime = 2  Bedtime = 3  F0400 F  Who involved in care discussions = 1  Who involved in care discussions = 1  F0400 G  Private phone = 1  Private phone = 1  F0400 H  Lock things = 1  Lock things = 4  F0500 A  Reading material = 1  Reading material = 1  F0500 B  Music = 3  Music = 1  F0500 C  Pets = 4  Pets = 3  F0500 D  News = 3  News = 4  F0500 E  Group activities = 4  Group activities = 2  F0500 F  Activities = 1  Activities = 1  F0500 G  Outside time = 1  Outside time = 1  F0500 H  Religious services = 1  Religious services = 4  Note:a1 = very important, 2 = somewhat important, 3 = not very important, 4 = not important at all. View Large Table 2. Recommender Predicted Resident Preferences—Resident 1 Consequent  score  liftscore  support  conf  lift  act  pred  Food choice  0.2526  0.3058  0.3053  0.8270  1.2094  1  1  Living place care  0.2275  0.2661  0.2714  0.8418  1.2826  1  1  Bed setup  0.2051  0.2537  0.2510  0.8166  1.3395  1  1  Hobbies  0.2020  0.2759  0.2343  0.8528  1.3761  1  1  Learning  0.1795  0.2281  0.2168  0.8449  1.4912  0  1  Staff care  0.1543  0.1913  0.1854  0.8505  1.3336  1  1  Family contact  0.1488  0.1701  0.1666  0.9089  1.2531  1  1  Choose mouth care  0.1478  0.1756  0.1751  0.8484  1.2293  1  1  Helps feel better  0.1443  0.1792  0.1740  0.8476  1.3171  1  1  Choose care professional  0.1415  0.1630  0.1645  0.8710  1.1652  0  1  Staff respect  0.1360  0.1745  0.1570  0.8704  1.3867  0  1  Bath frequency  0.1232  0.1510  0.1493  0.8420  1.2548  1  1  Privacy  0.1125  0.1565  0.1354  0.8574  1.4337  1  1  Choose hair care  0.1107  0.1377  0.1344  0.8398  1.2650  1  1  Bath time  0.1076  0.1333  0.1304  0.8416  1.2624  1  1  Choose nail care frequency  0.1076  0.1333  0.1304  0.8416  1.2624  1  1  Caregiver gender  0.0982  0.1368  0.1180  0.8603  1.4385  1  1  Choose used name  0.0982  0.1368  0.1180  0.8603  1.4385  1  1  Mental professional if sad  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Caregiver know bathroom needs  0.0758  0.0959  0.0924  0.8536  1.3795  1  1  Bedtime routine  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Choose wake time  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Follow morning routine  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Naps  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Room temp  0.0263  0.0428  0.0319  0.8213  1.8613  1  1  TV  0.0162  0.0246  0.0196  0.8343  1.5748  1  1  1-on-1 time  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Alone time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Children  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Friends contact  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Meet new people  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Room lighting  0.0146  0.0295  0.0180  0.8172  2.0758  1  1  Entertainment events  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Exercise  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Games  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Outdoor tasks  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Plant care  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Sports  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Watch movies with others  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Consequent  score  liftscore  support  conf  lift  act  pred  Food choice  0.2526  0.3058  0.3053  0.8270  1.2094  1  1  Living place care  0.2275  0.2661  0.2714  0.8418  1.2826  1  1  Bed setup  0.2051  0.2537  0.2510  0.8166  1.3395  1  1  Hobbies  0.2020  0.2759  0.2343  0.8528  1.3761  1  1  Learning  0.1795  0.2281  0.2168  0.8449  1.4912  0  1  Staff care  0.1543  0.1913  0.1854  0.8505  1.3336  1  1  Family contact  0.1488  0.1701  0.1666  0.9089  1.2531  1  1  Choose mouth care  0.1478  0.1756  0.1751  0.8484  1.2293  1  1  Helps feel better  0.1443  0.1792  0.1740  0.8476  1.3171  1  1  Choose care professional  0.1415  0.1630  0.1645  0.8710  1.1652  0  1  Staff respect  0.1360  0.1745  0.1570  0.8704  1.3867  0  1  Bath frequency  0.1232  0.1510  0.1493  0.8420  1.2548  1  1  Privacy  0.1125  0.1565  0.1354  0.8574  1.4337  1  1  Choose hair care  0.1107  0.1377  0.1344  0.8398  1.2650  1  1  Bath time  0.1076  0.1333  0.1304  0.8416  1.2624  1  1  Choose nail care frequency  0.1076  0.1333  0.1304  0.8416  1.2624  1  1  Caregiver gender  0.0982  0.1368  0.1180  0.8603  1.4385  1  1  Choose used name  0.0982  0.1368  0.1180  0.8603  1.4385  1  1  Mental professional if sad  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Caregiver know bathroom needs  0.0758  0.0959  0.0924  0.8536  1.3795  1  1  Bedtime routine  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Choose wake time  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Follow morning routine  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Naps  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Room temp  0.0263  0.0428  0.0319  0.8213  1.8613  1  1  TV  0.0162  0.0246  0.0196  0.8343  1.5748  1  1  1-on-1 time  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Alone time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Children  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Friends contact  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Meet new people  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Room lighting  0.0146  0.0295  0.0180  0.8172  2.0758  1  1  Entertainment events  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Exercise  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Games  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Outdoor tasks  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Plant care  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Sports  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Watch movies with others  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Note: act = actual; conf = confidence; pred = predicted. View Large Table 2. Recommender Predicted Resident Preferences—Resident 1 Consequent  score  liftscore  support  conf  lift  act  pred  Food choice  0.2526  0.3058  0.3053  0.8270  1.2094  1  1  Living place care  0.2275  0.2661  0.2714  0.8418  1.2826  1  1  Bed setup  0.2051  0.2537  0.2510  0.8166  1.3395  1  1  Hobbies  0.2020  0.2759  0.2343  0.8528  1.3761  1  1  Learning  0.1795  0.2281  0.2168  0.8449  1.4912  0  1  Staff care  0.1543  0.1913  0.1854  0.8505  1.3336  1  1  Family contact  0.1488  0.1701  0.1666  0.9089  1.2531  1  1  Choose mouth care  0.1478  0.1756  0.1751  0.8484  1.2293  1  1  Helps feel better  0.1443  0.1792  0.1740  0.8476  1.3171  1  1  Choose care professional  0.1415  0.1630  0.1645  0.8710  1.1652  0  1  Staff respect  0.1360  0.1745  0.1570  0.8704  1.3867  0  1  Bath frequency  0.1232  0.1510  0.1493  0.8420  1.2548  1  1  Privacy  0.1125  0.1565  0.1354  0.8574  1.4337  1  1  Choose hair care  0.1107  0.1377  0.1344  0.8398  1.2650  1  1  Bath time  0.1076  0.1333  0.1304  0.8416  1.2624  1  1  Choose nail care frequency  0.1076  0.1333  0.1304  0.8416  1.2624  1  1  Caregiver gender  0.0982  0.1368  0.1180  0.8603  1.4385  1  1  Choose used name  0.0982  0.1368  0.1180  0.8603  1.4385  1  1  Mental professional if sad  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Caregiver know bathroom needs  0.0758  0.0959  0.0924  0.8536  1.3795  1  1  Bedtime routine  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Choose wake time  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Follow morning routine  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Naps  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Room temp  0.0263  0.0428  0.0319  0.8213  1.8613  1  1  TV  0.0162  0.0246  0.0196  0.8343  1.5748  1  1  1-on-1 time  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Alone time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Children  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Friends contact  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Meet new people  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Room lighting  0.0146  0.0295  0.0180  0.8172  2.0758  1  1  Entertainment events  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Exercise  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Games  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Outdoor tasks  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Plant care  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Sports  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Watch movies with others  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Consequent  score  liftscore  support  conf  lift  act  pred  Food choice  0.2526  0.3058  0.3053  0.8270  1.2094  1  1  Living place care  0.2275  0.2661  0.2714  0.8418  1.2826  1  1  Bed setup  0.2051  0.2537  0.2510  0.8166  1.3395  1  1  Hobbies  0.2020  0.2759  0.2343  0.8528  1.3761  1  1  Learning  0.1795  0.2281  0.2168  0.8449  1.4912  0  1  Staff care  0.1543  0.1913  0.1854  0.8505  1.3336  1  1  Family contact  0.1488  0.1701  0.1666  0.9089  1.2531  1  1  Choose mouth care  0.1478  0.1756  0.1751  0.8484  1.2293  1  1  Helps feel better  0.1443  0.1792  0.1740  0.8476  1.3171  1  1  Choose care professional  0.1415  0.1630  0.1645  0.8710  1.1652  0  1  Staff respect  0.1360  0.1745  0.1570  0.8704  1.3867  0  1  Bath frequency  0.1232  0.1510  0.1493  0.8420  1.2548  1  1  Privacy  0.1125  0.1565  0.1354  0.8574  1.4337  1  1  Choose hair care  0.1107  0.1377  0.1344  0.8398  1.2650  1  1  Bath time  0.1076  0.1333  0.1304  0.8416  1.2624  1  1  Choose nail care frequency  0.1076  0.1333  0.1304  0.8416  1.2624  1  1  Caregiver gender  0.0982  0.1368  0.1180  0.8603  1.4385  1  1  Choose used name  0.0982  0.1368  0.1180  0.8603  1.4385  1  1  Mental professional if sad  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Caregiver know bathroom needs  0.0758  0.0959  0.0924  0.8536  1.3795  1  1  Bedtime routine  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Choose wake time  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Follow morning routine  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Naps  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Room temp  0.0263  0.0428  0.0319  0.8213  1.8613  1  1  TV  0.0162  0.0246  0.0196  0.8343  1.5748  1  1  1-on-1 time  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Alone time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Children  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Friends contact  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Meet new people  0.0149  0.0342  0.0159  0.9375  2.3042  0  1  Room lighting  0.0146  0.0295  0.0180  0.8172  2.0758  1  1  Entertainment events  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Exercise  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Games  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Outdoor tasks  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Plant care  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Sports  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Watch movies with others  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Note: act = actual; conf = confidence; pred = predicted. View Large Table 3. Recommender Predicted Resident Preferences—Resident 2 Consequent  score  liftscore  support  conf  lift  act  pred  Living place care  0.2280  0.2665  0.2727  0.8388  1.3200  0  1  Hobbies  0.2075  0.2789  0.2447  0.8434  1.3555  1  1  Food choice  0.1991  0.2440  0.2376  0.8409  1.2298  1  1  Learning  0.1969  0.2492  0.2382  0.8397  1.4698  1  1  Staff care  0.1384  0.1740  0.1666  0.8523  1.3451  1  1  Staff respect  0.1381  0.1736  0.1572  0.8842  1.3480  1  1  Helps feel better  0.1380  0.1758  0.1663  0.8528  1.3640  1  1  Choose mouth care  0.1371  0.1636  0.1624  0.8523  1.2325  1  1  Choose care professional  0.1337  0.1543  0.1552  0.8718  1.1662  1  1  Family contact  0.1202  0.1363  0.1360  0.9030  1.2558  1  1  Bed setup  0.1175  0.1454  0.1443  0.8294  1.3535  1  1  Choose nail care frequency  0.1076  0.1333  0.1304  0.8416  1.2624  0  1  Bath frequency  0.1061  0.1313  0.1286  0.8416  1.2609  0  1  Bath time  0.1058  0.1311  0.1282  0.8411  1.2705  1  1  Choose hair care  0.1049  0.1302  0.1273  0.8410  1.2650  0  1  Privacy  0.1049  0.1461  0.1261  0.8585  1.4354  1  1  Room temp  0.0994  0.1406  0.1206  0.8202  1.5735  1  1  Caregiver know bathroom needs  0.0993  0.1250  0.1201  0.8537  1.4079  1  1  Caregiver gender  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Choose used name  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Mental professional if sad  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Bedtime routine  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Choose wake time  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Follow morning routine  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Naps  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Children  0.0172  0.0299  0.0198  0.8750  1.7623  1  1  Room lighting  0.0162  0.0324  0.0196  0.8265  2.0936  1  1  1-on-1 time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Alone time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Meet new people  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Friends contact  0.0145  0.0292  0.0165  0.8816  2.0072  1  1  TV  0.0132  0.0199  0.0155  0.8499  1.5135  1  1  Exercise  0.0117  0.0248  0.0135  0.8661  2.1246  1  1  Outdoor tasks  0.0117  0.0268  0.0135  0.8661  2.2818  1  1  Entertainment events  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Games  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Plant care  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Sports  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Watch movies with others  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Reminisce  0.0102  0.0175  0.0123  0.8333  1.7172  1  1  Consequent  score  liftscore  support  conf  lift  act  pred  Living place care  0.2280  0.2665  0.2727  0.8388  1.3200  0  1  Hobbies  0.2075  0.2789  0.2447  0.8434  1.3555  1  1  Food choice  0.1991  0.2440  0.2376  0.8409  1.2298  1  1  Learning  0.1969  0.2492  0.2382  0.8397  1.4698  1  1  Staff care  0.1384  0.1740  0.1666  0.8523  1.3451  1  1  Staff respect  0.1381  0.1736  0.1572  0.8842  1.3480  1  1  Helps feel better  0.1380  0.1758  0.1663  0.8528  1.3640  1  1  Choose mouth care  0.1371  0.1636  0.1624  0.8523  1.2325  1  1  Choose care professional  0.1337  0.1543  0.1552  0.8718  1.1662  1  1  Family contact  0.1202  0.1363  0.1360  0.9030  1.2558  1  1  Bed setup  0.1175  0.1454  0.1443  0.8294  1.3535  1  1  Choose nail care frequency  0.1076  0.1333  0.1304  0.8416  1.2624  0  1  Bath frequency  0.1061  0.1313  0.1286  0.8416  1.2609  0  1  Bath time  0.1058  0.1311  0.1282  0.8411  1.2705  1  1  Choose hair care  0.1049  0.1302  0.1273  0.8410  1.2650  0  1  Privacy  0.1049  0.1461  0.1261  0.8585  1.4354  1  1  Room temp  0.0994  0.1406  0.1206  0.8202  1.5735  1  1  Caregiver know bathroom needs  0.0993  0.1250  0.1201  0.8537  1.4079  1  1  Caregiver gender  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Choose used name  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Mental professional if sad  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Bedtime routine  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Choose wake time  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Follow morning routine  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Naps  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Children  0.0172  0.0299  0.0198  0.8750  1.7623  1  1  Room lighting  0.0162  0.0324  0.0196  0.8265  2.0936  1  1  1-on-1 time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Alone time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Meet new people  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Friends contact  0.0145  0.0292  0.0165  0.8816  2.0072  1  1  TV  0.0132  0.0199  0.0155  0.8499  1.5135  1  1  Exercise  0.0117  0.0248  0.0135  0.8661  2.1246  1  1  Outdoor tasks  0.0117  0.0268  0.0135  0.8661  2.2818  1  1  Entertainment events  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Games  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Plant care  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Sports  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Watch movies with others  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Reminisce  0.0102  0.0175  0.0123  0.8333  1.7172  1  1  Note: act = actual; conf = confidence; pred = predicted. View Large Table 3. Recommender Predicted Resident Preferences—Resident 2 Consequent  score  liftscore  support  conf  lift  act  pred  Living place care  0.2280  0.2665  0.2727  0.8388  1.3200  0  1  Hobbies  0.2075  0.2789  0.2447  0.8434  1.3555  1  1  Food choice  0.1991  0.2440  0.2376  0.8409  1.2298  1  1  Learning  0.1969  0.2492  0.2382  0.8397  1.4698  1  1  Staff care  0.1384  0.1740  0.1666  0.8523  1.3451  1  1  Staff respect  0.1381  0.1736  0.1572  0.8842  1.3480  1  1  Helps feel better  0.1380  0.1758  0.1663  0.8528  1.3640  1  1  Choose mouth care  0.1371  0.1636  0.1624  0.8523  1.2325  1  1  Choose care professional  0.1337  0.1543  0.1552  0.8718  1.1662  1  1  Family contact  0.1202  0.1363  0.1360  0.9030  1.2558  1  1  Bed setup  0.1175  0.1454  0.1443  0.8294  1.3535  1  1  Choose nail care frequency  0.1076  0.1333  0.1304  0.8416  1.2624  0  1  Bath frequency  0.1061  0.1313  0.1286  0.8416  1.2609  0  1  Bath time  0.1058  0.1311  0.1282  0.8411  1.2705  1  1  Choose hair care  0.1049  0.1302  0.1273  0.8410  1.2650  0  1  Privacy  0.1049  0.1461  0.1261  0.8585  1.4354  1  1  Room temp  0.0994  0.1406  0.1206  0.8202  1.5735  1  1  Caregiver know bathroom needs  0.0993  0.1250  0.1201  0.8537  1.4079  1  1  Caregiver gender  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Choose used name  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Mental professional if sad  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Bedtime routine  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Choose wake time  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Follow morning routine  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Naps  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Children  0.0172  0.0299  0.0198  0.8750  1.7623  1  1  Room lighting  0.0162  0.0324  0.0196  0.8265  2.0936  1  1  1-on-1 time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Alone time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Meet new people  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Friends contact  0.0145  0.0292  0.0165  0.8816  2.0072  1  1  TV  0.0132  0.0199  0.0155  0.8499  1.5135  1  1  Exercise  0.0117  0.0248  0.0135  0.8661  2.1246  1  1  Outdoor tasks  0.0117  0.0268  0.0135  0.8661  2.2818  1  1  Entertainment events  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Games  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Plant care  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Sports  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Watch movies with others  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Reminisce  0.0102  0.0175  0.0123  0.8333  1.7172  1  1  Consequent  score  liftscore  support  conf  lift  act  pred  Living place care  0.2280  0.2665  0.2727  0.8388  1.3200  0  1  Hobbies  0.2075  0.2789  0.2447  0.8434  1.3555  1  1  Food choice  0.1991  0.2440  0.2376  0.8409  1.2298  1  1  Learning  0.1969  0.2492  0.2382  0.8397  1.4698  1  1  Staff care  0.1384  0.1740  0.1666  0.8523  1.3451  1  1  Staff respect  0.1381  0.1736  0.1572  0.8842  1.3480  1  1  Helps feel better  0.1380  0.1758  0.1663  0.8528  1.3640  1  1  Choose mouth care  0.1371  0.1636  0.1624  0.8523  1.2325  1  1  Choose care professional  0.1337  0.1543  0.1552  0.8718  1.1662  1  1  Family contact  0.1202  0.1363  0.1360  0.9030  1.2558  1  1  Bed setup  0.1175  0.1454  0.1443  0.8294  1.3535  1  1  Choose nail care frequency  0.1076  0.1333  0.1304  0.8416  1.2624  0  1  Bath frequency  0.1061  0.1313  0.1286  0.8416  1.2609  0  1  Bath time  0.1058  0.1311  0.1282  0.8411  1.2705  1  1  Choose hair care  0.1049  0.1302  0.1273  0.8410  1.2650  0  1  Privacy  0.1049  0.1461  0.1261  0.8585  1.4354  1  1  Room temp  0.0994  0.1406  0.1206  0.8202  1.5735  1  1  Caregiver know bathroom needs  0.0993  0.1250  0.1201  0.8537  1.4079  1  1  Caregiver gender  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Choose used name  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Mental professional if sad  0.0982  0.1368  0.1180  0.8603  1.4385  0  1  Bedtime routine  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Choose wake time  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Follow morning routine  0.0493  0.0766  0.0598  0.8648  1.6260  1  1  Naps  0.0493  0.0766  0.0598  0.8648  1.6260  0  1  Children  0.0172  0.0299  0.0198  0.8750  1.7623  1  1  Room lighting  0.0162  0.0324  0.0196  0.8265  2.0936  1  1  1-on-1 time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Alone time  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Meet new people  0.0149  0.0342  0.0159  0.9375  2.3042  1  1  Friends contact  0.0145  0.0292  0.0165  0.8816  2.0072  1  1  TV  0.0132  0.0199  0.0155  0.8499  1.5135  1  1  Exercise  0.0117  0.0248  0.0135  0.8661  2.1246  1  1  Outdoor tasks  0.0117  0.0268  0.0135  0.8661  2.2818  1  1  Entertainment events  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Games  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Plant care  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Sports  0.0115  0.0253  0.0133  0.8673  2.1844  1  1  Watch movies with others  0.0115  0.0253  0.0133  0.8673  2.1844  0  1  Reminisce  0.0102  0.0175  0.0123  0.8333  1.7172  1  1  Note: act = actual; conf = confidence; pred = predicted. View Large The recommendations for Resident 2 in Table 3 demonstrate how different responses to the MDS items produce a different set of recommended preferences to ask. This is due in large part to the fact that the MDS preferences for Resident 2 are indeed different from the preferences of Resident 1. In this case, the machine learner has recommended 40 of the 56 questions be considered. The very first recommended preference, Living place care (i.e., “How important is it to you to take care of the place you live?”), was not endorsed as important by the resident’s actual data. However, the subsequent 10 recommendations were correctly predicted. Obviously, a recommendation of asking an additional 39 or 40 questions would still be time-consuming and burdensome for a resident respondent. Therefore, the decreasing score metric can be used to identify a threshold or cut-off by which to configure the recommender for suggesting which questions to pursue with a resident. A defined threshold would allow us to configure the recommender to tailor the preference interview further. For instance, if a threshold of 0.13 is used, then for Resident 1, the first 12 questions shown in the list would be pursued, while for that same threshold, the first 10 questions would be pursued for Resident 2. The precise selection of the threshold is currently part of our future investigations and will rely on a number of factors including analysis of the correctness of the predictions (i.e., did the resident agree with the recommendation). We used a fivefold cross-validation to evaluate the quality of the recommender and to identify the best configuration of the combined apriori/logistic regression approach described above. In the fivefold cross-validation, we grouped the data into five partitions, using four partitions, or 80% of the available data to train the learner and the remaining partition, or 20% as test data. This was repeated five times, rotating each partition as a test set against the remaining training partitions. In our evaluation of the approach, we examined a number of variations of the parameterization of the algorithms and models and received roughly the same performance across those variations. We were able to obtain recall = .8021, precision = .7919, accuracy = .6979, and F1-score = .7953. Recall measures the ratio of correctly predicted rules compared to all of the predicted rules (either positive or negative). Similarly, precision measures the ratio of correctly predicted rules with respect to the rules predicted to be true. In both instances, a correctly predicted rule corresponds to an association rule in which the consequent is predicted to be important (i.e., very important or somewhat important) that is indeed important to the resident. The overall accuracy of the approach (i.e., ratio of correctly predicted preferences and nonpreferences) is approximately 70%. The F1-score provides a harmonic average of both precision and recall, and in this case is nearly 0.80. While precision and recall are high, the F1-score of nearly 0.80 emphasizes a balance in the ability of the learner to predict actual preferences. To put these results in perspective, we note that the quality of a learner and recommender is always dependent on the domain in which it is being applied. From this standpoint, the results here indicate that the learner provides recommendations that are relevant to a resident about four times in five. Contrast this with a recommender that uses a coin-flip to make the binary choice of preference or nonpreference, which would result in identifying a preference only 50% of the time. When translated into a savings in time to both the interviewer and resident, the potential is noteworthy. Discussion The PELI-NH is increasingly being used as a guide for providing preference-based person-centered care in NH communities. The amount of time needed to complete the 72 item PELI-NH is a considerable barrier. Our machine learning mechanism has the potential to reduce the time needed for completing the PELI-NH interview while still incorporating important resident preferences. Our initial work has shown that we can achieve a reasonable rate of accuracy in providing recommendations on potential preferences for a resident with a high rate of precision, a fruitful potential for providers. We are unaware of other recommendation systems that have been created for addressing preferences in the context of person-centered care. Clearly, however, the idea of developing recommendation systems is not a new one (Aggarwal, 2016). The work reported on here uses an implementation that has been applied in many contexts using two well-known methodologies (Agrawal & Srikant, 1994; Nelder & Wedderburn, 1972). Furthermore, the results we have attained are consistent with the evaluation metrics used in recommendation tasks (Gunawardana & Shani, 2009), and could be applied in all long-term services and support settings (e.g., NH, assisted living, and home- and community-based settings). The novelty lies in assisting provider communities with a methodology for conducting PELI-NH interviews that reduces respondent fatigue and staff time to conduct interviews while also alleviating the burden of additional paperwork. Through using a recommender, questions are tailored for each resident. This approach could be seen as a parallel to precision health, where treatments are tailored to the unique genetic, environmental, and lifestyle habits of individual patients. Understanding the unique preference inventory of each NH resident can aid in bringing preferences into practice to improve the quality of care and quality of life to best meet the psychosocial needs of each person. In addition, the recommendations can be used as a suggested next best preference to ask. A provider community could decide to focus on the top-3, -5, or -10 recommendations for each resident. Alternatively recommendations can be based upon items with a score or liftscore that exceeds a particular value. In our example, we used 0.13 as a threshold for illustration but ultimately, the chosen threshold will be based on the experiences of the staff that are administering the PELI interviews. Regardless of the threshold, the approach allows provider communities to select preference questions according to the ranking provided by the recommender. In addition, the preference recommender can overcome barriers such as the amount of time needed for documenting important preferences through integration into existing electronic medical records. It would also remediate the lack of support within the staff workflow, which has also been found to be a major barrier to the adoption of new technology (Schulz et al., 2015). In developing the recommender, we have focused on positive results from precision and recall for crafting a recommendation set, as is consistent with the work of Lin (2000). As such, we chose not to optimize specificity, which is an indicator of whether a preference is unimportant. Overall, we expect that as we continue to develop the algorithm, we will be able to continue to improve the rule-based recommender. In addition, we plan on exploring other potential methods for suggesting preferences. We envision this recommender system is a novel method for providing guidance to provider communities on how to best tailor preference interviews to residents. Next Steps The recommender that we have built (and the results shown here) are based on a dataset that is indicative of a specific set of communities. The recommendations for other communities and for specific residents evolve over time based on the preferences of the residents in the community. For instance, the preferences of communities in the Pacific Northwest may be different from the preferences of communities in the Southern United States. As such, it is stressed that the recommended questions should always be validated against a resident’s actual preferences. Finally, the score and liftscore thresholds used to recommend questions to be pursued need to be selected based on the experiences of a specific community and person, again taking account the issues of fatigue and paperwork burden. Next steps include building in a component to the recommender that seeks provider input in determining the thresholds. For example, the provider could indicate that they only have the capacity to ask 3, 5, or 10 additional questions. As we work to refine the recommender, our goal is to use it as an active learning system, with the hope of gaining the benefits of collaborative filtering. In other words, the quality of the recommendations has the potential to improve with the increase in the number of resident interviews that are conducted. Each additional transaction allows us to refine the rule sets through the benefits of the wisdom of the crowd. In addition, we are interested in applying other recommendation strategies, including item-based collaborative filtering (Sarwar, Karypis, Konstan, & Reidl, 2001) which identifies similarity based on the ratings of products rather than the transactional co-occurrence approach described here. Limitations We evaluated the learning approach using a relatively small transaction dataset (n = 510) with an assumption that responses of individuals were independent when in fact the data contained PELI-NH responses of n = 255 NH residents asked at two points in time 3 months apart. This assumption is akin to assuming that shopping transactions are independent across separate shopping trips to a market. We also assumed that the responses of very important and somewhat important indicate “preference,” while other responses indicated “nonpreference.” This decision was based upon the findings of Van Haitsma and colleagues (2014) and emphasizes the fact that our approach is only concerned with recommending PELI-NH items to explore rather than trying to precisely predict a preference rating of very important, somewhat important, not very important, or not important at all for a resident. With respect to the data, despite a rigorous approach to sampling among participants and settings, all respondents had the cognitive capability to answer questions about their preferences. Therefore, preferences of individuals with moderate or severe cognitive impairment, or those who were unable to communicate are not represented in this study. In addition, the sample was drawn from large metropolitan areas on the East Coast of the United States and may not reflect the preferences of NH residents living in other areas of the country. Future work in this area will be improved as more data are added. For example, including the voices of people living with cognitive impairment and individuals living in different areas of the country will improve accuracy of the recommender. Conclusion At a time when NH providers are asked to improve the quality and satisfaction with care at a lower cost we must embrace technology to help improve the efficiency and effectiveness of care. The findings from this article highlight the applicability of machine learning and specifically recommender systems to the NH setting. We found that the recommender system was extremely accurate at predicting important preferences of NH residents. Because the recommender uses data from other residents to predict, the algorithm will become more accurate as larger datasets become available. Ultimately, the use of machine learning could assist NH providers in tailoring their preference assessments to maximize staff time while minimizing the burden of asking a resident dozens of questions. This work provides direction for future research and is critical in advancing the delivery of preference-based, person-centered care. Supplementary Material Supplementary data are available at The Gerontologist online. Funding This work was supported by generous funding from the National Institute of Nursing Research grant (R21NR011334 to K. Van Haitsma [PI]), the Patrick and Catherine Weldon Donaghue Medical Research Foundation, and the Ohio Department of Medicaid. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Nursing Research, the National Institutes of Health, the Donaghue Foundation, or the Ohio Department of Medicaid. Conflict of Interest None reported. Acknowledgments We would like to thank Karen Eshraghi and Christina Duntzee, the research team members who worked diligently to collect this data, and the older adults who participated in the project. References Abbott, K. M., Klumpp, R., Leser, K., Straker, J., Gannod, G., & Van Haitsma, K. ( 2018). Delivering person-centered care: Important preferences for recipients of long-term services and supports. Journal of the American Medical Directors Association , 19, 169– 173. doi: 10.1016/j.jamda.2017.10.005 Google Scholar CrossRef Search ADS PubMed  Aggarwal, C. C. ( 2016). Recommender systems: The textbook  ( 1st ed.). Springer International Publishing. Retrieved from https://doi.org/10.1007/978-3-319-29659-3 Google Scholar CrossRef Search ADS   Agrawal, R., & Srikant, R. ( 1994). Fast algorithms for mining association rules in large databases. In Proceedings of the 20th International Conference on Very Large Data Bases (pp. 487– 499). San Francisco: Morgan Kaufmann. Retrieved from http://dl.acm.org/citation.cfm?id=645920.672836 Ashcraft, A., Cherry, B., & Owen, D. ( 2007) Perceptions of job satisfaction and the regulatory environment among nurse aides and charge nurses in long-term care. Geriatric Nursing , 28, 183– 192. doi: 10.1016/j.gerinurse.2007.01.015 Google Scholar CrossRef Search ADS PubMed  Bangerter, L. R., Abbott, K., Heid, A., Eshraghi, K., & Van Haitsma, K. ( 2017). Using spontaneous commentary of nursing home residents to develop resident-centered measurement tools: A case study. Geriatric Nursing (New York, N.Y.) , 38, 548– 550. doi: 10.1016/j.gerinurse.2017.04.003 Google Scholar CrossRef Search ADS PubMed  Bendakir, N., & Aimeur, E. ( 2006). Using association rules for course recommendation. In Proceedings of the Association for the Advancement of Artificial Intelligence Workshop on Educational Data Mining (pp. 31– 40). Retrieved from https://www.aaai.org/Papers/Workshops/2006/WS-06-05/WS06-05-005.pdf Cakir, O., & Aras, M. E. ( 2012). A recommendation engine by using association rules. Procedia—Social and Behavioral Sciences , 62, 452– 456. doi: 10.1016/j.sbspro.2012.09.074 Google Scholar CrossRef Search ADS   Carpenter, B. D., Van Haitsma, K., Ruckdeschel, K., & Lawton, M. P. ( 2000). The psychosocial preferences of older adults: A pilot examination of content and structure. The Gerontologist , 40, 335– 348. doi: 10.1093/geront/40.3.335 Google Scholar CrossRef Search ADS PubMed  Centers for Medicare and Medicaid Services.( 2015). Impact act of 2014 and cross setting measures. Retrieved from https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Post-Acute-Care-Quality-Initiatives/IMPACT-Act-of-2014-and-Cross-Setting-Measures.html Clark, S., Elswick, S., Gabriel, M., Gurupur, V., & Wisniewski, P. ( 2016). Transitions of care: A patient-centered perspective of health information systems that support post-acute care. Journal of Integrated Design and Process Science , 20, 95– 110. doi: 10.3233/jid-2016-0008 Google Scholar CrossRef Search ADS   Curyto, K., Van Haitsma, K. S., & Towsley, G. L. ( 2016). Cognitive interviewing: Revising the preferences for everyday living inventory for use in the nursing home. Research in Gerontological Nursing , 9, 24– 34. doi: 10.3928/19404921- 20150522-04 Google Scholar CrossRef Search ADS PubMed  Edelen, M., Gage, B. J., Rose, A. J., Ahluwalia, S., Soo Jin DeSantis, A., Dunbar, M. S.,… Stucky, B. D. ( 2017). Development and maintenance of standardized cross setting patient assessment data for post-acute care: Summary report of findings from alpha 1 pilot testing . Santa Monica, CA: RAND Corporation. Retrieved from https://www.rand.org/pubs/research_reports/RR1895.html Google Scholar CrossRef Search ADS   Folstein, M. F., Folstein, S. E., & McHugh, P. R. ( 1975). “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research , 12, 189– 198.  doi: 10.1016/0022-3956(75) 90026-6 Google Scholar CrossRef Search ADS PubMed  Gunawardana, A., & Shani, G. ( 2009). A survey of accuracy evaluation metrics of recommendation tasks. The Journal of Machine Learning Research , 10, 2935– 2962. doi: 10.1145/ 1577069.1755883 Heid, A. R., Bangerter, M. A., Abbott, K. M., & Van Haitsma, K. ( 2017). Do family proxies get it right? Concordance in reports of nursing home residents’ everyday preferences. Journal of Applied Gerontology , 36, 667– 691. doi: 10.1177/ 0733464815581485 Google Scholar CrossRef Search ADS PubMed  Hess, S., Hensher, D., & Daly, A. ( 2012). Not bored yet—revisiting respondent fatigue in stated choice experiments. Transportation Research Part A: Policy and Practice , 46, 626– 644. doi: 10.1016/j.tra.2011.11.008 Google Scholar CrossRef Search ADS   Hu, R. ( 2010). Medical data mining based on association rules. Computer and Information Science , 3, 104– 108. doi: 10.5539/cis.v3n4p104 Google Scholar CrossRef Search ADS   Koh, H. C., & Tan, G. ( 2005). Data mining applications in healthcare. Journal of Healthcare Information Management , 19, 64– 72. doi: 10.4314/ijonas.v5i1.49926 Google Scholar PubMed  Lin, W. ( 2000). Association rule mining for collaborative recommender systems . Worchester Polytechnic Institute. Retrieved from https://web.wpi.edu/Pubs/ETD/Available/etd-0515100-145926/ unrestricted/wlin.pdf Nelder, J. A., & Wedderburn, R. W. M. ( 1972). Generalized linear models. Journal of the Royal Statistical Society , 135, 370– 384. Retrieved from https://docs.ufpr.br/~taconeli/CE225/Artigo.pdf Google Scholar CrossRef Search ADS   Patel, V. L., Shortliffe, E. H., Stefanelli, M., Szolovits, P., Michael, R., Bellazzi, R.,… Martin, C. K. ( 2009). The coming of age of artificial intelligence in medicine. Artificial Intelligence , 46, 152– 159. doi: 10.1016/j.artmed.2008.07.017 Saliba, D., & Buchanan, J. ( 2008). Development and validation of a revised nursing home assessment tool: MDS 3.0 . Santa Monica, CA: Rand Corporation Health. Sarwar, B., Karypis, G., Konstan, J., & Reidl, J. ( 2001). Item-based collaborative filtering recommendation algorithms. In Proceedings of the Tenth International Conference on World Wide Web—WWW ‘01 (pp. 285– 295). doi: 10.1145/371920.372071 Schulz, R., Wahl, H. W., Matthews, J. T., De Vito Dabbs, A., Beach, S. R., & Czaja, S. J. ( 2015). Advancing the aging and technology agenda in gerontology. The Gerontologist , 55, 724– 734. doi: 10.1093/geront/gnu071 Google Scholar CrossRef Search ADS PubMed  Simovici, D. A. ( 2012). Data mining of medical data: Opportunities and challenges in mining association rules . Potsdam: IALS. Retrieved from http://www.cs.umb.edu/~dsim/papersps/dmmd.pdf Trossman, S. ( 2002). The documentation dilemma. Nurses poised to address paperwork burden. Tar Heel Nurse , 64, 10– 11. Google Scholar PubMed  Van Haitsma, K., Abbott, K. M., Heid, A. R., Carpenter, B., Curyto, K., Kleban, M.,… Spector, A. ( 2014). The consistency of self-reported preferences for everyday living: Implications for person-centered care delivery. Journal of Gerontological Nursing , 40, 34– 46. doi: 10.3928/00989134-20140820-01 Google Scholar CrossRef Search ADS PubMed  Van Haitsma, K., Curyto, K., Spector, A., Towsley, G., Kleban, M., Carpenter, B.,… Koren, M. J. ( 2013). The preferences for everyday living inventory: Scale development and description of psychosocial preferences responses in community-dwelling elders. The Gerontologist , 53, 582– 595. doi: 10.1093/geront/gns102 Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press on behalf of The Gerontological Society of America. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)

Journal

The GerontologistOxford University Press

Published: May 21, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off