Reliability Study in Five Languages of the Translation of the Pain Observational Scale Algoplus

Reliability Study in Five Languages of the Translation of the Pain Observational Scale Algoplus Abstract Objective Acute pain is frequent and underestimated in older persons, especially when they suffer from cognitive impairment. Algoplus is an observational scale for acute pain evaluation, validated in French in older persons with communication disorders. We present the validation by an international expert team of the Algoplus scale in five languages: English, Spanish, Italian, Portuguese, and Turkish. Methods A total of 181 older consecutive patients were included in five countries (Spain, Australia, Italy, Portugal, and Turkey). Test-retest and inter-rater reliabilities were determined by weighted kappa coefficient for each item and internal consistency by Kuder-Richardson-20 (KD). Results Regarding test-retest reliability, the kappa coefficient for the five items ranged from 0.68 to 0.84. Inter-rater reliability kappa values ranged from 0.64 to 0.82. Internal consistency was indicated at a KD greater than 0.6. Satisfaction was good to excellent for all teams. Results show that reliability tests are good to excellent for all items of Algoplus. Conclusions This study shows evidence that Algoplus is an acceptable, reproducible, reliable, and easy-to-use tool. It provides a unique opportunity to include the translated Algoplus scale in daily assessment of older persons with communication disorders in many countries. Acute Pain, Cognitive Impairment, Evaluation Scale, Communication Disorders Introduction Self-reporting is the most reliable indicator of the presence and intensity of pain, even for patients with cognitive impairment [1–6]. When this is not possible, pain assessment observational scales are used. A number of observational pain assessment tools has been developed and validated [2,3] for older persons with communication disorders and moderate to severe dementia. The Algoplus scale has been validated in 349 patients with acute pain and communication disorders and is widely used in French-speaking countries [7]. It consists of five items: “facial expression,” “look,” “complaint,” “body position,” and “atypical behavior” and may complement the Doloplus scale [8,9] that has a larger number of items, including psychosocial items. Algoplus showed good discriminant validity, with adequate internal consistency (Kuder-Richardson-20 = 0.712), excellent inter-rater reliability (intraclass coefficient = 0.812), and high sensitivity to change during specific pain situations (before and after care session for bedsores, patient transfer from bed to chair…) and after starting pain management. Excellent correlations were observed between Algoplus and experts’ clinical judgment, acute pain clinical situations, or each comparator’s self-rating pain score. For patients with acute pain conditions, a score of 2 out of 5 on the Algoplus scale [7] was retained as the threshold for the presence of acute pain in older patients, with 87% sensitivity and 80% specificity. In addition, the very brief rating time of one minute is particularly relevant in acute care settings, where repetitive pain monitoring is required. There is a need for reliable, valid, international, and intercultural tools for detecting acute pain in older persons with communication disorders in long-term settings and emergency situations [10,11]. Very few scales exist for acute pain evaluation, and international large scale studies would benefit from validated and similar scales. The aim of the current study was to examine the test-retest and inter-rater reliability of translations of the Algoplus scale into English, Italian, Portuguese, Spanish, and Turkish in a large multicultural sample of older persons with communication disorders. Methods This was a multicenter study of the translated version of the Algoplus scale in five languages. Measures Algoplus was used to assess pain. The scale consists of five items (facial expression, look, complaints, body position, atypical behavior); each item is scored “yes” = 1 or “no” = 0. The existence of one behavior in each item is enough to tick “yes.” Each item ticked “yes” gives one point, for a total maximal score of 5. The Algoplus threshold is fixed at 2 out of 5 and a score greater than or equal to 2 suggests the presence of acute pain and thereafter an initiation of analgesic treatment. It takes only about one minute to complete and has sufficient items to avoid missing idiosyncratic pain expressions that may be specific to pain in patients with communication disorders. Prior to the study, the Algoplus questionnaire was forward-translated from French to another language and then backward-translated to French [9,12]. Two independent bilingual health professionals (their mother tongue was the second language) translated the questionnaire independently into their native tongue. A reconciliation meeting was conducted to obtain a consensus version, and then two native French speakers who were blinded to the original version retranslated the new scale into French. The last step of the translation procedure was the pretesting of the translated instrument using a small sample of older persons with communication disorders with a final reconciliation session including all participants (Figure 1): translation in English (A) and Supplementary Figure 1; translation in Spanish (B); Italian (C); Portuguese (D); Turkish (E); and French (F) and Supplementary Figure 1 (available online). Figure 1 View largeDownload slide Algoplus translation in English (A). Figure 1 View largeDownload slide Algoplus translation in English (A). Apart from pain scores, demographic information (e.g., age, gender) was gathered from all the participants and recorded on a case report form. The cognitive status of all residents was evaluated using the Mini Mental State Examination (MMSE) available in each language [13]. This test ranges from 0 to 30 points and is widely used to screen for cognitive impairment. Efforts were made to have the MMSE assessment undertaken as close to the pain assessment as possible (on the very day up to the last three months). At the end of the study, the qualitative opinion of the participating physicians concerning the acceptability of the Algoplus scale was sought, with special regard to its ease of use. Satisfaction with the scale was evaluated with five items on a Likert scale (not, little, satisfied, very satisfied, excellent). Ethics Committee The protocol was approved by the French Ethics Committee and, when required, it was also approved by the local ethics committee, and patient or surrogate informed consent was obtained. Informed consent was obtained by a doctor not involved in the pain evaluation. This protocol did not change the normal care of the patients. Investigators Five teams were selected on the basis of expertise and competence in geriatrics and in pain evaluation of older patients with communication disorders. Each team tested the scale in the native language with at least 25 older persons. Patients Men and women age 65 years or older with communication disorders, including stroke patients, patients suffering from cognitive disorders or from any type of dementia, with or without suspected pain, were included. Any change in behavior (posture and movement, facial expression, prostration, loss of appetite, vocal complaint, psychological and behavioral disturbances) was also an inclusion criterion. Pain is always a potential cause of behavior changes, and even though pain might not be the most likely cause, it must always be considered as a potential cause in patients with communication disorders. The enrollment of patients with suspected pain was aimed at overcoming the misdiagnoses of pain in patients affected by dementia. Excluded from the protocol were patients who had acute pain needing immediate pharmacological pain management (patients with any major behavioral changes, obvious signs of distress and agitation, or complaining overtly), patients who had recently received analgesic treatment (within the previous four hours), or who refused to participate. Finally, we excluded patients who were agitated and could not be observed quietly for a few minutes by the physicians/nurses of the unit, to allow adequate scoring on the Algoplus scale. The level of agitation was assessed clinically (interference with and opposition to daily care) or by administration of the neuropsychiatric inventory scale (NPI with cut off > 40/120). Withdrawn from the protocol were any patients who had a change of treatment between the retest evaluations or a change of treatment between the evaluations of both practitioners, and any patients who refused to participate after having given informed consent. The decision to initiate analgesics or to administer a rescue dose was essentially based on a clinical evaluation indicating obvious pain and based on an Algoplus score of 2 or higher. Procedure Prior to the start of the study, the teams discussed the inclusion/exclusion criteria and the methodology. The teams had all been involved in the validation of the Doloplus scale in foreign languages [9] and knew the procedure. They were provided with instructions for the use of the Algoplus scale, the agreement of the French ethics committee, and the recommendation that only two physicians/nurses should be involved per team. These individuals had not been involved in the translation of the scale and did not know the French version of the scale before starting the clinical work. An important recommendation was that the physicians/nurses should get fully trained, accustomed to, and confident in the use of the scale in their own language before starting the study. Considering the heterogeneity of older patients, it was recommended that, before starting the study, the teams should allow themselves a few days to become familiar with the scale and to include it in the clinical routine of the ward. Teams were encouraged to assess eight to 10 older patients with communication disorders and to discuss the Algoplus scores obtained collectively between all raters. On the day of the study, pain assessment was implemented during the provision of the usual care in order to integrate it as much as possible to the customary routine of the unit. Selection of the patients took place on the days where both physicians/nurses were available and was done after the morning round by the team of the ward. At least 25 patients per team were required, and they were assessed twice by a trained physician or nurse (Rater 1) at initial contact (t0) and again four hours later (t4) without any intervening treatment, and the same patient was assessed once by a second physician at t0 (Rater 2). Observations were timed to avoid potentially painful experiences, such as movement or invasive nursing care. During the study, each physician in each country assessed the patients by observing them for two to three minutes prior to scoring the Algoplus scale. The raters were blind to each other’s ratings and undertook the observation and completion of the Algoplus scale in a sequential fashion. There was no contact between the raters about the scores they obtained and no access to each other’s scores, nor to their own previous score in the case of retest assessments (for Rater 1). The assessment was easier and quicker if the investigator was familiar with the patient, especially concerning facial expression and body movement. All physicians were familiar with the patients in our study and provided daily medical care. Performing the test took an average of one minute per patient. The study protocol did not interfere with the normal care of the patients. Statistical Analysis Statistical analysis was performed using Stata software, version 13 (StataCorp, College Station, TX, USA). The tests were two-sided, with a type I error set at α an alpha of 0.05. Continuous data were presented as the mean ± standard deviation or the median (interquartile range) according to statistical distribution (assumption of normality checked using normal probability plots and Shapiro-Wilk’s test). Comparisons between independent groups (for example, among Algoplus < or ≥2 or between countries) were analyzed using analysis of variance (ANOVA) or Kruskal-Wallis tests when conditions for ANOVA were not met (normality, and homoscedasticity verified by the Bartlett test) for quantitative variables. Comparisons concerning categorical data were performed using chi-square test or Fisher’s exact test. In addition to these analyses, the following psychometric properties of the Algoplus scale were explored: 1) Acceptability: Data quality was considered acceptable if more than 95% of the Algoplus was fully computable. 2) Internal consistency was determined through Kuder-Richardson’s alpha coefficient (minimum accepted value = 0.60); item difficulty (proportion of patients with an item or proportion of “right” answers noted p), item variance (p(1-p)), and item test correlations (corrected item test point-biserial correlation coefficients, also known as a discrimination index) were calculated. 3) Reproducibility: The weighted (quadratic weights) Kappa coefficient was used for each item to determine the test-retest and inter-rater reliability of the Algoplus. The Kappa statistic [14] was calculated for each item of the Algoplus scale. This is generally thought to be a more robust measure than a simple percent agreement calculation. A Kappa value higher than 0.81 typically indicates excellent inter-rater reliability; between 0.61 and 0.80, the test is good, and between 0.41 and 0.60 it is moderate [15]. For Algoplus total score, Lin’s concordance coefficient was calculated and presented with 95% confidence interval. Results Completion of the Scales Concerning acceptability, 95% of the Algoplus was fully computable. In the five teams, two physicians had a training assessment before undertaking all the evaluations of the study. The assessment scales were completed in 181 patients, with no missing data, a very good completion score compared with other validations (8% to 9%) [9,16]. There were no significant differences between the settings concerning missing data and withdrawals, and no patient refused to participate. The number of scales collected and analyzed per language was 30 English, 50 Italian, 44 Portuguese, 28 Spanish, and 29 Turkish. Demographical Data Demographical data are presented in Table 1 for each of the five languages. Sixty-six percent women and 34% men, age 83 ± 8 years, with an MMSE of 14 ± 8, were included by five teams in five countries and five languages: English, Italian, Portuguese, Spanish, and Turkish. There was no difference in Algoplus pain evaluation by gender (1.8 ±1.5 women and 2.0 ±1.5 male, P = 0.27) or by score: 50% lower than 2, 50% 2 or higher in women, and 39% lower than 2, 61% 2 or higher in men (P = 0.16). MMSE and the proportions of patients with dementia are presented in Table 1. The commonest item was facial expression, look, and body position (Figure 2A). Forty-six percent of patients had Algoplus scores of lower than 2, and 54% had Algoplus of 2 or higher (Figure 2, B and D). Portuguese and Turkish patients had more pain (Figure 2C). Table 1 Demographic characteristics of patients per language   English  Italian  Portuguese  Spanish  Turkish  N  30  49  40  28  29  Age, y            Mean (SD)  85 (8.6)  86 (7.8)  82 (9.1)  85 (7.0)  79 (6.9)  [min–max]  [64–98]  [67–99]  [65–98]  [65–97]  [65–92]  Gender            Female (%)  77  68  50  86  55  Male (%)  23  32  50  14  45  MMSE            Mean (SD)  8.4 (7.7)  17.5 (5.3)  19.1 (5.7)  13.2 (6.1)  12.0 (9.3)  [min–max]  [0–24]  [10–28]  [5–29]  [0–24]  [0–28]  0–9 (%)  69  0  6  33  41  10–23 (%)  24  86  75  53  41  ≥24 (%)  7  14  19  13  17    English  Italian  Portuguese  Spanish  Turkish  N  30  49  40  28  29  Age, y            Mean (SD)  85 (8.6)  86 (7.8)  82 (9.1)  85 (7.0)  79 (6.9)  [min–max]  [64–98]  [67–99]  [65–98]  [65–97]  [65–92]  Gender            Female (%)  77  68  50  86  55  Male (%)  23  32  50  14  45  MMSE            Mean (SD)  8.4 (7.7)  17.5 (5.3)  19.1 (5.7)  13.2 (6.1)  12.0 (9.3)  [min–max]  [0–24]  [10–28]  [5–29]  [0–24]  [0–28]  0–9 (%)  69  0  6  33  41  10–23 (%)  24  86  75  53  41  ≥24 (%)  7  14  19  13  17  Figure 2 View largeDownload slide A) Proportion of patients for Algoplus item. B) Number of patients with Algoplus evaluation (0 to 5). C) Proportion of patients for Algoplus ≥ or < 2 in each language. D) Proportion of patients with Algoplus score in each language. Figure 2 View largeDownload slide A) Proportion of patients for Algoplus item. B) Number of patients with Algoplus evaluation (0 to 5). C) Proportion of patients for Algoplus ≥ or < 2 in each language. D) Proportion of patients with Algoplus score in each language. Comparative analyses showed a significant (P = 0.02) age difference between the patients with Algoplus scores lower than 2 (85 ± 7 years, 46% of the population) and Algoplus of 2 or higher (82 ± 9 years, 54% of the population). There was no correlation between pain (Algoplus ≥ 2) and MMSE (Spearman rho -0.1, P = 0.13). Fifty percent of patients received a level 1 analgesic (paracetamol and nonsteroidal anti-inflammatory drugs), and 50% received level 2 or 3 (weak and strong opioids). Portuguese and Turkish patients received the largest amount of opiates (0% English, 40% Spanish, 46% Italian, 74% Portuguese, 75% Turkish). Internal Consistency Internal consistency was determined through Kuder-Richardson’s alpha coefficient (KD20). Global KR20s were 0.65 for the first rater first evaluation, 0.66 for the second rater first evaluation, and 0.68 for the first rater second evaluation. Item difficulty, item variance, and item-rest correlation are presented in Table 2. KR20s were 0.5 English, 0.65 Italian, 0.63 Portuguese, 0.81 Spanish, and 0.53 Turkish. Table 2 Internal consistency of the Algoplus scale for all languages     Item difficulty  Item variance  Item-rest correlation  Rater 1 t1  Facial expression  0.50  0.25  0.41  Look  0.45  0.25  0.34  Complaints  0.28  0.20  0.31  Body position  0.48  0.25  0.55  Atypical behavior  0.20  0.16  0.43  Rater 2 t1  Facial expression  0.49  0.25  0.53  Look  0.49  0.25  0.38  Complaints  0.27  0.20  0.32  Body position  0.43  0.24  0.45  Atypical behavior  0.25  0.19  0.39  Rater 1 t ± 4h  Facial expression  0.49  0.25  0.50  Look  0.43  0.25  0.40  Complaints  0.21  0.16  0.38  Body position  0.44  0.25  0.54  Atypical behavior  0.17  0.14  0.35      Item difficulty  Item variance  Item-rest correlation  Rater 1 t1  Facial expression  0.50  0.25  0.41  Look  0.45  0.25  0.34  Complaints  0.28  0.20  0.31  Body position  0.48  0.25  0.55  Atypical behavior  0.20  0.16  0.43  Rater 2 t1  Facial expression  0.49  0.25  0.53  Look  0.49  0.25  0.38  Complaints  0.27  0.20  0.32  Body position  0.43  0.24  0.45  Atypical behavior  0.25  0.19  0.39  Rater 1 t ± 4h  Facial expression  0.49  0.25  0.50  Look  0.43  0.25  0.40  Complaints  0.21  0.16  0.38  Body position  0.44  0.25  0.54  Atypical behavior  0.17  0.14  0.35  Reproducibility In test-retest reproducibility, for each of the five items, concordance proportions ranged from 84.5% to 92.3% and Kappa values from 0.69 to 0.84. In inter-rater reproducibility, for each of the five items, concordance proportions ranged from 81.8% to 95% and Kappa values from 0.69 to 0.84. The Algoplus 2 or higher test-retest and inter-rater concordance and Kappa gave high values (Table 3). Agreement (test-retest and inter-rater) and Kappa (test-retest and inter-rater) are presented in Figure 3 and in Table 4. Table 3 Reproducibility (test-retest and inter-rater) of all items of Algoplus scale in all languages   Test-retest   Inter-rater     % Concordance  Kappa  % Concordance  Kappa  Facial expression  84.5  0.69  81.8  0.64  Look  84.5  0.69  82.9  0.66  Complaints  88.4  0.68  95.0  0.87  Body position  90.1  0.80  89.5  0.79  Atypical behavior  92.2  0.75  91.1  0.75  Algoplus ≥ 2  92.3  0.84  91.2  0.82  Algoplus  0.88 [0.85–0.91]  0.91 [0.88–0.93]  (Lin concordance correlation coefficient)    Test-retest   Inter-rater     % Concordance  Kappa  % Concordance  Kappa  Facial expression  84.5  0.69  81.8  0.64  Look  84.5  0.69  82.9  0.66  Complaints  88.4  0.68  95.0  0.87  Body position  90.1  0.80  89.5  0.79  Atypical behavior  92.2  0.75  91.1  0.75  Algoplus ≥ 2  92.3  0.84  91.2  0.82  Algoplus  0.88 [0.85–0.91]  0.91 [0.88–0.93]  (Lin concordance correlation coefficient)  Table 4 Reproducibility (test-retest and inter-rater) of all items of Algoplus scale by languages   Test-retest   Inter-rater   English  % agreement  Kappa  % agreement  Kappa  Facial expression  70.0  0.35  53.3  0.10  Look  66.7  0.02  76.7  0.39  Complaints  76.7  0.38  93.3  0.81  Body position  83.3  0.67  80.0  0.59  Atypical behavior  90.0  0.35  86.7  0.53  Algoplus ≥ 2  83.3  0.67  73.33  0.46  Algoplus  0.65 [0.45–0.85]  0.69 [0.50–0.88]  (Lin concordance correlation coefficient)  Italian  Test-retest  Inter-rater    % agreement  Kappa  % agreement  Kappa  Facial expression  88.0  0.75  88.0  0,76  Look  86.0  0.67  78.0  0,51  Complaints  94.0  0.84  96.0  0,90  Body position  86.0  0.70  84.0  0,66  Atypical behavior  90.0  0.61  90.0  0,65  Algoplus ≥ 2  96.0  0.92  94.0  0,88  Algoplus  0.89 [0.83–0.95]  0.91 [0.86–0.96]  (Lin concordance correlation coefficient)  Portuguese  Test-retest  Inter-rater    % agreement  Kappa  % agreement  Kappa  Facial expression  79.6  0.58  84.1  0.68  Look  88.6  0.77  90.9  0.82  Complaints  79.6  0.54  93.2  0.86  Body position  88.6  0.77  90.9  0.82  Atypical behavior  90.5  0.78  90.7  0.78  Algoplus ≥ 2  88.6  0.77  97.7  0.95  Algoplus  0.85 [0.77–0.93]  0.95 [0.93–0.98]  (Lin concordance correlation coefficient)  Spanish  Test-retest  Inter-rater    % agreement  Kappa  % agreement  Kappa  Facial expression  92.9  0.76  85.7  0.51  Look  89.3  0.78  75.0  0.51  Complaints  92.9  0.76  92.9  0.76  Body position  100.0  1.00  100.0  1.00  Atypical behavior  100.0  1.00  96.4  0.87  Algoplus ≥ 2  100.0  1.00  92.9  0.83  Algoplus  0.96 [0.93–0.99]  0.90 [0.83–0.97]  (Lin concordance correlation coefficient)  Turkish  Test-retest  Inter-rater    % agreement  Kappa  % agreement  Kappa  Facial expression  93.1  0.46  93.1  0.46  Look  89.7  0.35  93.1  0.63  Complaints  100.0  1.00  100.0  1.00  Body position  96.6  0.89  96.6  0.90  Atypical behavior  93.1  0.83  93.1  0.85  Algoplus ≥ 2  93.1  0.46  93.1  0.46  Algoplus  0.84 [0.72–0.95]  0.89 [0.82–0.97]  (Lin concordance correlation coefficient)    Test-retest   Inter-rater   English  % agreement  Kappa  % agreement  Kappa  Facial expression  70.0  0.35  53.3  0.10  Look  66.7  0.02  76.7  0.39  Complaints  76.7  0.38  93.3  0.81  Body position  83.3  0.67  80.0  0.59  Atypical behavior  90.0  0.35  86.7  0.53  Algoplus ≥ 2  83.3  0.67  73.33  0.46  Algoplus  0.65 [0.45–0.85]  0.69 [0.50–0.88]  (Lin concordance correlation coefficient)  Italian  Test-retest  Inter-rater    % agreement  Kappa  % agreement  Kappa  Facial expression  88.0  0.75  88.0  0,76  Look  86.0  0.67  78.0  0,51  Complaints  94.0  0.84  96.0  0,90  Body position  86.0  0.70  84.0  0,66  Atypical behavior  90.0  0.61  90.0  0,65  Algoplus ≥ 2  96.0  0.92  94.0  0,88  Algoplus  0.89 [0.83–0.95]  0.91 [0.86–0.96]  (Lin concordance correlation coefficient)  Portuguese  Test-retest  Inter-rater    % agreement  Kappa  % agreement  Kappa  Facial expression  79.6  0.58  84.1  0.68  Look  88.6  0.77  90.9  0.82  Complaints  79.6  0.54  93.2  0.86  Body position  88.6  0.77  90.9  0.82  Atypical behavior  90.5  0.78  90.7  0.78  Algoplus ≥ 2  88.6  0.77  97.7  0.95  Algoplus  0.85 [0.77–0.93]  0.95 [0.93–0.98]  (Lin concordance correlation coefficient)  Spanish  Test-retest  Inter-rater    % agreement  Kappa  % agreement  Kappa  Facial expression  92.9  0.76  85.7  0.51  Look  89.3  0.78  75.0  0.51  Complaints  92.9  0.76  92.9  0.76  Body position  100.0  1.00  100.0  1.00  Atypical behavior  100.0  1.00  96.4  0.87  Algoplus ≥ 2  100.0  1.00  92.9  0.83  Algoplus  0.96 [0.93–0.99]  0.90 [0.83–0.97]  (Lin concordance correlation coefficient)  Turkish  Test-retest  Inter-rater    % agreement  Kappa  % agreement  Kappa  Facial expression  93.1  0.46  93.1  0.46  Look  89.7  0.35  93.1  0.63  Complaints  100.0  1.00  100.0  1.00  Body position  96.6  0.89  96.6  0.90  Atypical behavior  93.1  0.83  93.1  0.85  Algoplus ≥ 2  93.1  0.46  93.1  0.46  Algoplus  0.84 [0.72–0.95]  0.89 [0.82–0.97]  (Lin concordance correlation coefficient)  Figure 3 View largeDownload slide Kappa and agreement test-retest and inter-rater for all languages. IR = inter-rater; TR = test-retest. Figure 3 View largeDownload slide Kappa and agreement test-retest and inter-rater for all languages. IR = inter-rater; TR = test-retest. Satisfaction The teams were “very satisfied” with the use of the scale with 90% participants. Discussion The aim of the present study was to validate the translation of the Algoplus scale into five languages, with regard to test-retest and inter-rater reliability. It was tested in long-term care settings in geriatric populations (83.5 ± 8.3 years), slightly older than the population in the Algoplus initial validation (81.6 years old) [7]. Interrater and test-retest reliabilities were overall good to excellent. Results showed that reliability tests and correlations were good or excellent for the Italian, Portuguese, Spanish, and Turkish versions while the reliability correlations were moderate for the English scale. However, despite a low kappa value, explained by the low prevalence of these items, agreement ranged from 54% to 70% and was deemed satisfactory. The discrepancies in the kappa values of the items “look” and “facial expression” between the English and the other-language scales may be linked to a number of factors. It is interesting to note that the English language group had the most vulnerable population, with the lowest MMSE score (8.3/30), no patient with a maximal score of 5 (Figure 2D), while the other languages all returned maximal scores, and that a very high proportion of patients had dementia including Alzheimer’s disease (70%) (Table 1). The Algoplus scale was originally tested in older patients with communication disorders but its reliability was not specifically explored in dementia, vascular dementia, Alzheimer’s disease, or Lewy’s dementia, where pain expression and perception have been reported not to be a single entity [17]. Psychological or medical conditions other than pain, like anxiety or depression, may also complicate pain assessment as they are present in 25% of acute pain patients [18]. The items “look” and “facial expression” have been reported to be possibly overrated or underrated in older patients with an inability to communicate and who are not in pain [7] as expression of pain, dementia, and depression may coexist. Other factors may be linked to differences in the samples under study within each language group, but our sample was too small to permit relating the findings to the type of institution, the level of incapacity, or the cultural expression of pain. Ethnicity was not recorded in our study although this point is important for future studies as Algoplus, initially validated in Caucasians, may underestimate acute pain (especially with the “look” item) in an Asian sample of patients [19]. The present findings replicate some results obtained in the Doloplus validation in several languages [9]. Doloplus and Algoplus are complementary scales for chronic and acute pain evaluations, respectively. We observed here, as we did in the Doloplus study, that the lowest kappa values were obtained when the total scores were lowest and, in the case of Algoplus, when the score was lower than 2. Hence the Spanish and Turkish scales demonstrate highest mean pain scores with highest kappa values, while the English sample had the lowest total score and lowest kappa values. This would suggest that, like Doloplus, the Algoplus scale might display better reliability in cases where pain is present with a lower inter-rater agreement with lower pain scores or with more severe dementia. Internal consistency is globally good (KR20 coefficient > 0.60) for Rater 1 (T0 and T + 4h) and Rater 2 evaluations and is moderate for English and Turkish languages. It is very unlikely that these findings are linked to the translation itself as all five translations were similarly carried out using the same method. It might be linked to the small number of observations or training performed on a lower number of residents. It may also be linked to the type of pain and the occurrence of acute pain paroxysms that might have given inconsistent test-retest and inter-rater pain scores. Portuguese and Turkish patients had the highest pain scores and opioid prescriptions before starting the study. This observation may underscore the fact that pain may have been more fluctuant and variable in intensity in these groups, with sudden and unexpected occurrences of neuropathic characteristics (electric shocks, stabbing, etc.) that may be distressing for the patient when discrimination of pain is altered. This finding may also be linked to cultural differences in pain experience [19]. The fact that nociceptive pain may also hide neuropathic pain in patients with communication disorders has been highlighted recently in an algorithm of pain management in older patients with communication disorders [20]. Our study provides evidence that the Algoplus scale is a reliable tool, and the staff were very satisfied with its use. The validity of the scale must now be assessed in larger samples in order to undertake a full construct validation with specificity and sensibility of the scale in each language. Improvements of the psychometric properties of the scale in each language will also be sought as these scales are intended for use in vulnerable groups of older persons with communication disorders. Recording of anxiety and depression symptoms, and of dementia characteristics, should also be undertaken in order to differentiate pain from other types of emotion or distress. In conclusion, the study shows that the reliability testing of the translations of the Algoplus scale allows a unique opportunity to include this scale in daily pain assessment and in studies concerning older persons with communication disorders. Future studies should focus on enriching the validation of the scale in each language for optimization of pain management and initiation among the staff of a dynamic awareness of systematic pain assessment with a valid and easy-to-use tool. Acknowledgments We acknowledge Pedro Santos, João Raminhos, and Elisabete Passos for their participation in this study. Supplementary Data Supplementary Data may be found online at http://painmedicine.oxfordjournals.org. References 1 Defrin R, Amanzio M, de Tommaso M, et al.   Experimental pain processing in individuals with cognitive impairment: Current state of the science. Pain  2015; 156 8: 1396– 408. Google Scholar CrossRef Search ADS PubMed  2 Hadjistavropoulos T, Herr K, Turk DC, et al.   An interdisciplinary expert consensus statement on assessment of pain in older persons. Clin J Pain  2007; 23: S1– 43. Google Scholar CrossRef Search ADS PubMed  3 Hadjistavropoulos T, Herr K, Prkachin KM, et al.   Pain assessment in elderly adults with dementia. Lancet Neurol  2014; 13 12: 1216– 27. Google Scholar CrossRef Search ADS PubMed  4 Herr K, Coyne PJ, Key T, et al.   Pain assessment in the non verbal patient: Position statement with clinical practice recommendations. Pain Manag Nurs  2006; 7 2: 44– 52. Google Scholar CrossRef Search ADS PubMed  5 Van der Steen JT, Sampson EL, Van den Block L, et al.   EU-COST action td 1005 Collaborators. Tools to assess pain or lack of comfort in dementia: A content analysis. J Pain Symptom Manage  2015; 50 5: 659– 75. Google Scholar CrossRef Search ADS PubMed  6 Zwakhalen S, Hamers JP, Abu-Saad HH, Berger MP. Pain in elderly people with severe dementia: A systematic review of behavioural pain assessment tools. BMC Geriatr  2006; 27: 3– 17. Google Scholar CrossRef Search ADS   7 Rat P, Jouve E, Pickering G, Donnarel L, et al.   Validation of an acute pain-behavior scale for older persons with inability to communicate verbally: Algoplus. Eur J Pain  2010; 15: 1988 e1– 198, e10. 8 Lefevre-Chapiro S. T he Doloplus scale – evaluating pain in the elderly. Eur J Pall Care  2001; 8 5: 191– 3. 9 Pickering G, Gibson SJ, Serbouti S, et al.   Reliability study in five languages of the translation of the pain behavioural scale Doloplus. Eur J Pain  2010; 14 5: 545, e1– 10. Google Scholar CrossRef Search ADS   10 Fry M, Chenoweth L, Arendts G. Assessment and management of acute pain in the older person with cognitive impairment: A qualitative study. Int Emerg Nurs  2015; 24: 54– 60. Google Scholar CrossRef Search ADS PubMed  11 Hwang U, Belland LK, Handel DA, et al.   Is all pain is treated equally? A multicenter evaluation of acute pain care by age. Pain  2014; 155 12: 2568– 74. Google Scholar CrossRef Search ADS PubMed  12 Cull A, Sprangers M, Bjordal K, Aaronson N , on behalf of the EORTC Quality of Life Study Group. EORTC Quality of Life Study Group Translation Procedure . Brussels: EORTC; 1998. 13 Folstein MF, Folstein SE, McHugh PR. Mini-mental state: A practical method for grading the cognitive state of patients for the clinician. J Psychiatric Res  1975; 12: 189– 98. Google Scholar CrossRef Search ADS   14 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics  1977; 33 1: 159– 74. Google Scholar CrossRef Search ADS PubMed  15 Altman DG. Practical Statistics for Medical Research . London: Chapman and Hall; 1991. 16 Feldt KS. The checklist of nonverbal pain indicators. Pain Manag Nurs  2000; 1: 13– 21. Google Scholar CrossRef Search ADS PubMed  17 Scherder E, Oosterman J, Swaab D, et al.   Recent developments in pain in dementia. Brit Med J  2005; 330: 461– 4. Google Scholar CrossRef Search ADS PubMed  18 Schuler M, Njoo N, Hestermann M, Osler P, Hauer K. Acute and chronic pain in geriatrics: Clinical characteristics of pain and the influence of cognition. Pain Med  2004; 5: 253– 62. Google Scholar CrossRef Search ADS PubMed  19 Pickering ME, Bunna P, Rat P, et al.   Acute pain evaluation with Algoplus scale in Cambodian patients. Pain Med  2013; 14 12: 1971– 6. Google Scholar CrossRef Search ADS PubMed  20 Pickering G, Marcoux M, Chapiro S, et al.   An algorithm for neuropathic pain management in older persons. Int J Nursing Studies  2015; 310 22: 2443– 50. © 2017 American Academy of Pain Medicine. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Pain Medicine Oxford University Press

Loading next page...
 
/lp/ou_press/reliability-study-in-five-languages-of-the-translation-of-the-pain-mJCGeqzy0D
Publisher
Blackwell Publishing Inc.
Copyright
© 2017 American Academy of Pain Medicine. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com
ISSN
1526-2375
eISSN
1526-4637
D.O.I.
10.1093/pm/pnw356
Publisher site
See Article on Publisher Site

Abstract

Abstract Objective Acute pain is frequent and underestimated in older persons, especially when they suffer from cognitive impairment. Algoplus is an observational scale for acute pain evaluation, validated in French in older persons with communication disorders. We present the validation by an international expert team of the Algoplus scale in five languages: English, Spanish, Italian, Portuguese, and Turkish. Methods A total of 181 older consecutive patients were included in five countries (Spain, Australia, Italy, Portugal, and Turkey). Test-retest and inter-rater reliabilities were determined by weighted kappa coefficient for each item and internal consistency by Kuder-Richardson-20 (KD). Results Regarding test-retest reliability, the kappa coefficient for the five items ranged from 0.68 to 0.84. Inter-rater reliability kappa values ranged from 0.64 to 0.82. Internal consistency was indicated at a KD greater than 0.6. Satisfaction was good to excellent for all teams. Results show that reliability tests are good to excellent for all items of Algoplus. Conclusions This study shows evidence that Algoplus is an acceptable, reproducible, reliable, and easy-to-use tool. It provides a unique opportunity to include the translated Algoplus scale in daily assessment of older persons with communication disorders in many countries. Acute Pain, Cognitive Impairment, Evaluation Scale, Communication Disorders Introduction Self-reporting is the most reliable indicator of the presence and intensity of pain, even for patients with cognitive impairment [1–6]. When this is not possible, pain assessment observational scales are used. A number of observational pain assessment tools has been developed and validated [2,3] for older persons with communication disorders and moderate to severe dementia. The Algoplus scale has been validated in 349 patients with acute pain and communication disorders and is widely used in French-speaking countries [7]. It consists of five items: “facial expression,” “look,” “complaint,” “body position,” and “atypical behavior” and may complement the Doloplus scale [8,9] that has a larger number of items, including psychosocial items. Algoplus showed good discriminant validity, with adequate internal consistency (Kuder-Richardson-20 = 0.712), excellent inter-rater reliability (intraclass coefficient = 0.812), and high sensitivity to change during specific pain situations (before and after care session for bedsores, patient transfer from bed to chair…) and after starting pain management. Excellent correlations were observed between Algoplus and experts’ clinical judgment, acute pain clinical situations, or each comparator’s self-rating pain score. For patients with acute pain conditions, a score of 2 out of 5 on the Algoplus scale [7] was retained as the threshold for the presence of acute pain in older patients, with 87% sensitivity and 80% specificity. In addition, the very brief rating time of one minute is particularly relevant in acute care settings, where repetitive pain monitoring is required. There is a need for reliable, valid, international, and intercultural tools for detecting acute pain in older persons with communication disorders in long-term settings and emergency situations [10,11]. Very few scales exist for acute pain evaluation, and international large scale studies would benefit from validated and similar scales. The aim of the current study was to examine the test-retest and inter-rater reliability of translations of the Algoplus scale into English, Italian, Portuguese, Spanish, and Turkish in a large multicultural sample of older persons with communication disorders. Methods This was a multicenter study of the translated version of the Algoplus scale in five languages. Measures Algoplus was used to assess pain. The scale consists of five items (facial expression, look, complaints, body position, atypical behavior); each item is scored “yes” = 1 or “no” = 0. The existence of one behavior in each item is enough to tick “yes.” Each item ticked “yes” gives one point, for a total maximal score of 5. The Algoplus threshold is fixed at 2 out of 5 and a score greater than or equal to 2 suggests the presence of acute pain and thereafter an initiation of analgesic treatment. It takes only about one minute to complete and has sufficient items to avoid missing idiosyncratic pain expressions that may be specific to pain in patients with communication disorders. Prior to the study, the Algoplus questionnaire was forward-translated from French to another language and then backward-translated to French [9,12]. Two independent bilingual health professionals (their mother tongue was the second language) translated the questionnaire independently into their native tongue. A reconciliation meeting was conducted to obtain a consensus version, and then two native French speakers who were blinded to the original version retranslated the new scale into French. The last step of the translation procedure was the pretesting of the translated instrument using a small sample of older persons with communication disorders with a final reconciliation session including all participants (Figure 1): translation in English (A) and Supplementary Figure 1; translation in Spanish (B); Italian (C); Portuguese (D); Turkish (E); and French (F) and Supplementary Figure 1 (available online). Figure 1 View largeDownload slide Algoplus translation in English (A). Figure 1 View largeDownload slide Algoplus translation in English (A). Apart from pain scores, demographic information (e.g., age, gender) was gathered from all the participants and recorded on a case report form. The cognitive status of all residents was evaluated using the Mini Mental State Examination (MMSE) available in each language [13]. This test ranges from 0 to 30 points and is widely used to screen for cognitive impairment. Efforts were made to have the MMSE assessment undertaken as close to the pain assessment as possible (on the very day up to the last three months). At the end of the study, the qualitative opinion of the participating physicians concerning the acceptability of the Algoplus scale was sought, with special regard to its ease of use. Satisfaction with the scale was evaluated with five items on a Likert scale (not, little, satisfied, very satisfied, excellent). Ethics Committee The protocol was approved by the French Ethics Committee and, when required, it was also approved by the local ethics committee, and patient or surrogate informed consent was obtained. Informed consent was obtained by a doctor not involved in the pain evaluation. This protocol did not change the normal care of the patients. Investigators Five teams were selected on the basis of expertise and competence in geriatrics and in pain evaluation of older patients with communication disorders. Each team tested the scale in the native language with at least 25 older persons. Patients Men and women age 65 years or older with communication disorders, including stroke patients, patients suffering from cognitive disorders or from any type of dementia, with or without suspected pain, were included. Any change in behavior (posture and movement, facial expression, prostration, loss of appetite, vocal complaint, psychological and behavioral disturbances) was also an inclusion criterion. Pain is always a potential cause of behavior changes, and even though pain might not be the most likely cause, it must always be considered as a potential cause in patients with communication disorders. The enrollment of patients with suspected pain was aimed at overcoming the misdiagnoses of pain in patients affected by dementia. Excluded from the protocol were patients who had acute pain needing immediate pharmacological pain management (patients with any major behavioral changes, obvious signs of distress and agitation, or complaining overtly), patients who had recently received analgesic treatment (within the previous four hours), or who refused to participate. Finally, we excluded patients who were agitated and could not be observed quietly for a few minutes by the physicians/nurses of the unit, to allow adequate scoring on the Algoplus scale. The level of agitation was assessed clinically (interference with and opposition to daily care) or by administration of the neuropsychiatric inventory scale (NPI with cut off > 40/120). Withdrawn from the protocol were any patients who had a change of treatment between the retest evaluations or a change of treatment between the evaluations of both practitioners, and any patients who refused to participate after having given informed consent. The decision to initiate analgesics or to administer a rescue dose was essentially based on a clinical evaluation indicating obvious pain and based on an Algoplus score of 2 or higher. Procedure Prior to the start of the study, the teams discussed the inclusion/exclusion criteria and the methodology. The teams had all been involved in the validation of the Doloplus scale in foreign languages [9] and knew the procedure. They were provided with instructions for the use of the Algoplus scale, the agreement of the French ethics committee, and the recommendation that only two physicians/nurses should be involved per team. These individuals had not been involved in the translation of the scale and did not know the French version of the scale before starting the clinical work. An important recommendation was that the physicians/nurses should get fully trained, accustomed to, and confident in the use of the scale in their own language before starting the study. Considering the heterogeneity of older patients, it was recommended that, before starting the study, the teams should allow themselves a few days to become familiar with the scale and to include it in the clinical routine of the ward. Teams were encouraged to assess eight to 10 older patients with communication disorders and to discuss the Algoplus scores obtained collectively between all raters. On the day of the study, pain assessment was implemented during the provision of the usual care in order to integrate it as much as possible to the customary routine of the unit. Selection of the patients took place on the days where both physicians/nurses were available and was done after the morning round by the team of the ward. At least 25 patients per team were required, and they were assessed twice by a trained physician or nurse (Rater 1) at initial contact (t0) and again four hours later (t4) without any intervening treatment, and the same patient was assessed once by a second physician at t0 (Rater 2). Observations were timed to avoid potentially painful experiences, such as movement or invasive nursing care. During the study, each physician in each country assessed the patients by observing them for two to three minutes prior to scoring the Algoplus scale. The raters were blind to each other’s ratings and undertook the observation and completion of the Algoplus scale in a sequential fashion. There was no contact between the raters about the scores they obtained and no access to each other’s scores, nor to their own previous score in the case of retest assessments (for Rater 1). The assessment was easier and quicker if the investigator was familiar with the patient, especially concerning facial expression and body movement. All physicians were familiar with the patients in our study and provided daily medical care. Performing the test took an average of one minute per patient. The study protocol did not interfere with the normal care of the patients. Statistical Analysis Statistical analysis was performed using Stata software, version 13 (StataCorp, College Station, TX, USA). The tests were two-sided, with a type I error set at α an alpha of 0.05. Continuous data were presented as the mean ± standard deviation or the median (interquartile range) according to statistical distribution (assumption of normality checked using normal probability plots and Shapiro-Wilk’s test). Comparisons between independent groups (for example, among Algoplus < or ≥2 or between countries) were analyzed using analysis of variance (ANOVA) or Kruskal-Wallis tests when conditions for ANOVA were not met (normality, and homoscedasticity verified by the Bartlett test) for quantitative variables. Comparisons concerning categorical data were performed using chi-square test or Fisher’s exact test. In addition to these analyses, the following psychometric properties of the Algoplus scale were explored: 1) Acceptability: Data quality was considered acceptable if more than 95% of the Algoplus was fully computable. 2) Internal consistency was determined through Kuder-Richardson’s alpha coefficient (minimum accepted value = 0.60); item difficulty (proportion of patients with an item or proportion of “right” answers noted p), item variance (p(1-p)), and item test correlations (corrected item test point-biserial correlation coefficients, also known as a discrimination index) were calculated. 3) Reproducibility: The weighted (quadratic weights) Kappa coefficient was used for each item to determine the test-retest and inter-rater reliability of the Algoplus. The Kappa statistic [14] was calculated for each item of the Algoplus scale. This is generally thought to be a more robust measure than a simple percent agreement calculation. A Kappa value higher than 0.81 typically indicates excellent inter-rater reliability; between 0.61 and 0.80, the test is good, and between 0.41 and 0.60 it is moderate [15]. For Algoplus total score, Lin’s concordance coefficient was calculated and presented with 95% confidence interval. Results Completion of the Scales Concerning acceptability, 95% of the Algoplus was fully computable. In the five teams, two physicians had a training assessment before undertaking all the evaluations of the study. The assessment scales were completed in 181 patients, with no missing data, a very good completion score compared with other validations (8% to 9%) [9,16]. There were no significant differences between the settings concerning missing data and withdrawals, and no patient refused to participate. The number of scales collected and analyzed per language was 30 English, 50 Italian, 44 Portuguese, 28 Spanish, and 29 Turkish. Demographical Data Demographical data are presented in Table 1 for each of the five languages. Sixty-six percent women and 34% men, age 83 ± 8 years, with an MMSE of 14 ± 8, were included by five teams in five countries and five languages: English, Italian, Portuguese, Spanish, and Turkish. There was no difference in Algoplus pain evaluation by gender (1.8 ±1.5 women and 2.0 ±1.5 male, P = 0.27) or by score: 50% lower than 2, 50% 2 or higher in women, and 39% lower than 2, 61% 2 or higher in men (P = 0.16). MMSE and the proportions of patients with dementia are presented in Table 1. The commonest item was facial expression, look, and body position (Figure 2A). Forty-six percent of patients had Algoplus scores of lower than 2, and 54% had Algoplus of 2 or higher (Figure 2, B and D). Portuguese and Turkish patients had more pain (Figure 2C). Table 1 Demographic characteristics of patients per language   English  Italian  Portuguese  Spanish  Turkish  N  30  49  40  28  29  Age, y            Mean (SD)  85 (8.6)  86 (7.8)  82 (9.1)  85 (7.0)  79 (6.9)  [min–max]  [64–98]  [67–99]  [65–98]  [65–97]  [65–92]  Gender            Female (%)  77  68  50  86  55  Male (%)  23  32  50  14  45  MMSE            Mean (SD)  8.4 (7.7)  17.5 (5.3)  19.1 (5.7)  13.2 (6.1)  12.0 (9.3)  [min–max]  [0–24]  [10–28]  [5–29]  [0–24]  [0–28]  0–9 (%)  69  0  6  33  41  10–23 (%)  24  86  75  53  41  ≥24 (%)  7  14  19  13  17    English  Italian  Portuguese  Spanish  Turkish  N  30  49  40  28  29  Age, y            Mean (SD)  85 (8.6)  86 (7.8)  82 (9.1)  85 (7.0)  79 (6.9)  [min–max]  [64–98]  [67–99]  [65–98]  [65–97]  [65–92]  Gender            Female (%)  77  68  50  86  55  Male (%)  23  32  50  14  45  MMSE            Mean (SD)  8.4 (7.7)  17.5 (5.3)  19.1 (5.7)  13.2 (6.1)  12.0 (9.3)  [min–max]  [0–24]  [10–28]  [5–29]  [0–24]  [0–28]  0–9 (%)  69  0  6  33  41  10–23 (%)  24  86  75  53  41  ≥24 (%)  7  14  19  13  17  Figure 2 View largeDownload slide A) Proportion of patients for Algoplus item. B) Number of patients with Algoplus evaluation (0 to 5). C) Proportion of patients for Algoplus ≥ or < 2 in each language. D) Proportion of patients with Algoplus score in each language. Figure 2 View largeDownload slide A) Proportion of patients for Algoplus item. B) Number of patients with Algoplus evaluation (0 to 5). C) Proportion of patients for Algoplus ≥ or < 2 in each language. D) Proportion of patients with Algoplus score in each language. Comparative analyses showed a significant (P = 0.02) age difference between the patients with Algoplus scores lower than 2 (85 ± 7 years, 46% of the population) and Algoplus of 2 or higher (82 ± 9 years, 54% of the population). There was no correlation between pain (Algoplus ≥ 2) and MMSE (Spearman rho -0.1, P = 0.13). Fifty percent of patients received a level 1 analgesic (paracetamol and nonsteroidal anti-inflammatory drugs), and 50% received level 2 or 3 (weak and strong opioids). Portuguese and Turkish patients received the largest amount of opiates (0% English, 40% Spanish, 46% Italian, 74% Portuguese, 75% Turkish). Internal Consistency Internal consistency was determined through Kuder-Richardson’s alpha coefficient (KD20). Global KR20s were 0.65 for the first rater first evaluation, 0.66 for the second rater first evaluation, and 0.68 for the first rater second evaluation. Item difficulty, item variance, and item-rest correlation are presented in Table 2. KR20s were 0.5 English, 0.65 Italian, 0.63 Portuguese, 0.81 Spanish, and 0.53 Turkish. Table 2 Internal consistency of the Algoplus scale for all languages     Item difficulty  Item variance  Item-rest correlation  Rater 1 t1  Facial expression  0.50  0.25  0.41  Look  0.45  0.25  0.34  Complaints  0.28  0.20  0.31  Body position  0.48  0.25  0.55  Atypical behavior  0.20  0.16  0.43  Rater 2 t1  Facial expression  0.49  0.25  0.53  Look  0.49  0.25  0.38  Complaints  0.27  0.20  0.32  Body position  0.43  0.24  0.45  Atypical behavior  0.25  0.19  0.39  Rater 1 t ± 4h  Facial expression  0.49  0.25  0.50  Look  0.43  0.25  0.40  Complaints  0.21  0.16  0.38  Body position  0.44  0.25  0.54  Atypical behavior  0.17  0.14  0.35      Item difficulty  Item variance  Item-rest correlation  Rater 1 t1  Facial expression  0.50  0.25  0.41  Look  0.45  0.25  0.34  Complaints  0.28  0.20  0.31  Body position  0.48  0.25  0.55  Atypical behavior  0.20  0.16  0.43  Rater 2 t1  Facial expression  0.49  0.25  0.53  Look  0.49  0.25  0.38  Complaints  0.27  0.20  0.32  Body position  0.43  0.24  0.45  Atypical behavior  0.25  0.19  0.39  Rater 1 t ± 4h  Facial expression  0.49  0.25  0.50  Look  0.43  0.25  0.40  Complaints  0.21  0.16  0.38  Body position  0.44  0.25  0.54  Atypical behavior  0.17  0.14  0.35  Reproducibility In test-retest reproducibility, for each of the five items, concordance proportions ranged from 84.5% to 92.3% and Kappa values from 0.69 to 0.84. In inter-rater reproducibility, for each of the five items, concordance proportions ranged from 81.8% to 95% and Kappa values from 0.69 to 0.84. The Algoplus 2 or higher test-retest and inter-rater concordance and Kappa gave high values (Table 3). Agreement (test-retest and inter-rater) and Kappa (test-retest and inter-rater) are presented in Figure 3 and in Table 4. Table 3 Reproducibility (test-retest and inter-rater) of all items of Algoplus scale in all languages   Test-retest   Inter-rater     % Concordance  Kappa  % Concordance  Kappa  Facial expression  84.5  0.69  81.8  0.64  Look  84.5  0.69  82.9  0.66  Complaints  88.4  0.68  95.0  0.87  Body position  90.1  0.80  89.5  0.79  Atypical behavior  92.2  0.75  91.1  0.75  Algoplus ≥ 2  92.3  0.84  91.2  0.82  Algoplus  0.88 [0.85–0.91]  0.91 [0.88–0.93]  (Lin concordance correlation coefficient)    Test-retest   Inter-rater     % Concordance  Kappa  % Concordance  Kappa  Facial expression  84.5  0.69  81.8  0.64  Look  84.5  0.69  82.9  0.66  Complaints  88.4  0.68  95.0  0.87  Body position  90.1  0.80  89.5  0.79  Atypical behavior  92.2  0.75  91.1  0.75  Algoplus ≥ 2  92.3  0.84  91.2  0.82  Algoplus  0.88 [0.85–0.91]  0.91 [0.88–0.93]  (Lin concordance correlation coefficient)  Table 4 Reproducibility (test-retest and inter-rater) of all items of Algoplus scale by languages   Test-retest   Inter-rater   English  % agreement  Kappa  % agreement  Kappa  Facial expression  70.0  0.35  53.3  0.10  Look  66.7  0.02  76.7  0.39  Complaints  76.7  0.38  93.3  0.81  Body position  83.3  0.67  80.0  0.59  Atypical behavior  90.0  0.35  86.7  0.53  Algoplus ≥ 2  83.3  0.67  73.33  0.46  Algoplus  0.65 [0.45–0.85]  0.69 [0.50–0.88]  (Lin concordance correlation coefficient)  Italian  Test-retest  Inter-rater    % agreement  Kappa  % agreement  Kappa  Facial expression  88.0  0.75  88.0  0,76  Look  86.0  0.67  78.0  0,51  Complaints  94.0  0.84  96.0  0,90  Body position  86.0  0.70  84.0  0,66  Atypical behavior  90.0  0.61  90.0  0,65  Algoplus ≥ 2  96.0  0.92  94.0  0,88  Algoplus  0.89 [0.83–0.95]  0.91 [0.86–0.96]  (Lin concordance correlation coefficient)  Portuguese  Test-retest  Inter-rater    % agreement  Kappa  % agreement  Kappa  Facial expression  79.6  0.58  84.1  0.68  Look  88.6  0.77  90.9  0.82  Complaints  79.6  0.54  93.2  0.86  Body position  88.6  0.77  90.9  0.82  Atypical behavior  90.5  0.78  90.7  0.78  Algoplus ≥ 2  88.6  0.77  97.7  0.95  Algoplus  0.85 [0.77–0.93]  0.95 [0.93–0.98]  (Lin concordance correlation coefficient)  Spanish  Test-retest  Inter-rater    % agreement  Kappa  % agreement  Kappa  Facial expression  92.9  0.76  85.7  0.51  Look  89.3  0.78  75.0  0.51  Complaints  92.9  0.76  92.9  0.76  Body position  100.0  1.00  100.0  1.00  Atypical behavior  100.0  1.00  96.4  0.87  Algoplus ≥ 2  100.0  1.00  92.9  0.83  Algoplus  0.96 [0.93–0.99]  0.90 [0.83–0.97]  (Lin concordance correlation coefficient)  Turkish  Test-retest  Inter-rater    % agreement  Kappa  % agreement  Kappa  Facial expression  93.1  0.46  93.1  0.46  Look  89.7  0.35  93.1  0.63  Complaints  100.0  1.00  100.0  1.00  Body position  96.6  0.89  96.6  0.90  Atypical behavior  93.1  0.83  93.1  0.85  Algoplus ≥ 2  93.1  0.46  93.1  0.46  Algoplus  0.84 [0.72–0.95]  0.89 [0.82–0.97]  (Lin concordance correlation coefficient)    Test-retest   Inter-rater   English  % agreement  Kappa  % agreement  Kappa  Facial expression  70.0  0.35  53.3  0.10  Look  66.7  0.02  76.7  0.39  Complaints  76.7  0.38  93.3  0.81  Body position  83.3  0.67  80.0  0.59  Atypical behavior  90.0  0.35  86.7  0.53  Algoplus ≥ 2  83.3  0.67  73.33  0.46  Algoplus  0.65 [0.45–0.85]  0.69 [0.50–0.88]  (Lin concordance correlation coefficient)  Italian  Test-retest  Inter-rater    % agreement  Kappa  % agreement  Kappa  Facial expression  88.0  0.75  88.0  0,76  Look  86.0  0.67  78.0  0,51  Complaints  94.0  0.84  96.0  0,90  Body position  86.0  0.70  84.0  0,66  Atypical behavior  90.0  0.61  90.0  0,65  Algoplus ≥ 2  96.0  0.92  94.0  0,88  Algoplus  0.89 [0.83–0.95]  0.91 [0.86–0.96]  (Lin concordance correlation coefficient)  Portuguese  Test-retest  Inter-rater    % agreement  Kappa  % agreement  Kappa  Facial expression  79.6  0.58  84.1  0.68  Look  88.6  0.77  90.9  0.82  Complaints  79.6  0.54  93.2  0.86  Body position  88.6  0.77  90.9  0.82  Atypical behavior  90.5  0.78  90.7  0.78  Algoplus ≥ 2  88.6  0.77  97.7  0.95  Algoplus  0.85 [0.77–0.93]  0.95 [0.93–0.98]  (Lin concordance correlation coefficient)  Spanish  Test-retest  Inter-rater    % agreement  Kappa  % agreement  Kappa  Facial expression  92.9  0.76  85.7  0.51  Look  89.3  0.78  75.0  0.51  Complaints  92.9  0.76  92.9  0.76  Body position  100.0  1.00  100.0  1.00  Atypical behavior  100.0  1.00  96.4  0.87  Algoplus ≥ 2  100.0  1.00  92.9  0.83  Algoplus  0.96 [0.93–0.99]  0.90 [0.83–0.97]  (Lin concordance correlation coefficient)  Turkish  Test-retest  Inter-rater    % agreement  Kappa  % agreement  Kappa  Facial expression  93.1  0.46  93.1  0.46  Look  89.7  0.35  93.1  0.63  Complaints  100.0  1.00  100.0  1.00  Body position  96.6  0.89  96.6  0.90  Atypical behavior  93.1  0.83  93.1  0.85  Algoplus ≥ 2  93.1  0.46  93.1  0.46  Algoplus  0.84 [0.72–0.95]  0.89 [0.82–0.97]  (Lin concordance correlation coefficient)  Figure 3 View largeDownload slide Kappa and agreement test-retest and inter-rater for all languages. IR = inter-rater; TR = test-retest. Figure 3 View largeDownload slide Kappa and agreement test-retest and inter-rater for all languages. IR = inter-rater; TR = test-retest. Satisfaction The teams were “very satisfied” with the use of the scale with 90% participants. Discussion The aim of the present study was to validate the translation of the Algoplus scale into five languages, with regard to test-retest and inter-rater reliability. It was tested in long-term care settings in geriatric populations (83.5 ± 8.3 years), slightly older than the population in the Algoplus initial validation (81.6 years old) [7]. Interrater and test-retest reliabilities were overall good to excellent. Results showed that reliability tests and correlations were good or excellent for the Italian, Portuguese, Spanish, and Turkish versions while the reliability correlations were moderate for the English scale. However, despite a low kappa value, explained by the low prevalence of these items, agreement ranged from 54% to 70% and was deemed satisfactory. The discrepancies in the kappa values of the items “look” and “facial expression” between the English and the other-language scales may be linked to a number of factors. It is interesting to note that the English language group had the most vulnerable population, with the lowest MMSE score (8.3/30), no patient with a maximal score of 5 (Figure 2D), while the other languages all returned maximal scores, and that a very high proportion of patients had dementia including Alzheimer’s disease (70%) (Table 1). The Algoplus scale was originally tested in older patients with communication disorders but its reliability was not specifically explored in dementia, vascular dementia, Alzheimer’s disease, or Lewy’s dementia, where pain expression and perception have been reported not to be a single entity [17]. Psychological or medical conditions other than pain, like anxiety or depression, may also complicate pain assessment as they are present in 25% of acute pain patients [18]. The items “look” and “facial expression” have been reported to be possibly overrated or underrated in older patients with an inability to communicate and who are not in pain [7] as expression of pain, dementia, and depression may coexist. Other factors may be linked to differences in the samples under study within each language group, but our sample was too small to permit relating the findings to the type of institution, the level of incapacity, or the cultural expression of pain. Ethnicity was not recorded in our study although this point is important for future studies as Algoplus, initially validated in Caucasians, may underestimate acute pain (especially with the “look” item) in an Asian sample of patients [19]. The present findings replicate some results obtained in the Doloplus validation in several languages [9]. Doloplus and Algoplus are complementary scales for chronic and acute pain evaluations, respectively. We observed here, as we did in the Doloplus study, that the lowest kappa values were obtained when the total scores were lowest and, in the case of Algoplus, when the score was lower than 2. Hence the Spanish and Turkish scales demonstrate highest mean pain scores with highest kappa values, while the English sample had the lowest total score and lowest kappa values. This would suggest that, like Doloplus, the Algoplus scale might display better reliability in cases where pain is present with a lower inter-rater agreement with lower pain scores or with more severe dementia. Internal consistency is globally good (KR20 coefficient > 0.60) for Rater 1 (T0 and T + 4h) and Rater 2 evaluations and is moderate for English and Turkish languages. It is very unlikely that these findings are linked to the translation itself as all five translations were similarly carried out using the same method. It might be linked to the small number of observations or training performed on a lower number of residents. It may also be linked to the type of pain and the occurrence of acute pain paroxysms that might have given inconsistent test-retest and inter-rater pain scores. Portuguese and Turkish patients had the highest pain scores and opioid prescriptions before starting the study. This observation may underscore the fact that pain may have been more fluctuant and variable in intensity in these groups, with sudden and unexpected occurrences of neuropathic characteristics (electric shocks, stabbing, etc.) that may be distressing for the patient when discrimination of pain is altered. This finding may also be linked to cultural differences in pain experience [19]. The fact that nociceptive pain may also hide neuropathic pain in patients with communication disorders has been highlighted recently in an algorithm of pain management in older patients with communication disorders [20]. Our study provides evidence that the Algoplus scale is a reliable tool, and the staff were very satisfied with its use. The validity of the scale must now be assessed in larger samples in order to undertake a full construct validation with specificity and sensibility of the scale in each language. Improvements of the psychometric properties of the scale in each language will also be sought as these scales are intended for use in vulnerable groups of older persons with communication disorders. Recording of anxiety and depression symptoms, and of dementia characteristics, should also be undertaken in order to differentiate pain from other types of emotion or distress. In conclusion, the study shows that the reliability testing of the translations of the Algoplus scale allows a unique opportunity to include this scale in daily pain assessment and in studies concerning older persons with communication disorders. Future studies should focus on enriching the validation of the scale in each language for optimization of pain management and initiation among the staff of a dynamic awareness of systematic pain assessment with a valid and easy-to-use tool. Acknowledgments We acknowledge Pedro Santos, João Raminhos, and Elisabete Passos for their participation in this study. Supplementary Data Supplementary Data may be found online at http://painmedicine.oxfordjournals.org. References 1 Defrin R, Amanzio M, de Tommaso M, et al.   Experimental pain processing in individuals with cognitive impairment: Current state of the science. Pain  2015; 156 8: 1396– 408. Google Scholar CrossRef Search ADS PubMed  2 Hadjistavropoulos T, Herr K, Turk DC, et al.   An interdisciplinary expert consensus statement on assessment of pain in older persons. Clin J Pain  2007; 23: S1– 43. Google Scholar CrossRef Search ADS PubMed  3 Hadjistavropoulos T, Herr K, Prkachin KM, et al.   Pain assessment in elderly adults with dementia. Lancet Neurol  2014; 13 12: 1216– 27. Google Scholar CrossRef Search ADS PubMed  4 Herr K, Coyne PJ, Key T, et al.   Pain assessment in the non verbal patient: Position statement with clinical practice recommendations. Pain Manag Nurs  2006; 7 2: 44– 52. Google Scholar CrossRef Search ADS PubMed  5 Van der Steen JT, Sampson EL, Van den Block L, et al.   EU-COST action td 1005 Collaborators. Tools to assess pain or lack of comfort in dementia: A content analysis. J Pain Symptom Manage  2015; 50 5: 659– 75. Google Scholar CrossRef Search ADS PubMed  6 Zwakhalen S, Hamers JP, Abu-Saad HH, Berger MP. Pain in elderly people with severe dementia: A systematic review of behavioural pain assessment tools. BMC Geriatr  2006; 27: 3– 17. Google Scholar CrossRef Search ADS   7 Rat P, Jouve E, Pickering G, Donnarel L, et al.   Validation of an acute pain-behavior scale for older persons with inability to communicate verbally: Algoplus. Eur J Pain  2010; 15: 1988 e1– 198, e10. 8 Lefevre-Chapiro S. T he Doloplus scale – evaluating pain in the elderly. Eur J Pall Care  2001; 8 5: 191– 3. 9 Pickering G, Gibson SJ, Serbouti S, et al.   Reliability study in five languages of the translation of the pain behavioural scale Doloplus. Eur J Pain  2010; 14 5: 545, e1– 10. Google Scholar CrossRef Search ADS   10 Fry M, Chenoweth L, Arendts G. Assessment and management of acute pain in the older person with cognitive impairment: A qualitative study. Int Emerg Nurs  2015; 24: 54– 60. Google Scholar CrossRef Search ADS PubMed  11 Hwang U, Belland LK, Handel DA, et al.   Is all pain is treated equally? A multicenter evaluation of acute pain care by age. Pain  2014; 155 12: 2568– 74. Google Scholar CrossRef Search ADS PubMed  12 Cull A, Sprangers M, Bjordal K, Aaronson N , on behalf of the EORTC Quality of Life Study Group. EORTC Quality of Life Study Group Translation Procedure . Brussels: EORTC; 1998. 13 Folstein MF, Folstein SE, McHugh PR. Mini-mental state: A practical method for grading the cognitive state of patients for the clinician. J Psychiatric Res  1975; 12: 189– 98. Google Scholar CrossRef Search ADS   14 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics  1977; 33 1: 159– 74. Google Scholar CrossRef Search ADS PubMed  15 Altman DG. Practical Statistics for Medical Research . London: Chapman and Hall; 1991. 16 Feldt KS. The checklist of nonverbal pain indicators. Pain Manag Nurs  2000; 1: 13– 21. Google Scholar CrossRef Search ADS PubMed  17 Scherder E, Oosterman J, Swaab D, et al.   Recent developments in pain in dementia. Brit Med J  2005; 330: 461– 4. Google Scholar CrossRef Search ADS PubMed  18 Schuler M, Njoo N, Hestermann M, Osler P, Hauer K. Acute and chronic pain in geriatrics: Clinical characteristics of pain and the influence of cognition. Pain Med  2004; 5: 253– 62. Google Scholar CrossRef Search ADS PubMed  19 Pickering ME, Bunna P, Rat P, et al.   Acute pain evaluation with Algoplus scale in Cambodian patients. Pain Med  2013; 14 12: 1971– 6. Google Scholar CrossRef Search ADS PubMed  20 Pickering G, Marcoux M, Chapiro S, et al.   An algorithm for neuropathic pain management in older persons. Int J Nursing Studies  2015; 310 22: 2443– 50. © 2017 American Academy of Pain Medicine. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com

Journal

Pain MedicineOxford University Press

Published: Feb 1, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 12 million articles from more than
10,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Unlimited reading

Read as many articles as you need. Full articles with original layout, charts and figures. Read online, from anywhere.

Stay up to date

Keep up with your field with Personalized Recommendations and Follow Journals to get automatic updates.

Organize your research

It’s easy to organize your research with our built-in tools.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve Freelancer

DeepDyve Pro

Price
FREE
$49/month

$360/year
Save searches from Google Scholar, PubMed
Create lists to organize your research
Export lists, citations
Access to DeepDyve database
Abstract access only
Unlimited access to over
18 million full-text articles
Print
20 pages/month
PDF Discount
20% off