To Know Another’s Pain: A Meta-analysis of Caregivers’ and Healthcare Providers’ Pain Assessment Accuracy

To Know Another’s Pain: A Meta-analysis of Caregivers’ and Healthcare Providers’ Pain... Abstract Background Acute and chronic pain affects millions of adults yet it is often inadequately assessed and treated. Purpose The purpose of the present meta-analysis was to examine the overall level of pain assessment accuracy among caregivers and providers and identify patient, observer, and assessment level factors that moderate pain assessment accuracy. Methods A systematic literature search was conducted in PubMed and PsycINFO to identify studies addressing providers’ pain assessment accuracy, or studies that compared patients’ self-report of pain with observers’ (healthcare providers, caregivers, and strangers) assessment of pain. We present two separate meta-analyses examining the overall effect of under-/overestimation of pain and correlational pain assessment accuracy. Results Seventy-six articles meeting inclusion criteria yielded 94 independent effect sizes for the correlational accuracy meta-analysis. Ninety articles yielded 103 independent effect sizes for the paired comparison meta-analysis. The correlational pain assessment meta-analysis showed that in general, observers were significantly better than chance when assessing pain; however, the paired comparison meta-analysis showed that observers significantly underestimated patients’ pain. Patient’s age and gender, pain type, and provider type moderated these effects. Conclusions Results suggest that certain healthcare providers and caregivers need training to more accurately assess patient pain and that there are particular groups of patients who may be at a greater risk for having their pain inaccurately assessed. Pain assessment, Provider accuracy, Caregiver accuracy, Chronic pain, Acute pain Introduction Pain is a major public health challenge affecting millions of Americans and contributing to morbidity, mortality, disability, healthcare costs, and economic burdens [1]. The majority of acute and chronic pain sufferers have sought medical attention for their pain and chronic pain is one of the most frequent reasons for patients seeking primary care [2, 3]. Despite the number of patients who seek care, chronic pain often goes untreated or undertreated [4–6]. Acute pain is often not recognized and left untreated [7, 8]. In the emergency department, acute pain is often undertreated [8–12] or overmedicated, especially among vulnerable patient groups such as older adults [13, 14]. Untreated pain negatively impacts overall activity, mobility, relationships with others, ability to tolerate treatment, and enjoyment of life [15]. Pain is also associated with multiple comorbid symptoms such as sleep disturbance, fatigue, depression, and anxiety [16, 17]. In addition to the problem of undertreated and untreated pain, overtreatment is also problematic. There is a growing opioid crisis in the USA; the use of prescription opioids has increased substantially over the past decade and is one of the most commonly prescribed groups of medication for controlling pain [14]. Because of health and safety problems with use of opioids and addiction, various healthcare organizations have called for alternative pain relief [1]. In order to provide appropriate levels of pain treatment to avoid both over- and undertreatment of pain, providers and their caregivers must accurately assess patient pain. However, assessment of patient pain can be challenging for physicians and caregivers given that pain is an inherently subjective experience. The International Association for the Study of Pain defines pain as, “an unpleasant sensory and emotional experience that is associated with actual or potential tissue damage or described in such terms” [18, 19]. Pain is a complex experience that requires a multidimensional approach to treatment that takes into account not only the biological but also the social, psychological, and environmental impact on the pain experience and consequently how the experience of pain may impact subsequent health behaviors such as future health seeking [20, 21]. However, according to the Joint Commission on Accreditation of Healthcare Organization’s report on the Core Principles of Pain Assessment and Management, pain is always subjective and a patient’s self-report of pain is the single most reliable indicator of pain [22]. Because of the subjective nature of pain, it is more difficult for observers to understand the experience and respond accordingly compared to other symptoms or diseases that patients may feel and express [23]. There also appears to be an opportunity for improvement around pain assessment, communication, and management given the low levels of satisfaction reported by pain patients [4]. In one qualitative study, chronic pain patients reported feeling misunderstood by their providers, hurried in their appointments, disrespected, treated by providers as though their illness was not real, and labeled by their providers as “drug seekers” [24]. Likewise, physicians have to be aware of deceptive patients who seek medical care in order to obtain pharmacological treatments [25] and physicians and caregivers may become irritated by a patient’s repeated complaints and negative affect associated with pain [23]. As healthcare services increasingly move out of hospitals, the care of patients with pain is often provided with the help of family caregivers. Adequate management of symptoms is accomplished by establishing communication between patients, their caregivers, and an interdisciplinary team of healthcare providers [26]. Two Types of Pain Assessment Accuracy There are a number of ways researchers report accuracy of pain assessment and often multiple accuracy assessments are reported within the same research study. Two of the most common are (a) the correlation between patient-reported pain and observer-judged pain and (b) the paired comparison between patient-reported pain and observer-judged pain. In both cases, the criteria is always a patient’s reported pain as clinicians are encouraged to standardize pain assessment and treat pain assessment as the fifth vital sign [27]. Correlational accuracy Correlational accuracy measures the covariation between patient-reported pain and observer-judged pain and indicates how well patient-reported pain is associated with observer-judged pain. Assessing accuracy in this way does not indicate the direction of inaccuracy (i.e., either over or underestimation). With a correlation coefficient, one does not know how far apart the patient-reported and the observer-judged pain are or which is higher than the other, only whether they covary across pain reports. However, this is frequently how researchers compare patient and observer pain scores. One reason may be that pain ratings from patients and observers do not have to be on the same scale (e.g., some pain rating scales range from 0 to 10 or 0 to 100). A higher correlational accuracy means the observers can discriminate higher intensity levels of pain from lower intensity levels of pain. That is, the correlational approach results in a measure of generality, offering an ability to detect where a patient stands in relation to other patients on a given characteristic such as pain intensity or whether a given patient’s pain has changed over time. Paired comparison accuracy Paired comparison accuracy measures the direction (overestimates or underestimates) and to what degree observers accurately estimate a patient’s pain. The paired comparison method assesses accuracy by calculating the difference between each patient’s pain rating and observer’s pain rating. For this measure, the pain ratings made by patients and observers must be on the same scale and conceptually identical because they are compared directly by subtracting one from the other. Importantly, these two kinds of accuracy are statistically independent of each other and have distinct implications for patient care. A physician who has high correlational pain assessment accuracy may be able to distinguish which patients are in more pain than others. However, a physician may also have low paired comparison accuracy, which would mean they also have a tendency to systematically over or underestimate a patient’s pain. This provider may prescribe relatively correct doses of pain medication, but absolutely the wrong amount for all patients (i.e., no patient gets as much as they need or every patient is over-medicated). We need to look at both correlational and paired comparison accuracy to have a complete understanding of accuracy in patient pain assessment. Moderators of Pain Assessment Accuracy In addition to understanding overall correlational and paired comparison accuracy in patient pain assessment, there are a number of potential moderators of accuracy that may be clinically relevant. If we can identify patient populations or contexts in which patients are more likely to have their pain under or overestimated or less accurately assessed, we can better focus training and pain management interventions. A patient’s characteristics, the assessment method, and the observers have been shown in previous research to potentially moderate pain assessment accuracy. Patient characteristics such as gender and age [28] can impact accuracy. Different types of healthcare providers may vary greatly in their assessment of pain [29] and may differ from caregivers, such as parents and spouses [30, 31]. Characteristics of the pain and the pain assessment method are also important to explore. We know that both chronic and acute pain patients exhibit similar facial expressions of pain [32], but we do not know how this impacts provider or caregiver accuracy. Likewise, pain assessment scales may be highly correlated [33], but it is possible that characteristics of the scale may impact how accurately providers and caregivers assess patient pain. The potential moderators of pain assessment accuracy have not been systematically or quantitatively assessed for these two types of accuracy. Present Research Many individual studies have assessed the accuracy of pain assessments. A recent review qualitatively summarized pain assessment accuracy among healthcare providers [34]. The present study meta-analyzes pain assessment findings to provide an overall, quantitative and systematic evaluation of healthcare providers’ and caregivers’ accuracy in perceiving patient pain. Because the correlational and paired comparison accuracies are conceptually distinct, a meta-analysis across these studies is not possible. Therefore, we present two separate meta-analyses. Methods Definition of Pain Assessment Accuracy For the purpose of this meta-analysis, pain assessment accuracy was defined as the direct comparison between patients’ self-report of pain and observers’ assessment of pain. Pain assessment accuracy was reported in studies as a Pearson correlation coefficient (r), intraclass correlation coefficient (ICC), weighted kappa coefficient, means and standard deviations for the patient and observer, or a difference in the means (d). The patient population included any patients who self-reported pain, including children and older adults with dementia, when a self-report was present. The observer population included any observer including healthcare providers (i.e. physicians, nurses, midwives), informal caregivers (i.e., partners, family members), or other observers who viewed patients in pain. Search Strategy We performed a broad systematic literature search for peer-reviewed articles that contained the terms “pain assessment”, “judgments of pain”, “pain detection”, “pain”, and “pain intensity” combined with terms related to providers, caregivers, and patients (including “provider”, “physician”, “nurse”, “clinician”, “caregiver”, “partner”, “spouse”, and “patient”). The following databases were searched up to March 2016: PubMed (coverage 1946-present) and PsycINFO (coverage 1894-present). The reference lists of relevant studies and systematic reviews were investigated. We also reviewed the reference lists of all articles identified. Articles were included if they were published in peer-reviewed journals and based on patients experiencing some level of pain. Articles were excluded if, (a) they did not directly compare an observer’s judgment of pain with a patient’s verbal or written self-report of pain (the criteria) or if such a comparison could not be calculated based on information provided in the article, (b) observers had access to a patient’s self-reported pain prior to inferring that patient’s pain, or (c) patients were made-up vignettes or scenarios and not actually suffering from pain. Although not an exclusion criteria, no non-English language publications satisfied the inclusion criteria. See Fig. 1 for PRISMA diagram. Fig. 1. View largeDownload slide Flow chart to illustrate the process by which articles were selected or rejected for inclusion in the meta-analyses. Fig. 1. View largeDownload slide Flow chart to illustrate the process by which articles were selected or rejected for inclusion in the meta-analyses. Coding of Potential Moderators A trained research assistant coded all moderators with the first and second author reviewing all data for accuracy and resolving any discrepancies through consensus discussion. Potential moderators coded were number of patients, number of observers, patient age group (children, adults, older adults), patient and observer gender, acute or chronic pain, pain assessment instrument, timing of pain assessment (during or after the pain experience), type of provider and years of clinical experience (for providers only), location of research study, and publication year (coded both as a continuous variable and as a dichotomous variable looking before and after 1996, when the American Pain Society encouraged clinicians to standardize pain assessment and treat pain assessment as the fifth vital sign [27]) (see Table A1 for available moderators in each study). We coded additional potential moderators (e.g., patient and observer race/ethnicity); however, we were unable to include these in our analysis because they were insufficiently reported in the original studies. Effect Size Extraction and Analyses Articles that reported pain assessment accuracy in more than one way (correlation, paired comparison, independent means comparison) are included in more than one meta-analysis. However, within the individual meta-analyses all effect sizes come from independent samples. When multiple pain assessment comparisons were reported for the same group of observers, we averaged across these comparisons to maintain independence of effect sizes. For the correlational meta-analysis, the unit of analysis is the observer. The effect size was calculated using the number of patients if the number of observers was not provided. For the paired comparison meta-analysis, the unit of analysis was the number of ratings or comparisons made or the number of patients in the study. When the patients and observers used different pain scales, we standardized the scales for inclusion in the paired comparison meta-analysis (this only occurred in one study). The effect size used for the correlational meta-analysis was the Pearson correlation coefficient (r) while the standard difference of the means (d) was used for the paired comparison meta-analysis. For studies that used the correlational method, results were already expressed in this metric or the studies’ test statistics were converted to r using standard formulas [35]. All correlational effect sizes were converted using Fisher’s r-to-z transformation (rz) to normalize the data. Effect sizes were averaged to create an unweighted mean effect size across studies. In addition, individual effect sizes were weighted by sample size, averaged to create a weighted mean effect size across studies, and a combined Z was calculated as an indicator of the statistical significance of the set of studies [35]. For significant Zs, a “file drawer N”, an estimate of the number of additional studies with effect sizes averaging r = .00 necessary to bring the combined Z to a nonsignificant level, was calculated. This number takes into account potentially unpublished studies with nonsignificant findings [36]. Cochrane’s Q test for homogeneity was performed to see if effect sizes within the meta-analysis differed from each other more than could be expected due to sampling error. Contrast analyses were computed to examine the impact of moderators on pain assessment accuracy (meta-regression for continuous variables and meta-analysis of variance (ANOVA) for dichotomous variables). Fixed effects comparisons compared different levels of the coded moderators. All analyses were facilitated by Comprehensive Meta-Analysis software [37]. Results Pain Assessment Accuracy: Correlation Seventy-six articles meeting inclusion criteria yielded 94 independent effect sizes for the correlational accuracy meta-analysis (Table 1). The random effects mean effect size was r = .46 (95% CI: 0.41, 0.50) and the fixed, weighted mean effect size was r = .45 (95% CI: 0.44, 0.47) (Table 2). The significant combined Z = 42.27 (p < .001) indicates that observers were significantly better than chance (chance r = 0) at assessing patient pain. It would take 31,427 effect sizes (334.3 missing studies for each study included in the meta-analysis) to nullify the effects. The effect sizes were significantly heterogeneous so moderator analyses were conducted. Table 1 Characteristics and effect sizes of studies included in pain assessment correlational accuracy meta-analysis First author  Study year  Observer type  Observer N  Assessment method  Correlation effect size (r)§  95% CI Lower limit  95% CI Upper limit  Akin [56]  2013  Caregivers  119  NRS/Likert  .76***  0.67  0.83    2013  Nurses  7  NRS/Likert  .40  −0.51  0.89  Allen [30]  2002  Caregivers  176  NRS/Likert  .28***  0.14  0.41  Baxt [31]  2004  Caregivers  276  Faces  .43***  0.33  0.52  Bergh [57]  1999  Nurses  39  VAS  .39**  0.08  0.63  Blomqvist [58]  1999  Nurses  29  NRS/Likert  .63***  0.35  0.81  Broberger [59]  2005  Caregivers  54  VAS  .50***  0.27  0.68    2005  Nurses  29  VAS  .41*  0.05  0.68  Chambers [60]  1998  Caregivers  110  Faces  .72***  0.62  0.80  Choiniere [61]  1990  Nurses  82  VAS  .47***  0.28  0.62  Clipp [62]  1992  Caregivers  30  Other  .46**  0.12  0.70  Colwell [63]  1996  Nurses  44  NRS/Likert  .70***  0.51  0.83  Cremeans-Smith [64]  2003  Caregivers  114  NRS/Likert  .42***  0.26  0.56    2003  Physicians  5  NRS/Likert  .08  −0.86  0.90  Dawber [65]  2016  Caregivers  50  Other  .58***  0.36  0.74    2016  Nurses  50  Other  .31*  0.03  0.54  de Bock [66]  1994  Physicians  40  VAS  .36*  0.05  0.60  Doherty [67]  1993  Caregivers  20  VAS  .45*  0.01  0.74  Drayer [68]  1999  Other healthcare  45  VAS  .36*  0.07  0.59  Everett [69]  1994  Nurses  27  VAS  .70***  0.44  0.85  Forrest [70]  1989  Physicians  8  VAS  .64  −0.12  0.93  Fridh [71]  1990  Nurses  12  VAS  .29  −0.34  0.74  Gil [72]  2004  Caregivers  35  NRS/Likert  .50**  0.20  0.71  Harrison [73]  1993  Nurses  199  VAS  .34***  0.22  0.46  Hays [74]  1995  Caregivers  304  NRS/Likert  .56***  0.48  0.63  Heikkinen [75]  2005  Nurses  45  Other  .85***  0.75  0.92  Heuss [76]  2012  Nurses  12  VAS  .35  −0.28  0.77    2012  Physicians  9  VAS  .37  −0.39  0.83  Higginson [77]  2008  Caregivers  64  VAS  .69  0.53  0.80  Hodgkins [78]  1985  Physicians  21  VAS  .62**  0.26  0.83  Horgas [79]  2001  Nurses  16  Other  −.25  −0.66  0.28  Idvall [80]  2005  Nurses  267  NRS/Likert  .58***  0.49  0.65  Kelly [81]  2002  Caregivers  78  VAS  .63***  0.47  0.75  Kristjanson [82]  1998  Caregivers  78  Other  .44***  0.24  0.60  Lautenbacher [28]  2013  Nurses  21  NRS/Likert  .10***  −0.35  0.51    2013  Other observers  21  NRS/Likert  .07  −0.37  0.49  Lin [83]  2001  Caregivers  89  NRS/Likert  .58***  0.42  0.70  Lobchuk [84]  1997  Caregivers  37  Other  .31  −0.02  0.58  Lobchuk [20]  2002  Caregivers  98  NRS/Likert  .69***  0.57  0.78  Madison [85]  1995  Caregivers  18  VAS  .49*  0.03  0.78  Maguire [86]  2014  Physicians  200  VAS  .16*  0.02  0.29  Manne [87]  1992  Caregivers  85  Faces  .32**  0.11  0.50    1992  Nurses  9  Faces  .63  −0.06  0.91  Mantyselka [5]  2001  Physicians  28  NRS/Likert  .29  −0.09  0.60  McKinley [88]  1991  Nurses  97  VAS  .35***  0.16  0.51  McMillan [89]  2003  Caregivers  264  NRS/Likert  .40***  0.29  0.50  McPherson [90]  2008  Caregivers  66  NRS/Likert  .44***  0.22  0.62  Miller [91]  1996  Caregivers  20  VAS  .65***  0.29  0.85    1996  Nurses  20  VAS  .39  −0.06  0.71  O’Brien [92]  1988  Caregivers  42  VAS  .23  −0.08  0.50  Oi-Ling [93]  2005  Caregivers  30  NRS/Likert  .38*  0.02  0.65  Paice [94]  1991  Nurses  30  NRS/Likert  .06  −0.31  0.41    1991  Physicians  30  NRS/Likert  .12  −0.25  0.46  Perreault [95]  2006  Other healthcare  9  NRS/Likert  .55  −0.18  0.89  Powers [96]  1987  Nurses  33  VAS  .32  −0.03  0.60  Prkachin [97]  1994  Other observers  5  NRS/Likert  .37  −0.76  0.94  Prkachin [98]  2001  Other observers  82  NRS/Likert  .36***  0.16  0.54    2001  Physicians  34  Other  .36*  0.02  0.62  Redinbaugh [99]  2002  Caregivers  31  NRS/Likert  .55***  0.24  0.76  Resnizky [100]  2006  Caregivers  143  Other  .51***  0.38  0.62  Rhondali [101]  2012  Nurses  20  NRS/Likert  .66***  0.31  0.85  Ruben [7]  2013  Other observers  262  VAS  .10  −0.02  0.22  Ruben [102]  2016  Other observers  95  NRS/Likert  .12  −0.08  0.31  Salmon [103]  1996  Nurses  15  VAS  .62**  0.16  0.86  Schneider [104]  1992  Caregivers  40  Other  .72***  0.53  0.84    1992  Nurses  40  Other  .60***  0.35  0.77  Shega [105]  2004  Caregivers  115  Other  .12  −0.06  0.30  Silveira [106]  2010  Caregivers  142  NRS/Likert  .48***  0.34  0.60  Singer [107]  1999  Physicians  1171  Faces  .47***  0.42  0.51  Singer [108]  2002  Caregivers  57  Faces  .47***  0.24  0.65    2002  Other healthcare  57  VAS  .08  −0.18  0.33  Sjöström [109]  1997  Nurses  90  VAS  .55***  0.39  0.68    1997  Physicians  90  VAS  .68***  0.55  0.78  Sneeuw [110]  1997  Caregivers  103  NRS/Likert  .23*  0.04  0.41  Sneeuw [111]  1998  Caregivers  307  VAS  .63***  0.56  0.69  Sneeuw [112]  1999  Caregivers  90  Faces  .64***  0.50  0.75    1999  Nurses  35  Faces  .66***  0.42  0.81    1999  Physicians  15  Faces  .50  −0.02  0.81  Stephenson [113]  1994  Nurses  11  VAS  .59  −0.02  0.88  St-Laurent-Gagnon [114]  1999  Caregivers  104  Faces  .76***  0.67  0.83  Suarez-Almazor [115]  2001  Physicians  5  VAS  .42  −0.73  0.95  Sutherland [116]  1988  Physicians  22  VAS  .66***  0.33  0.85  van Herk [117]  2009  Caregivers  122  NRS/Likert  .28**  0.11  0.44    2009  Nurses  171  NRS/Likert  .20**  0.05  0.34  Van Der Does [118]  1989  Nurses  30  VAS  .39*  0.03  0.66  Vervoort [119]  2009  Caregivers  62  NRS/Likert  .16  −0.09  0.39  Walkenstein [120]  1982  Nurses  8  Other  .44  −0.38  0.87  Weiner [121]  1999  Caregivers  42  NRS/Likert  −.19  −0.47  0.12    1999  Nurses  42  NRS/Likert  .34*  0.04  0.58  Wennman-Larsen [122]  2007  Caregivers  54  NRS/Likert  .64***  0.45  0.77  Yesilbalkan [123]  2010  Caregivers  80  NRS/Likert  .40***  0.20  0.57  Zalon [124]  1993  Nurses  119  VAS  .30***  0.13  0.46  Zhukovsky [125]  2015  Caregivers  60  Other  .40***  0.16  0.59    2015  Physicians  14  Other  .70**  0.27  0.90  First author  Study year  Observer type  Observer N  Assessment method  Correlation effect size (r)§  95% CI Lower limit  95% CI Upper limit  Akin [56]  2013  Caregivers  119  NRS/Likert  .76***  0.67  0.83    2013  Nurses  7  NRS/Likert  .40  −0.51  0.89  Allen [30]  2002  Caregivers  176  NRS/Likert  .28***  0.14  0.41  Baxt [31]  2004  Caregivers  276  Faces  .43***  0.33  0.52  Bergh [57]  1999  Nurses  39  VAS  .39**  0.08  0.63  Blomqvist [58]  1999  Nurses  29  NRS/Likert  .63***  0.35  0.81  Broberger [59]  2005  Caregivers  54  VAS  .50***  0.27  0.68    2005  Nurses  29  VAS  .41*  0.05  0.68  Chambers [60]  1998  Caregivers  110  Faces  .72***  0.62  0.80  Choiniere [61]  1990  Nurses  82  VAS  .47***  0.28  0.62  Clipp [62]  1992  Caregivers  30  Other  .46**  0.12  0.70  Colwell [63]  1996  Nurses  44  NRS/Likert  .70***  0.51  0.83  Cremeans-Smith [64]  2003  Caregivers  114  NRS/Likert  .42***  0.26  0.56    2003  Physicians  5  NRS/Likert  .08  −0.86  0.90  Dawber [65]  2016  Caregivers  50  Other  .58***  0.36  0.74    2016  Nurses  50  Other  .31*  0.03  0.54  de Bock [66]  1994  Physicians  40  VAS  .36*  0.05  0.60  Doherty [67]  1993  Caregivers  20  VAS  .45*  0.01  0.74  Drayer [68]  1999  Other healthcare  45  VAS  .36*  0.07  0.59  Everett [69]  1994  Nurses  27  VAS  .70***  0.44  0.85  Forrest [70]  1989  Physicians  8  VAS  .64  −0.12  0.93  Fridh [71]  1990  Nurses  12  VAS  .29  −0.34  0.74  Gil [72]  2004  Caregivers  35  NRS/Likert  .50**  0.20  0.71  Harrison [73]  1993  Nurses  199  VAS  .34***  0.22  0.46  Hays [74]  1995  Caregivers  304  NRS/Likert  .56***  0.48  0.63  Heikkinen [75]  2005  Nurses  45  Other  .85***  0.75  0.92  Heuss [76]  2012  Nurses  12  VAS  .35  −0.28  0.77    2012  Physicians  9  VAS  .37  −0.39  0.83  Higginson [77]  2008  Caregivers  64  VAS  .69  0.53  0.80  Hodgkins [78]  1985  Physicians  21  VAS  .62**  0.26  0.83  Horgas [79]  2001  Nurses  16  Other  −.25  −0.66  0.28  Idvall [80]  2005  Nurses  267  NRS/Likert  .58***  0.49  0.65  Kelly [81]  2002  Caregivers  78  VAS  .63***  0.47  0.75  Kristjanson [82]  1998  Caregivers  78  Other  .44***  0.24  0.60  Lautenbacher [28]  2013  Nurses  21  NRS/Likert  .10***  −0.35  0.51    2013  Other observers  21  NRS/Likert  .07  −0.37  0.49  Lin [83]  2001  Caregivers  89  NRS/Likert  .58***  0.42  0.70  Lobchuk [84]  1997  Caregivers  37  Other  .31  −0.02  0.58  Lobchuk [20]  2002  Caregivers  98  NRS/Likert  .69***  0.57  0.78  Madison [85]  1995  Caregivers  18  VAS  .49*  0.03  0.78  Maguire [86]  2014  Physicians  200  VAS  .16*  0.02  0.29  Manne [87]  1992  Caregivers  85  Faces  .32**  0.11  0.50    1992  Nurses  9  Faces  .63  −0.06  0.91  Mantyselka [5]  2001  Physicians  28  NRS/Likert  .29  −0.09  0.60  McKinley [88]  1991  Nurses  97  VAS  .35***  0.16  0.51  McMillan [89]  2003  Caregivers  264  NRS/Likert  .40***  0.29  0.50  McPherson [90]  2008  Caregivers  66  NRS/Likert  .44***  0.22  0.62  Miller [91]  1996  Caregivers  20  VAS  .65***  0.29  0.85    1996  Nurses  20  VAS  .39  −0.06  0.71  O’Brien [92]  1988  Caregivers  42  VAS  .23  −0.08  0.50  Oi-Ling [93]  2005  Caregivers  30  NRS/Likert  .38*  0.02  0.65  Paice [94]  1991  Nurses  30  NRS/Likert  .06  −0.31  0.41    1991  Physicians  30  NRS/Likert  .12  −0.25  0.46  Perreault [95]  2006  Other healthcare  9  NRS/Likert  .55  −0.18  0.89  Powers [96]  1987  Nurses  33  VAS  .32  −0.03  0.60  Prkachin [97]  1994  Other observers  5  NRS/Likert  .37  −0.76  0.94  Prkachin [98]  2001  Other observers  82  NRS/Likert  .36***  0.16  0.54    2001  Physicians  34  Other  .36*  0.02  0.62  Redinbaugh [99]  2002  Caregivers  31  NRS/Likert  .55***  0.24  0.76  Resnizky [100]  2006  Caregivers  143  Other  .51***  0.38  0.62  Rhondali [101]  2012  Nurses  20  NRS/Likert  .66***  0.31  0.85  Ruben [7]  2013  Other observers  262  VAS  .10  −0.02  0.22  Ruben [102]  2016  Other observers  95  NRS/Likert  .12  −0.08  0.31  Salmon [103]  1996  Nurses  15  VAS  .62**  0.16  0.86  Schneider [104]  1992  Caregivers  40  Other  .72***  0.53  0.84    1992  Nurses  40  Other  .60***  0.35  0.77  Shega [105]  2004  Caregivers  115  Other  .12  −0.06  0.30  Silveira [106]  2010  Caregivers  142  NRS/Likert  .48***  0.34  0.60  Singer [107]  1999  Physicians  1171  Faces  .47***  0.42  0.51  Singer [108]  2002  Caregivers  57  Faces  .47***  0.24  0.65    2002  Other healthcare  57  VAS  .08  −0.18  0.33  Sjöström [109]  1997  Nurses  90  VAS  .55***  0.39  0.68    1997  Physicians  90  VAS  .68***  0.55  0.78  Sneeuw [110]  1997  Caregivers  103  NRS/Likert  .23*  0.04  0.41  Sneeuw [111]  1998  Caregivers  307  VAS  .63***  0.56  0.69  Sneeuw [112]  1999  Caregivers  90  Faces  .64***  0.50  0.75    1999  Nurses  35  Faces  .66***  0.42  0.81    1999  Physicians  15  Faces  .50  −0.02  0.81  Stephenson [113]  1994  Nurses  11  VAS  .59  −0.02  0.88  St-Laurent-Gagnon [114]  1999  Caregivers  104  Faces  .76***  0.67  0.83  Suarez-Almazor [115]  2001  Physicians  5  VAS  .42  −0.73  0.95  Sutherland [116]  1988  Physicians  22  VAS  .66***  0.33  0.85  van Herk [117]  2009  Caregivers  122  NRS/Likert  .28**  0.11  0.44    2009  Nurses  171  NRS/Likert  .20**  0.05  0.34  Van Der Does [118]  1989  Nurses  30  VAS  .39*  0.03  0.66  Vervoort [119]  2009  Caregivers  62  NRS/Likert  .16  −0.09  0.39  Walkenstein [120]  1982  Nurses  8  Other  .44  −0.38  0.87  Weiner [121]  1999  Caregivers  42  NRS/Likert  −.19  −0.47  0.12    1999  Nurses  42  NRS/Likert  .34*  0.04  0.58  Wennman-Larsen [122]  2007  Caregivers  54  NRS/Likert  .64***  0.45  0.77  Yesilbalkan [123]  2010  Caregivers  80  NRS/Likert  .40***  0.20  0.57  Zalon [124]  1993  Nurses  119  VAS  .30***  0.13  0.46  Zhukovsky [125]  2015  Caregivers  60  Other  .40***  0.16  0.59    2015  Physicians  14  Other  .70**  0.27  0.90  CI confidence interval; NRS numeric rating scale; VAS visual analogue scale. Faces is Faces Pain Scale. §t-statistic converted to r using formula r = t2(t2+df) *p < .05; **p < .01; ***p < .001. View Large Table 1 Characteristics and effect sizes of studies included in pain assessment correlational accuracy meta-analysis First author  Study year  Observer type  Observer N  Assessment method  Correlation effect size (r)§  95% CI Lower limit  95% CI Upper limit  Akin [56]  2013  Caregivers  119  NRS/Likert  .76***  0.67  0.83    2013  Nurses  7  NRS/Likert  .40  −0.51  0.89  Allen [30]  2002  Caregivers  176  NRS/Likert  .28***  0.14  0.41  Baxt [31]  2004  Caregivers  276  Faces  .43***  0.33  0.52  Bergh [57]  1999  Nurses  39  VAS  .39**  0.08  0.63  Blomqvist [58]  1999  Nurses  29  NRS/Likert  .63***  0.35  0.81  Broberger [59]  2005  Caregivers  54  VAS  .50***  0.27  0.68    2005  Nurses  29  VAS  .41*  0.05  0.68  Chambers [60]  1998  Caregivers  110  Faces  .72***  0.62  0.80  Choiniere [61]  1990  Nurses  82  VAS  .47***  0.28  0.62  Clipp [62]  1992  Caregivers  30  Other  .46**  0.12  0.70  Colwell [63]  1996  Nurses  44  NRS/Likert  .70***  0.51  0.83  Cremeans-Smith [64]  2003  Caregivers  114  NRS/Likert  .42***  0.26  0.56    2003  Physicians  5  NRS/Likert  .08  −0.86  0.90  Dawber [65]  2016  Caregivers  50  Other  .58***  0.36  0.74    2016  Nurses  50  Other  .31*  0.03  0.54  de Bock [66]  1994  Physicians  40  VAS  .36*  0.05  0.60  Doherty [67]  1993  Caregivers  20  VAS  .45*  0.01  0.74  Drayer [68]  1999  Other healthcare  45  VAS  .36*  0.07  0.59  Everett [69]  1994  Nurses  27  VAS  .70***  0.44  0.85  Forrest [70]  1989  Physicians  8  VAS  .64  −0.12  0.93  Fridh [71]  1990  Nurses  12  VAS  .29  −0.34  0.74  Gil [72]  2004  Caregivers  35  NRS/Likert  .50**  0.20  0.71  Harrison [73]  1993  Nurses  199  VAS  .34***  0.22  0.46  Hays [74]  1995  Caregivers  304  NRS/Likert  .56***  0.48  0.63  Heikkinen [75]  2005  Nurses  45  Other  .85***  0.75  0.92  Heuss [76]  2012  Nurses  12  VAS  .35  −0.28  0.77    2012  Physicians  9  VAS  .37  −0.39  0.83  Higginson [77]  2008  Caregivers  64  VAS  .69  0.53  0.80  Hodgkins [78]  1985  Physicians  21  VAS  .62**  0.26  0.83  Horgas [79]  2001  Nurses  16  Other  −.25  −0.66  0.28  Idvall [80]  2005  Nurses  267  NRS/Likert  .58***  0.49  0.65  Kelly [81]  2002  Caregivers  78  VAS  .63***  0.47  0.75  Kristjanson [82]  1998  Caregivers  78  Other  .44***  0.24  0.60  Lautenbacher [28]  2013  Nurses  21  NRS/Likert  .10***  −0.35  0.51    2013  Other observers  21  NRS/Likert  .07  −0.37  0.49  Lin [83]  2001  Caregivers  89  NRS/Likert  .58***  0.42  0.70  Lobchuk [84]  1997  Caregivers  37  Other  .31  −0.02  0.58  Lobchuk [20]  2002  Caregivers  98  NRS/Likert  .69***  0.57  0.78  Madison [85]  1995  Caregivers  18  VAS  .49*  0.03  0.78  Maguire [86]  2014  Physicians  200  VAS  .16*  0.02  0.29  Manne [87]  1992  Caregivers  85  Faces  .32**  0.11  0.50    1992  Nurses  9  Faces  .63  −0.06  0.91  Mantyselka [5]  2001  Physicians  28  NRS/Likert  .29  −0.09  0.60  McKinley [88]  1991  Nurses  97  VAS  .35***  0.16  0.51  McMillan [89]  2003  Caregivers  264  NRS/Likert  .40***  0.29  0.50  McPherson [90]  2008  Caregivers  66  NRS/Likert  .44***  0.22  0.62  Miller [91]  1996  Caregivers  20  VAS  .65***  0.29  0.85    1996  Nurses  20  VAS  .39  −0.06  0.71  O’Brien [92]  1988  Caregivers  42  VAS  .23  −0.08  0.50  Oi-Ling [93]  2005  Caregivers  30  NRS/Likert  .38*  0.02  0.65  Paice [94]  1991  Nurses  30  NRS/Likert  .06  −0.31  0.41    1991  Physicians  30  NRS/Likert  .12  −0.25  0.46  Perreault [95]  2006  Other healthcare  9  NRS/Likert  .55  −0.18  0.89  Powers [96]  1987  Nurses  33  VAS  .32  −0.03  0.60  Prkachin [97]  1994  Other observers  5  NRS/Likert  .37  −0.76  0.94  Prkachin [98]  2001  Other observers  82  NRS/Likert  .36***  0.16  0.54    2001  Physicians  34  Other  .36*  0.02  0.62  Redinbaugh [99]  2002  Caregivers  31  NRS/Likert  .55***  0.24  0.76  Resnizky [100]  2006  Caregivers  143  Other  .51***  0.38  0.62  Rhondali [101]  2012  Nurses  20  NRS/Likert  .66***  0.31  0.85  Ruben [7]  2013  Other observers  262  VAS  .10  −0.02  0.22  Ruben [102]  2016  Other observers  95  NRS/Likert  .12  −0.08  0.31  Salmon [103]  1996  Nurses  15  VAS  .62**  0.16  0.86  Schneider [104]  1992  Caregivers  40  Other  .72***  0.53  0.84    1992  Nurses  40  Other  .60***  0.35  0.77  Shega [105]  2004  Caregivers  115  Other  .12  −0.06  0.30  Silveira [106]  2010  Caregivers  142  NRS/Likert  .48***  0.34  0.60  Singer [107]  1999  Physicians  1171  Faces  .47***  0.42  0.51  Singer [108]  2002  Caregivers  57  Faces  .47***  0.24  0.65    2002  Other healthcare  57  VAS  .08  −0.18  0.33  Sjöström [109]  1997  Nurses  90  VAS  .55***  0.39  0.68    1997  Physicians  90  VAS  .68***  0.55  0.78  Sneeuw [110]  1997  Caregivers  103  NRS/Likert  .23*  0.04  0.41  Sneeuw [111]  1998  Caregivers  307  VAS  .63***  0.56  0.69  Sneeuw [112]  1999  Caregivers  90  Faces  .64***  0.50  0.75    1999  Nurses  35  Faces  .66***  0.42  0.81    1999  Physicians  15  Faces  .50  −0.02  0.81  Stephenson [113]  1994  Nurses  11  VAS  .59  −0.02  0.88  St-Laurent-Gagnon [114]  1999  Caregivers  104  Faces  .76***  0.67  0.83  Suarez-Almazor [115]  2001  Physicians  5  VAS  .42  −0.73  0.95  Sutherland [116]  1988  Physicians  22  VAS  .66***  0.33  0.85  van Herk [117]  2009  Caregivers  122  NRS/Likert  .28**  0.11  0.44    2009  Nurses  171  NRS/Likert  .20**  0.05  0.34  Van Der Does [118]  1989  Nurses  30  VAS  .39*  0.03  0.66  Vervoort [119]  2009  Caregivers  62  NRS/Likert  .16  −0.09  0.39  Walkenstein [120]  1982  Nurses  8  Other  .44  −0.38  0.87  Weiner [121]  1999  Caregivers  42  NRS/Likert  −.19  −0.47  0.12    1999  Nurses  42  NRS/Likert  .34*  0.04  0.58  Wennman-Larsen [122]  2007  Caregivers  54  NRS/Likert  .64***  0.45  0.77  Yesilbalkan [123]  2010  Caregivers  80  NRS/Likert  .40***  0.20  0.57  Zalon [124]  1993  Nurses  119  VAS  .30***  0.13  0.46  Zhukovsky [125]  2015  Caregivers  60  Other  .40***  0.16  0.59    2015  Physicians  14  Other  .70**  0.27  0.90  First author  Study year  Observer type  Observer N  Assessment method  Correlation effect size (r)§  95% CI Lower limit  95% CI Upper limit  Akin [56]  2013  Caregivers  119  NRS/Likert  .76***  0.67  0.83    2013  Nurses  7  NRS/Likert  .40  −0.51  0.89  Allen [30]  2002  Caregivers  176  NRS/Likert  .28***  0.14  0.41  Baxt [31]  2004  Caregivers  276  Faces  .43***  0.33  0.52  Bergh [57]  1999  Nurses  39  VAS  .39**  0.08  0.63  Blomqvist [58]  1999  Nurses  29  NRS/Likert  .63***  0.35  0.81  Broberger [59]  2005  Caregivers  54  VAS  .50***  0.27  0.68    2005  Nurses  29  VAS  .41*  0.05  0.68  Chambers [60]  1998  Caregivers  110  Faces  .72***  0.62  0.80  Choiniere [61]  1990  Nurses  82  VAS  .47***  0.28  0.62  Clipp [62]  1992  Caregivers  30  Other  .46**  0.12  0.70  Colwell [63]  1996  Nurses  44  NRS/Likert  .70***  0.51  0.83  Cremeans-Smith [64]  2003  Caregivers  114  NRS/Likert  .42***  0.26  0.56    2003  Physicians  5  NRS/Likert  .08  −0.86  0.90  Dawber [65]  2016  Caregivers  50  Other  .58***  0.36  0.74    2016  Nurses  50  Other  .31*  0.03  0.54  de Bock [66]  1994  Physicians  40  VAS  .36*  0.05  0.60  Doherty [67]  1993  Caregivers  20  VAS  .45*  0.01  0.74  Drayer [68]  1999  Other healthcare  45  VAS  .36*  0.07  0.59  Everett [69]  1994  Nurses  27  VAS  .70***  0.44  0.85  Forrest [70]  1989  Physicians  8  VAS  .64  −0.12  0.93  Fridh [71]  1990  Nurses  12  VAS  .29  −0.34  0.74  Gil [72]  2004  Caregivers  35  NRS/Likert  .50**  0.20  0.71  Harrison [73]  1993  Nurses  199  VAS  .34***  0.22  0.46  Hays [74]  1995  Caregivers  304  NRS/Likert  .56***  0.48  0.63  Heikkinen [75]  2005  Nurses  45  Other  .85***  0.75  0.92  Heuss [76]  2012  Nurses  12  VAS  .35  −0.28  0.77    2012  Physicians  9  VAS  .37  −0.39  0.83  Higginson [77]  2008  Caregivers  64  VAS  .69  0.53  0.80  Hodgkins [78]  1985  Physicians  21  VAS  .62**  0.26  0.83  Horgas [79]  2001  Nurses  16  Other  −.25  −0.66  0.28  Idvall [80]  2005  Nurses  267  NRS/Likert  .58***  0.49  0.65  Kelly [81]  2002  Caregivers  78  VAS  .63***  0.47  0.75  Kristjanson [82]  1998  Caregivers  78  Other  .44***  0.24  0.60  Lautenbacher [28]  2013  Nurses  21  NRS/Likert  .10***  −0.35  0.51    2013  Other observers  21  NRS/Likert  .07  −0.37  0.49  Lin [83]  2001  Caregivers  89  NRS/Likert  .58***  0.42  0.70  Lobchuk [84]  1997  Caregivers  37  Other  .31  −0.02  0.58  Lobchuk [20]  2002  Caregivers  98  NRS/Likert  .69***  0.57  0.78  Madison [85]  1995  Caregivers  18  VAS  .49*  0.03  0.78  Maguire [86]  2014  Physicians  200  VAS  .16*  0.02  0.29  Manne [87]  1992  Caregivers  85  Faces  .32**  0.11  0.50    1992  Nurses  9  Faces  .63  −0.06  0.91  Mantyselka [5]  2001  Physicians  28  NRS/Likert  .29  −0.09  0.60  McKinley [88]  1991  Nurses  97  VAS  .35***  0.16  0.51  McMillan [89]  2003  Caregivers  264  NRS/Likert  .40***  0.29  0.50  McPherson [90]  2008  Caregivers  66  NRS/Likert  .44***  0.22  0.62  Miller [91]  1996  Caregivers  20  VAS  .65***  0.29  0.85    1996  Nurses  20  VAS  .39  −0.06  0.71  O’Brien [92]  1988  Caregivers  42  VAS  .23  −0.08  0.50  Oi-Ling [93]  2005  Caregivers  30  NRS/Likert  .38*  0.02  0.65  Paice [94]  1991  Nurses  30  NRS/Likert  .06  −0.31  0.41    1991  Physicians  30  NRS/Likert  .12  −0.25  0.46  Perreault [95]  2006  Other healthcare  9  NRS/Likert  .55  −0.18  0.89  Powers [96]  1987  Nurses  33  VAS  .32  −0.03  0.60  Prkachin [97]  1994  Other observers  5  NRS/Likert  .37  −0.76  0.94  Prkachin [98]  2001  Other observers  82  NRS/Likert  .36***  0.16  0.54    2001  Physicians  34  Other  .36*  0.02  0.62  Redinbaugh [99]  2002  Caregivers  31  NRS/Likert  .55***  0.24  0.76  Resnizky [100]  2006  Caregivers  143  Other  .51***  0.38  0.62  Rhondali [101]  2012  Nurses  20  NRS/Likert  .66***  0.31  0.85  Ruben [7]  2013  Other observers  262  VAS  .10  −0.02  0.22  Ruben [102]  2016  Other observers  95  NRS/Likert  .12  −0.08  0.31  Salmon [103]  1996  Nurses  15  VAS  .62**  0.16  0.86  Schneider [104]  1992  Caregivers  40  Other  .72***  0.53  0.84    1992  Nurses  40  Other  .60***  0.35  0.77  Shega [105]  2004  Caregivers  115  Other  .12  −0.06  0.30  Silveira [106]  2010  Caregivers  142  NRS/Likert  .48***  0.34  0.60  Singer [107]  1999  Physicians  1171  Faces  .47***  0.42  0.51  Singer [108]  2002  Caregivers  57  Faces  .47***  0.24  0.65    2002  Other healthcare  57  VAS  .08  −0.18  0.33  Sjöström [109]  1997  Nurses  90  VAS  .55***  0.39  0.68    1997  Physicians  90  VAS  .68***  0.55  0.78  Sneeuw [110]  1997  Caregivers  103  NRS/Likert  .23*  0.04  0.41  Sneeuw [111]  1998  Caregivers  307  VAS  .63***  0.56  0.69  Sneeuw [112]  1999  Caregivers  90  Faces  .64***  0.50  0.75    1999  Nurses  35  Faces  .66***  0.42  0.81    1999  Physicians  15  Faces  .50  −0.02  0.81  Stephenson [113]  1994  Nurses  11  VAS  .59  −0.02  0.88  St-Laurent-Gagnon [114]  1999  Caregivers  104  Faces  .76***  0.67  0.83  Suarez-Almazor [115]  2001  Physicians  5  VAS  .42  −0.73  0.95  Sutherland [116]  1988  Physicians  22  VAS  .66***  0.33  0.85  van Herk [117]  2009  Caregivers  122  NRS/Likert  .28**  0.11  0.44    2009  Nurses  171  NRS/Likert  .20**  0.05  0.34  Van Der Does [118]  1989  Nurses  30  VAS  .39*  0.03  0.66  Vervoort [119]  2009  Caregivers  62  NRS/Likert  .16  −0.09  0.39  Walkenstein [120]  1982  Nurses  8  Other  .44  −0.38  0.87  Weiner [121]  1999  Caregivers  42  NRS/Likert  −.19  −0.47  0.12    1999  Nurses  42  NRS/Likert  .34*  0.04  0.58  Wennman-Larsen [122]  2007  Caregivers  54  NRS/Likert  .64***  0.45  0.77  Yesilbalkan [123]  2010  Caregivers  80  NRS/Likert  .40***  0.20  0.57  Zalon [124]  1993  Nurses  119  VAS  .30***  0.13  0.46  Zhukovsky [125]  2015  Caregivers  60  Other  .40***  0.16  0.59    2015  Physicians  14  Other  .70**  0.27  0.90  CI confidence interval; NRS numeric rating scale; VAS visual analogue scale. Faces is Faces Pain Scale. §t-statistic converted to r using formula r = t2(t2+df) *p < .05; **p < .01; ***p < .001. View Large Table 2 Overall effect size summary for pain assessment accuracy reported as correlations and paired comparisons   Effect sizes N  Range of effect sizes  Fixed effect size (95% CI)  Random effect size (95% CI)  Combined Z  Test of homogeneity  Correlation  94  −.25–.85  0.45 (0.44, 0.47)  0.46 (0.41, 0.50)  42.27***  416.52***  Paired Comparison  103  −4.66–.88  −0.13 (−0.15, −0.10)  −0.26 (−0.35, −0.17)  −10.88***  1,286.90***    Effect sizes N  Range of effect sizes  Fixed effect size (95% CI)  Random effect size (95% CI)  Combined Z  Test of homogeneity  Correlation  94  −.25–.85  0.45 (0.44, 0.47)  0.46 (0.41, 0.50)  42.27***  416.52***  Paired Comparison  103  −4.66–.88  −0.13 (−0.15, −0.10)  −0.26 (−0.35, −0.17)  −10.88***  1,286.90***  ***p < .001. Correlation effect sizes are r and paired comparison effect sizes are d. View Large Table 2 Overall effect size summary for pain assessment accuracy reported as correlations and paired comparisons   Effect sizes N  Range of effect sizes  Fixed effect size (95% CI)  Random effect size (95% CI)  Combined Z  Test of homogeneity  Correlation  94  −.25–.85  0.45 (0.44, 0.47)  0.46 (0.41, 0.50)  42.27***  416.52***  Paired Comparison  103  −4.66–.88  −0.13 (−0.15, −0.10)  −0.26 (−0.35, −0.17)  −10.88***  1,286.90***    Effect sizes N  Range of effect sizes  Fixed effect size (95% CI)  Random effect size (95% CI)  Combined Z  Test of homogeneity  Correlation  94  −.25–.85  0.45 (0.44, 0.47)  0.46 (0.41, 0.50)  42.27***  416.52***  Paired Comparison  103  −4.66–.88  −0.13 (−0.15, −0.10)  −0.26 (−0.35, −0.17)  −10.88***  1,286.90***  ***p < .001. Correlation effect sizes are r and paired comparison effect sizes are d. View Large Characteristics of the study: correlation Studies were published from 1982 to 2016. Study year was not significant as a continuous moderator (β = −.00, 95% CI: −0.01, 0.00) or in a dichotomous analysis examining differences in the effect size before or after 1996 (Table 3). The majority of studies were conducted in the USA or Canada (54%) or in Europe (34%). Studies conducted in the USA/Canada had significantly lower accuracy than studies conducted in Europe or Asia (Q = 17.17, p < .001). Table 3 Moderators of correlational pain assessment accuracy Moderator  Categories  Effect size (r)  Number of effect sizes  Fixed effects comparisons  Year  <1996  .45  30  0.00    ≥1996  .45  64    Country  Asia/Middle East  .51  8  17.17***    Europe  .51  28      USA/Canada  .43  54      Other  .41  4    Patient Gender  <50% male  .46  33  1.60    ≥50% male  .44  44    Patient Age Group  Children  .52  18  9.32*    Adults  .44  53      Older adults  .45  7      Mixed  .44  16    Observer type  Informal caregivers  .50  40  69.79***    Nurses  .45  31      Physicians  .45  15      Other healthcare providers  .23  3      Other observers  .15  5    Observer Gender  <50% male  .47  52  4.13*    ≥50% male  .54  5    Type of Pain  Acute  .45  39  2.72    Chronic  .47  47      Mixed  .35  4    Pain Condition  Arthritis/musculoskeletal pain  .28  12  110.69***    Burn  .55  3      Cancer  .51  35      Surgery or procedure  .53  27      Laboratory  .11  5    Assessment Method  Faces Pain Scale  .54  10  11.58**    NRS/Likert  .44  35      VAS  .44  34      Other, mixed  .46  14    Location of assessment  Inpatient  .49  32  77.22***    Outpatient  .43  40      Laboratory  .11  3      Mixed  .54  19    Moderator  Categories  Effect size (r)  Number of effect sizes  Fixed effects comparisons  Year  <1996  .45  30  0.00    ≥1996  .45  64    Country  Asia/Middle East  .51  8  17.17***    Europe  .51  28      USA/Canada  .43  54      Other  .41  4    Patient Gender  <50% male  .46  33  1.60    ≥50% male  .44  44    Patient Age Group  Children  .52  18  9.32*    Adults  .44  53      Older adults  .45  7      Mixed  .44  16    Observer type  Informal caregivers  .50  40  69.79***    Nurses  .45  31      Physicians  .45  15      Other healthcare providers  .23  3      Other observers  .15  5    Observer Gender  <50% male  .47  52  4.13*    ≥50% male  .54  5    Type of Pain  Acute  .45  39  2.72    Chronic  .47  47      Mixed  .35  4    Pain Condition  Arthritis/musculoskeletal pain  .28  12  110.69***    Burn  .55  3      Cancer  .51  35      Surgery or procedure  .53  27      Laboratory  .11  5    Assessment Method  Faces Pain Scale  .54  10  11.58**    NRS/Likert  .44  35      VAS  .44  34      Other, mixed  .46  14    Location of assessment  Inpatient  .49  32  77.22***    Outpatient  .43  40      Laboratory  .11  3      Mixed  .54  19    *p < .05; **p < .01; ***p < .001. NRS numeric rating scale; VAS visual analogue scale. View Large Table 3 Moderators of correlational pain assessment accuracy Moderator  Categories  Effect size (r)  Number of effect sizes  Fixed effects comparisons  Year  <1996  .45  30  0.00    ≥1996  .45  64    Country  Asia/Middle East  .51  8  17.17***    Europe  .51  28      USA/Canada  .43  54      Other  .41  4    Patient Gender  <50% male  .46  33  1.60    ≥50% male  .44  44    Patient Age Group  Children  .52  18  9.32*    Adults  .44  53      Older adults  .45  7      Mixed  .44  16    Observer type  Informal caregivers  .50  40  69.79***    Nurses  .45  31      Physicians  .45  15      Other healthcare providers  .23  3      Other observers  .15  5    Observer Gender  <50% male  .47  52  4.13*    ≥50% male  .54  5    Type of Pain  Acute  .45  39  2.72    Chronic  .47  47      Mixed  .35  4    Pain Condition  Arthritis/musculoskeletal pain  .28  12  110.69***    Burn  .55  3      Cancer  .51  35      Surgery or procedure  .53  27      Laboratory  .11  5    Assessment Method  Faces Pain Scale  .54  10  11.58**    NRS/Likert  .44  35      VAS  .44  34      Other, mixed  .46  14    Location of assessment  Inpatient  .49  32  77.22***    Outpatient  .43  40      Laboratory  .11  3      Mixed  .54  19    Moderator  Categories  Effect size (r)  Number of effect sizes  Fixed effects comparisons  Year  <1996  .45  30  0.00    ≥1996  .45  64    Country  Asia/Middle East  .51  8  17.17***    Europe  .51  28      USA/Canada  .43  54      Other  .41  4    Patient Gender  <50% male  .46  33  1.60    ≥50% male  .44  44    Patient Age Group  Children  .52  18  9.32*    Adults  .44  53      Older adults  .45  7      Mixed  .44  16    Observer type  Informal caregivers  .50  40  69.79***    Nurses  .45  31      Physicians  .45  15      Other healthcare providers  .23  3      Other observers  .15  5    Observer Gender  <50% male  .47  52  4.13*    ≥50% male  .54  5    Type of Pain  Acute  .45  39  2.72    Chronic  .47  47      Mixed  .35  4    Pain Condition  Arthritis/musculoskeletal pain  .28  12  110.69***    Burn  .55  3      Cancer  .51  35      Surgery or procedure  .53  27      Laboratory  .11  5    Assessment Method  Faces Pain Scale  .54  10  11.58**    NRS/Likert  .44  35      VAS  .44  34      Other, mixed  .46  14    Location of assessment  Inpatient  .49  32  77.22***    Outpatient  .43  40      Laboratory  .11  3      Mixed  .54  19    *p < .05; **p < .01; ***p < .001. NRS numeric rating scale; VAS visual analogue scale. View Large Characteristics of the patients: correlation The number of patients in the studies ranged from 8 to 1104 (M = 112.03, SD = 114.07). Patient age group (Q = 9.32, p < .05) significantly impacted pain assessment accuracy with accuracy highest when children were being assessed. Patient gender did not moderate correlational pain assessment accuracy (Q = 1.60, p = .21). Characteristics of the observer: correlation The number of observers in the study ranged from 5 to 1171 (M = 82.19, SD = 133.71). Observer characteristics including the observer type (Q = 69.62, p < .001) and gender (Q = 4.13, p < .05) were significant moderators in pain assessment accuracy. The 40 effect sizes which measured accuracy for caregivers such as a spouse or parent showed the highest accuracy (r = .50), while other healthcare providers (r = .23) and other observers (r = .15) showed the lowest accuracy levels. Fifty-seven studies reported on observer gender. Five studies that reported more than 50% of their observers were male had higher pain assessment accuracy (r = .54) than studies reporting a greater number of female observers (r = .47). Of the studies that documented an average number of years of clinical experience for healthcare providers (k = 14), a regression showed no impact as a significant moderator of pain assessment accuracy (β = −.02, 95% CI: −0.06, 0.01). Characteristics of the pain and assessment: correlation In order to be classified as acute, chronic, or mixed, studies had to state the type of pain experienced by the patients included in their sample. Pain assessment accuracy did not differ by pain type (acute, chronic, or a mix of the two). Observers were more accurate assessing pain related to burns, cancer, and surgeries or other procedures, than for arthritis, musculoskeletal, or laboratory-induced pain (Q = 110.69, p < .001). Pain assessments were less accurate in a laboratory or outpatient setting than in an inpatient or mixed setting (Q = 77.22, p < .001). Pain assessments were most accurate when using the Faces Pain Scale as opposed to the Numeric Rating Scale (NRS), Visual Analogue Scale (VAS), or other pain assessment methods (Q = 11.58, p < .01). Pain Assessment Accuracy: Paired Comparison Ninety articles yielded 103 independent effect sizes for the paired comparison meta-analysis (Table 4). The random effects mean effect size was d = −.26 (95% CI: −0.35, −0.17) and the fixed, weighted mean effect size was d = −.13 (95% CI: −0.15, −0.10) (Table 2). The significant combined Z = −10.88 (p < .001) indicates that observers significantly underestimated patient pain. It would take 4139 effect sizes (40.2 missing studies for each study included in the meta-analysis) to nullify the effects. The effect sizes were significantly heterogeneous so moderator analyses were conducted. Table 4 Characteristics and effect sizes of studies included in pain assessment paired comparison accuracy meta-analysis First author  Study year  Observer type  Observer N  Assessment method  Effect size (d)  SE  95% CI lower limit  95% CI upper limit  Direction of pain assessment  Akin [56]  2013  Caregivers  119  NRS/Likert  .17  0.13  −0.08  0.42  Overestimation    2013  Nurses  119  NRS/Likert  .00  0.13  −0.26  0.25    Bergh [57]  1999  Nurses  24  VAS  .23  0.21  −0.18  0.63  Overestimation    1999  Nurses  8  VAS  −.36  0.36  −1.07  0.36  Underestimation    1999  Nurses  7  VAS  −1.65**  0.58  −2.79  −0.51  Underestimation  Bowman [126]  1994  Nurses  16  VAS  −1.06***  0.31  −1.67  −0.44  Underestimation  Broberger [59]  2005  Caregivers  52  NRS/Likert  .32*  0.14  0.04  0.60  Overestimation    2005  Nurses  33  NRS/Likert  .50**  0.18  0.14  0.87  Overestimation  Cano [127]  2004  Caregivers  109  NRS/Likert  .34***  0.10  0.14  0.53  Overestimation  Chambers [60]  1998  Caregivers  104  Faces  −.15  0.10  −0.35  0.04  Underestimation  Chambers [128]  1999  Caregivers  75  Faces  .34**  0.12  0.11  0.57  Overestimation  Coran [129]  2013  Physicians  10  NRS/Likert  −.23  0.35  −0.91  0.45  Underestimation  Dar [130]  1992  Caregivers  40  NRS/Likert  .02  0.22  −0.42  0.46  Overestimation  Dobkin [131]  2003  Physicians  182  VAS  .19**  0.07  0.05  0.34  Overestimation  Everett [69]  1994  Nurses  27  VAS  .01  0.24  −0.46  0.48  Overestimation  Forrest [70]  1989  Physicians  8  VAS  -.53  0.38  −1.28  0.22  Underestimation  Gil [72]  2004  Caregivers  35  NRS/Likert  .33  0.24  −0.14  0.80  Overestimation  Goulet [132]  2013  Other healthcare  1643  NRS/Likert  −.08***  0.02  −0.13  −0.03  Underestimation  Green [133]  2009  Other observers  130  NRS/Likert  −1.47***  0.27  −1.99  −0.95  Underestimation  Guru [134]  2000  Nurses  71  Other  −.63***  0.17  −0.96  −0.29  Underestimation    2000  Physicians  71  Other  −.28  0.17  −0.61  0.06  Underestimation  Hall-Lord [135]  1998  Nurses  44  NRS/Likert  −.24  0.21  −0.65  0.16  Underestimation    1998  Nurses  37  NRS/Likert  −.47*  0.22  −0.90  −0.04  Underestimation  Hays [74]  1995  Caregivers  304  NRS/Likert  .01  0.08  −0.15  0.17  Overestimation  Heikkinen [75]  2005  Nurses  45  VAS  −.09  0.21  −0.50  0.32  Underestimation  Heuss [76]  2012  Nurses  12  VAS  .44  0.30  −0.14  1.02  Overestimation    2012  Physicians  9  VAS  .39  0.34  −0.28  1.06  Overestimation  Higginson [77]  2008  Caregivers  64  VAS  −.10  0.18  −0.44  0.25  Underestimation  Hodgkins [78]  1985  Physicians  21  VAS  −.23  0.31  −0.83  0.38  Underestimation    1985  Physicians  21  VAS  −.33  0.22  −0.77  0.11  Underestimation  Holmes [136]  1989  Nurses  53  VAS  .32*  0.14  0.04  0.59  Overestimation  Horgas [79]  2001  Nurses  16  Other  .22  0.33  −0.43  0.86  Overestimation  Idvall [137]  2002  Nurses  196  NRS/Likert  −.27**  0.10  −0.47  −0.08  Underestimation  Idvall [80]  2005  Nurses  267  NRS/Likert  −.27**  0.09  −0.44  −0.10  Underestimation  Kappesser [138]  2006  Other healthcare  120  Other  −1.83***  0.38  −2.58  −1.08  Underestimation  Ketovuori [139]  1987  Nurses  62  Other  .64*  0.25  0.14  1.13  Overestimation  Kristjanson [82]  1998  Caregivers  78  Other  .25  0.16  −0.06  0.57  Overestimation  Krivo [140]  1996  Nurses  48  VAS  −.75***  0.21  −1.16  −0.34  Underestimation    1996  Physicians  50  VAS  −.88***  0.21  −1.29  −0.46  Underestimation  Lamontagne [29]  1991  Nurses  13  VAS  −.70*  0.31  −1.31  −0.10  Underestimation    1991  Physicians  13  VAS  −1.03**  0.34  −1.71  −0.36  Underestimation  Laugsand [141]  2010  Other healthcare  1928  NRS/Likert  −.47***  0.03  −0.53  −0.40  Underestimation  Lieberman [142]  1996  Physicians  147  VAS  −.27*  0.12  −0.50  −0.04  Underestimation  Lin [83]  2001  Caregivers  89  NRS/Likert  .07  0.11  −0.13  0.28  Overestimation  Lobchuk [84]  1997  Caregivers  37  NRS/Likert  .38  0.23  −0.08  0.84  Overestimation  Lobchuk [20]  2002  Caregivers  98  Other  .17  0.14  −0.11  0.45  Overestimation  Madison [85]  1995  Caregivers  18  VAS  .12  0.33  −0.53  0.78  Overestimation  Maguire [86]  2014  Physicians  200  VAS  −1.11***  0.11  −1.32  −0.90  Underestimation  Manne [87]a  1992  Nurses  9  Faces  −.07  0.35  −0.75  0.62  Underestimation  Mäntyselkä [5]  2001  Physicians  28  VAS  −.16  0.19  −0.53  0.22  Underestimation  Marquié [143]  2003  Physicians  172  VAS  −.74***  0.09  −0.91  −0.57  Underestimation  Martire [144]  2006  Caregivers  137  VAS  .49***  0.12  0.25  0.73  Overestimation  McMillan [89]  2003  Caregivers  264  NRS/Likert  .42***  0.06  0.29  0.54  Overestimation  McPherson [90]  2008  Caregivers  66  NRS/Likert  .19  0.12  −0.05  0.43  Overestimation  Melotti [145]  2009  Nurses  17  NRS/Likert  .11  0.34  −0.56  0.79  Overestimation  Milne [146]  2006  Caregivers  51  Other  .43*  0.20  0.04  0.82  Overestimation  Modić Stanke [147]  2010  Nurses  31  NRS/Likert  −.47  0.61  −1.66  0.72  Underestimation    2010  Other observers  32  NRS/Likert  −.79  0.61  −1.99  0.41  Underestimation  Molassiotis [148]  2010  Caregivers  82  NRS/Likert  .00  0.15  −0.29  0.29    Oechsle [26]  2013  Caregivers  39  VAS  .53*  0.23  0.08  0.98  Overestimation    2013  Physicians  40  NRS/Likert  .00  0.22  −0.44  0.44    Ovayolu [149]  2015  Caregivers  220  NRS/Likert  .08  0.10  −0.11  0.26  Overestimation  Perreault [150]  2005  Other healthcare  78  NRS/Likert  −.28*  0.12  −0.51  −0.06  Underestimation  Perreault [95]  2006  Other healthcare  9  NRS/Likert  −.26  0.35  −0.96  0.43  Underestimation  Prkachin [97]  1994  Other observers  5  NRS/Likert  −.39  0.49  −1.36  0.58  Underestimation  Pronina [151]  2014  Other observers  120  Other  −1.36***  0.27  −1.88  −0.84  Underestimation  Puntillo [152]  2003  Nurses  37  NRS/Likert  −1.33***  0.19  −1.70  −0.96  Underestimation  Redinbaugh [99]  2002  Caregivers  31  VAS  .88***  0.21  0.47  1.29  Overestimation  Riemsma [153]  2000  Caregivers  177  VAS  .49***  0.08  0.33  0.65  Overestimation  Robinson [154]  2003  Other observers  29  VAS  −1.81***  0.42  −2.63  −0.98  Underestimation  Ruben [7]  2013  Other observers  55  NRS/Likert  −2.27***  0.34  −2.94  −1.60  Underestimation    2013  Other observers  13  NRS/Likert  −1.34***  0.41  −2.15  −0.53  Underestimation    2013  Other observers  21  VAS  −1.19***  0.36  −1.89  −0.48  Underestimation  Ruben [102]  2016  Other observers  172  VAS  −.26***  0.08  −0.41  −0.10  Underestimation  Santos [155]  2014  Caregivers  75  Faces  −4.66***  0.31  −5.27  −4.04  Underestimation    2014  Caregivers  63  Faces  −.43*  0.18  −0.79  −0.08  Underestimation  Schneider [104]  1992  Caregivers  40  NRS/Likert  −.16  0.16  −0.48  0.15  Underestimation    1992  Nurses  40  NRS/Likert  −.43**  0.17  −0.75  −0.10  Underestimation  Shugarman [156]  2010  Nurses  94  VAS  −.29**  0.11  −0.51  −0.06  Underestimation  Silveria [106]  2010  Caregivers  142  VAS  .20*  0.08  0.03  0.37  Overestimation  Sjöström [109]  1997  Other healthcare  60  Other  −1.07***  0.16  −1.37  −0.76  Underestimation  Sloman [157]  2005  Nurses  95  Faces  −.23*  0.10  −0.43  −0.03  Underestimation  Sneeuw [110]  1997  Caregivers  103  Faces  .11  0.10  −0.08  0.31  Overestimation  Sneeuw [111]  1998  Caregivers  307  Faces  .11*  0.06  0.00  0.22  Overestimation  Sneeuw [112]  1999  Caregivers  90  VAS  .25  0.15  −0.04  0.54  Overestimation    1999  Nurses  35  VAS  −.09  0.20  −0.48  0.31  Underestimation    1999  Physicians  15  VAS  −.34  0.28  −0.89  0.21  Underestimation  Stalnikowicz [158]  2005  Nurses  70  VAS  −.82***  0.18  −1.16  −0.47  Underestimation    2005  Physicians  70  VAS  −1.21***  0.18  −1.57  −0.85  Underestimation  Stephenson [113]  1994  Nurses  23  VAS  −.31  0.21  −0.73  0.10  Underestimation  Suarez-Almazor [115]  2001  Physicians  5  VAS  −1.08*  0.46  −1.99  −0.17  Underestimation  Sullivan [159]  2006  Other observers  60  NRS/Likert  −1.96***  0.37  −2.68  −1.24  Underestimation  Sullivan [160]  2006  Other observers  20  NRS/Likert  −.90**  0.29  −1.46  −0.34  Underestimation  Sutherland [116]  1988  Physicians  22  VAS  −.73***  0.22  −1.16  −0.29  Underestimation  Thomas [161]  1999  Physicians  30  VAS  −.75**  0.27  −1.28  −0.23  Underestimation  Todd [162]  1994  Physicians  65  VAS  −.28*  0.14  −0.56  0.00  Underestimation  Van der Does [118]  1989  Nurses  145  VAS  .32**  0.12  0.08  0.56  Overestimation  Vervoort [119]  2009  Caregivers  62  NRS/Likert  −.01  0.13  −0.26  0.24  Underestimation  Walkenstein [120]  1982  Nurses  44  Other  −.49**  0.16  −0.80  −0.17  Underestimation  Wennman-Larsen [122]  2007  Caregivers  54  NRS/Likert  .20  0.14  −0.07  0.47  Overestimation  Yeager [163]  1995  Caregivers  86  VAS  .24*  0.11  0.02  0.45  Overestimation  Yesilbalkan [123]  2010  Caregivers  80  NRS/Likert  .10  0.11  −0.12  0.32  Overestimation  Zalon [124]  1993  Nurses  119  VAS  −.19*  0.09  −0.37  −0.01  Underestimation  First author  Study year  Observer type  Observer N  Assessment method  Effect size (d)  SE  95% CI lower limit  95% CI upper limit  Direction of pain assessment  Akin [56]  2013  Caregivers  119  NRS/Likert  .17  0.13  −0.08  0.42  Overestimation    2013  Nurses  119  NRS/Likert  .00  0.13  −0.26  0.25    Bergh [57]  1999  Nurses  24  VAS  .23  0.21  −0.18  0.63  Overestimation    1999  Nurses  8  VAS  −.36  0.36  −1.07  0.36  Underestimation    1999  Nurses  7  VAS  −1.65**  0.58  −2.79  −0.51  Underestimation  Bowman [126]  1994  Nurses  16  VAS  −1.06***  0.31  −1.67  −0.44  Underestimation  Broberger [59]  2005  Caregivers  52  NRS/Likert  .32*  0.14  0.04  0.60  Overestimation    2005  Nurses  33  NRS/Likert  .50**  0.18  0.14  0.87  Overestimation  Cano [127]  2004  Caregivers  109  NRS/Likert  .34***  0.10  0.14  0.53  Overestimation  Chambers [60]  1998  Caregivers  104  Faces  −.15  0.10  −0.35  0.04  Underestimation  Chambers [128]  1999  Caregivers  75  Faces  .34**  0.12  0.11  0.57  Overestimation  Coran [129]  2013  Physicians  10  NRS/Likert  −.23  0.35  −0.91  0.45  Underestimation  Dar [130]  1992  Caregivers  40  NRS/Likert  .02  0.22  −0.42  0.46  Overestimation  Dobkin [131]  2003  Physicians  182  VAS  .19**  0.07  0.05  0.34  Overestimation  Everett [69]  1994  Nurses  27  VAS  .01  0.24  −0.46  0.48  Overestimation  Forrest [70]  1989  Physicians  8  VAS  -.53  0.38  −1.28  0.22  Underestimation  Gil [72]  2004  Caregivers  35  NRS/Likert  .33  0.24  −0.14  0.80  Overestimation  Goulet [132]  2013  Other healthcare  1643  NRS/Likert  −.08***  0.02  −0.13  −0.03  Underestimation  Green [133]  2009  Other observers  130  NRS/Likert  −1.47***  0.27  −1.99  −0.95  Underestimation  Guru [134]  2000  Nurses  71  Other  −.63***  0.17  −0.96  −0.29  Underestimation    2000  Physicians  71  Other  −.28  0.17  −0.61  0.06  Underestimation  Hall-Lord [135]  1998  Nurses  44  NRS/Likert  −.24  0.21  −0.65  0.16  Underestimation    1998  Nurses  37  NRS/Likert  −.47*  0.22  −0.90  −0.04  Underestimation  Hays [74]  1995  Caregivers  304  NRS/Likert  .01  0.08  −0.15  0.17  Overestimation  Heikkinen [75]  2005  Nurses  45  VAS  −.09  0.21  −0.50  0.32  Underestimation  Heuss [76]  2012  Nurses  12  VAS  .44  0.30  −0.14  1.02  Overestimation    2012  Physicians  9  VAS  .39  0.34  −0.28  1.06  Overestimation  Higginson [77]  2008  Caregivers  64  VAS  −.10  0.18  −0.44  0.25  Underestimation  Hodgkins [78]  1985  Physicians  21  VAS  −.23  0.31  −0.83  0.38  Underestimation    1985  Physicians  21  VAS  −.33  0.22  −0.77  0.11  Underestimation  Holmes [136]  1989  Nurses  53  VAS  .32*  0.14  0.04  0.59  Overestimation  Horgas [79]  2001  Nurses  16  Other  .22  0.33  −0.43  0.86  Overestimation  Idvall [137]  2002  Nurses  196  NRS/Likert  −.27**  0.10  −0.47  −0.08  Underestimation  Idvall [80]  2005  Nurses  267  NRS/Likert  −.27**  0.09  −0.44  −0.10  Underestimation  Kappesser [138]  2006  Other healthcare  120  Other  −1.83***  0.38  −2.58  −1.08  Underestimation  Ketovuori [139]  1987  Nurses  62  Other  .64*  0.25  0.14  1.13  Overestimation  Kristjanson [82]  1998  Caregivers  78  Other  .25  0.16  −0.06  0.57  Overestimation  Krivo [140]  1996  Nurses  48  VAS  −.75***  0.21  −1.16  −0.34  Underestimation    1996  Physicians  50  VAS  −.88***  0.21  −1.29  −0.46  Underestimation  Lamontagne [29]  1991  Nurses  13  VAS  −.70*  0.31  −1.31  −0.10  Underestimation    1991  Physicians  13  VAS  −1.03**  0.34  −1.71  −0.36  Underestimation  Laugsand [141]  2010  Other healthcare  1928  NRS/Likert  −.47***  0.03  −0.53  −0.40  Underestimation  Lieberman [142]  1996  Physicians  147  VAS  −.27*  0.12  −0.50  −0.04  Underestimation  Lin [83]  2001  Caregivers  89  NRS/Likert  .07  0.11  −0.13  0.28  Overestimation  Lobchuk [84]  1997  Caregivers  37  NRS/Likert  .38  0.23  −0.08  0.84  Overestimation  Lobchuk [20]  2002  Caregivers  98  Other  .17  0.14  −0.11  0.45  Overestimation  Madison [85]  1995  Caregivers  18  VAS  .12  0.33  −0.53  0.78  Overestimation  Maguire [86]  2014  Physicians  200  VAS  −1.11***  0.11  −1.32  −0.90  Underestimation  Manne [87]a  1992  Nurses  9  Faces  −.07  0.35  −0.75  0.62  Underestimation  Mäntyselkä [5]  2001  Physicians  28  VAS  −.16  0.19  −0.53  0.22  Underestimation  Marquié [143]  2003  Physicians  172  VAS  −.74***  0.09  −0.91  −0.57  Underestimation  Martire [144]  2006  Caregivers  137  VAS  .49***  0.12  0.25  0.73  Overestimation  McMillan [89]  2003  Caregivers  264  NRS/Likert  .42***  0.06  0.29  0.54  Overestimation  McPherson [90]  2008  Caregivers  66  NRS/Likert  .19  0.12  −0.05  0.43  Overestimation  Melotti [145]  2009  Nurses  17  NRS/Likert  .11  0.34  −0.56  0.79  Overestimation  Milne [146]  2006  Caregivers  51  Other  .43*  0.20  0.04  0.82  Overestimation  Modić Stanke [147]  2010  Nurses  31  NRS/Likert  −.47  0.61  −1.66  0.72  Underestimation    2010  Other observers  32  NRS/Likert  −.79  0.61  −1.99  0.41  Underestimation  Molassiotis [148]  2010  Caregivers  82  NRS/Likert  .00  0.15  −0.29  0.29    Oechsle [26]  2013  Caregivers  39  VAS  .53*  0.23  0.08  0.98  Overestimation    2013  Physicians  40  NRS/Likert  .00  0.22  −0.44  0.44    Ovayolu [149]  2015  Caregivers  220  NRS/Likert  .08  0.10  −0.11  0.26  Overestimation  Perreault [150]  2005  Other healthcare  78  NRS/Likert  −.28*  0.12  −0.51  −0.06  Underestimation  Perreault [95]  2006  Other healthcare  9  NRS/Likert  −.26  0.35  −0.96  0.43  Underestimation  Prkachin [97]  1994  Other observers  5  NRS/Likert  −.39  0.49  −1.36  0.58  Underestimation  Pronina [151]  2014  Other observers  120  Other  −1.36***  0.27  −1.88  −0.84  Underestimation  Puntillo [152]  2003  Nurses  37  NRS/Likert  −1.33***  0.19  −1.70  −0.96  Underestimation  Redinbaugh [99]  2002  Caregivers  31  VAS  .88***  0.21  0.47  1.29  Overestimation  Riemsma [153]  2000  Caregivers  177  VAS  .49***  0.08  0.33  0.65  Overestimation  Robinson [154]  2003  Other observers  29  VAS  −1.81***  0.42  −2.63  −0.98  Underestimation  Ruben [7]  2013  Other observers  55  NRS/Likert  −2.27***  0.34  −2.94  −1.60  Underestimation    2013  Other observers  13  NRS/Likert  −1.34***  0.41  −2.15  −0.53  Underestimation    2013  Other observers  21  VAS  −1.19***  0.36  −1.89  −0.48  Underestimation  Ruben [102]  2016  Other observers  172  VAS  −.26***  0.08  −0.41  −0.10  Underestimation  Santos [155]  2014  Caregivers  75  Faces  −4.66***  0.31  −5.27  −4.04  Underestimation    2014  Caregivers  63  Faces  −.43*  0.18  −0.79  −0.08  Underestimation  Schneider [104]  1992  Caregivers  40  NRS/Likert  −.16  0.16  −0.48  0.15  Underestimation    1992  Nurses  40  NRS/Likert  −.43**  0.17  −0.75  −0.10  Underestimation  Shugarman [156]  2010  Nurses  94  VAS  −.29**  0.11  −0.51  −0.06  Underestimation  Silveria [106]  2010  Caregivers  142  VAS  .20*  0.08  0.03  0.37  Overestimation  Sjöström [109]  1997  Other healthcare  60  Other  −1.07***  0.16  −1.37  −0.76  Underestimation  Sloman [157]  2005  Nurses  95  Faces  −.23*  0.10  −0.43  −0.03  Underestimation  Sneeuw [110]  1997  Caregivers  103  Faces  .11  0.10  −0.08  0.31  Overestimation  Sneeuw [111]  1998  Caregivers  307  Faces  .11*  0.06  0.00  0.22  Overestimation  Sneeuw [112]  1999  Caregivers  90  VAS  .25  0.15  −0.04  0.54  Overestimation    1999  Nurses  35  VAS  −.09  0.20  −0.48  0.31  Underestimation    1999  Physicians  15  VAS  −.34  0.28  −0.89  0.21  Underestimation  Stalnikowicz [158]  2005  Nurses  70  VAS  −.82***  0.18  −1.16  −0.47  Underestimation    2005  Physicians  70  VAS  −1.21***  0.18  −1.57  −0.85  Underestimation  Stephenson [113]  1994  Nurses  23  VAS  −.31  0.21  −0.73  0.10  Underestimation  Suarez-Almazor [115]  2001  Physicians  5  VAS  −1.08*  0.46  −1.99  −0.17  Underestimation  Sullivan [159]  2006  Other observers  60  NRS/Likert  −1.96***  0.37  −2.68  −1.24  Underestimation  Sullivan [160]  2006  Other observers  20  NRS/Likert  −.90**  0.29  −1.46  −0.34  Underestimation  Sutherland [116]  1988  Physicians  22  VAS  −.73***  0.22  −1.16  −0.29  Underestimation  Thomas [161]  1999  Physicians  30  VAS  −.75**  0.27  −1.28  −0.23  Underestimation  Todd [162]  1994  Physicians  65  VAS  −.28*  0.14  −0.56  0.00  Underestimation  Van der Does [118]  1989  Nurses  145  VAS  .32**  0.12  0.08  0.56  Overestimation  Vervoort [119]  2009  Caregivers  62  NRS/Likert  −.01  0.13  −0.26  0.24  Underestimation  Walkenstein [120]  1982  Nurses  44  Other  −.49**  0.16  −0.80  −0.17  Underestimation  Wennman-Larsen [122]  2007  Caregivers  54  NRS/Likert  .20  0.14  −0.07  0.47  Overestimation  Yeager [163]  1995  Caregivers  86  VAS  .24*  0.11  0.02  0.45  Overestimation  Yesilbalkan [123]  2010  Caregivers  80  NRS/Likert  .10  0.11  −0.12  0.32  Overestimation  Zalon [124]  1993  Nurses  119  VAS  −.19*  0.09  −0.37  −0.01  Underestimation  aStudies used different pain scales for the patients and observers and we standardized results for analysis. d is standard difference in means. SE is standard error. NRS is numeric rating scale, VAS is visual analogue scale. Faces is Faces Pain Scale. *p < .05; **p < .01; ***p < .001. View Large Table 4 Characteristics and effect sizes of studies included in pain assessment paired comparison accuracy meta-analysis First author  Study year  Observer type  Observer N  Assessment method  Effect size (d)  SE  95% CI lower limit  95% CI upper limit  Direction of pain assessment  Akin [56]  2013  Caregivers  119  NRS/Likert  .17  0.13  −0.08  0.42  Overestimation    2013  Nurses  119  NRS/Likert  .00  0.13  −0.26  0.25    Bergh [57]  1999  Nurses  24  VAS  .23  0.21  −0.18  0.63  Overestimation    1999  Nurses  8  VAS  −.36  0.36  −1.07  0.36  Underestimation    1999  Nurses  7  VAS  −1.65**  0.58  −2.79  −0.51  Underestimation  Bowman [126]  1994  Nurses  16  VAS  −1.06***  0.31  −1.67  −0.44  Underestimation  Broberger [59]  2005  Caregivers  52  NRS/Likert  .32*  0.14  0.04  0.60  Overestimation    2005  Nurses  33  NRS/Likert  .50**  0.18  0.14  0.87  Overestimation  Cano [127]  2004  Caregivers  109  NRS/Likert  .34***  0.10  0.14  0.53  Overestimation  Chambers [60]  1998  Caregivers  104  Faces  −.15  0.10  −0.35  0.04  Underestimation  Chambers [128]  1999  Caregivers  75  Faces  .34**  0.12  0.11  0.57  Overestimation  Coran [129]  2013  Physicians  10  NRS/Likert  −.23  0.35  −0.91  0.45  Underestimation  Dar [130]  1992  Caregivers  40  NRS/Likert  .02  0.22  −0.42  0.46  Overestimation  Dobkin [131]  2003  Physicians  182  VAS  .19**  0.07  0.05  0.34  Overestimation  Everett [69]  1994  Nurses  27  VAS  .01  0.24  −0.46  0.48  Overestimation  Forrest [70]  1989  Physicians  8  VAS  -.53  0.38  −1.28  0.22  Underestimation  Gil [72]  2004  Caregivers  35  NRS/Likert  .33  0.24  −0.14  0.80  Overestimation  Goulet [132]  2013  Other healthcare  1643  NRS/Likert  −.08***  0.02  −0.13  −0.03  Underestimation  Green [133]  2009  Other observers  130  NRS/Likert  −1.47***  0.27  −1.99  −0.95  Underestimation  Guru [134]  2000  Nurses  71  Other  −.63***  0.17  −0.96  −0.29  Underestimation    2000  Physicians  71  Other  −.28  0.17  −0.61  0.06  Underestimation  Hall-Lord [135]  1998  Nurses  44  NRS/Likert  −.24  0.21  −0.65  0.16  Underestimation    1998  Nurses  37  NRS/Likert  −.47*  0.22  −0.90  −0.04  Underestimation  Hays [74]  1995  Caregivers  304  NRS/Likert  .01  0.08  −0.15  0.17  Overestimation  Heikkinen [75]  2005  Nurses  45  VAS  −.09  0.21  −0.50  0.32  Underestimation  Heuss [76]  2012  Nurses  12  VAS  .44  0.30  −0.14  1.02  Overestimation    2012  Physicians  9  VAS  .39  0.34  −0.28  1.06  Overestimation  Higginson [77]  2008  Caregivers  64  VAS  −.10  0.18  −0.44  0.25  Underestimation  Hodgkins [78]  1985  Physicians  21  VAS  −.23  0.31  −0.83  0.38  Underestimation    1985  Physicians  21  VAS  −.33  0.22  −0.77  0.11  Underestimation  Holmes [136]  1989  Nurses  53  VAS  .32*  0.14  0.04  0.59  Overestimation  Horgas [79]  2001  Nurses  16  Other  .22  0.33  −0.43  0.86  Overestimation  Idvall [137]  2002  Nurses  196  NRS/Likert  −.27**  0.10  −0.47  −0.08  Underestimation  Idvall [80]  2005  Nurses  267  NRS/Likert  −.27**  0.09  −0.44  −0.10  Underestimation  Kappesser [138]  2006  Other healthcare  120  Other  −1.83***  0.38  −2.58  −1.08  Underestimation  Ketovuori [139]  1987  Nurses  62  Other  .64*  0.25  0.14  1.13  Overestimation  Kristjanson [82]  1998  Caregivers  78  Other  .25  0.16  −0.06  0.57  Overestimation  Krivo [140]  1996  Nurses  48  VAS  −.75***  0.21  −1.16  −0.34  Underestimation    1996  Physicians  50  VAS  −.88***  0.21  −1.29  −0.46  Underestimation  Lamontagne [29]  1991  Nurses  13  VAS  −.70*  0.31  −1.31  −0.10  Underestimation    1991  Physicians  13  VAS  −1.03**  0.34  −1.71  −0.36  Underestimation  Laugsand [141]  2010  Other healthcare  1928  NRS/Likert  −.47***  0.03  −0.53  −0.40  Underestimation  Lieberman [142]  1996  Physicians  147  VAS  −.27*  0.12  −0.50  −0.04  Underestimation  Lin [83]  2001  Caregivers  89  NRS/Likert  .07  0.11  −0.13  0.28  Overestimation  Lobchuk [84]  1997  Caregivers  37  NRS/Likert  .38  0.23  −0.08  0.84  Overestimation  Lobchuk [20]  2002  Caregivers  98  Other  .17  0.14  −0.11  0.45  Overestimation  Madison [85]  1995  Caregivers  18  VAS  .12  0.33  −0.53  0.78  Overestimation  Maguire [86]  2014  Physicians  200  VAS  −1.11***  0.11  −1.32  −0.90  Underestimation  Manne [87]a  1992  Nurses  9  Faces  −.07  0.35  −0.75  0.62  Underestimation  Mäntyselkä [5]  2001  Physicians  28  VAS  −.16  0.19  −0.53  0.22  Underestimation  Marquié [143]  2003  Physicians  172  VAS  −.74***  0.09  −0.91  −0.57  Underestimation  Martire [144]  2006  Caregivers  137  VAS  .49***  0.12  0.25  0.73  Overestimation  McMillan [89]  2003  Caregivers  264  NRS/Likert  .42***  0.06  0.29  0.54  Overestimation  McPherson [90]  2008  Caregivers  66  NRS/Likert  .19  0.12  −0.05  0.43  Overestimation  Melotti [145]  2009  Nurses  17  NRS/Likert  .11  0.34  −0.56  0.79  Overestimation  Milne [146]  2006  Caregivers  51  Other  .43*  0.20  0.04  0.82  Overestimation  Modić Stanke [147]  2010  Nurses  31  NRS/Likert  −.47  0.61  −1.66  0.72  Underestimation    2010  Other observers  32  NRS/Likert  −.79  0.61  −1.99  0.41  Underestimation  Molassiotis [148]  2010  Caregivers  82  NRS/Likert  .00  0.15  −0.29  0.29    Oechsle [26]  2013  Caregivers  39  VAS  .53*  0.23  0.08  0.98  Overestimation    2013  Physicians  40  NRS/Likert  .00  0.22  −0.44  0.44    Ovayolu [149]  2015  Caregivers  220  NRS/Likert  .08  0.10  −0.11  0.26  Overestimation  Perreault [150]  2005  Other healthcare  78  NRS/Likert  −.28*  0.12  −0.51  −0.06  Underestimation  Perreault [95]  2006  Other healthcare  9  NRS/Likert  −.26  0.35  −0.96  0.43  Underestimation  Prkachin [97]  1994  Other observers  5  NRS/Likert  −.39  0.49  −1.36  0.58  Underestimation  Pronina [151]  2014  Other observers  120  Other  −1.36***  0.27  −1.88  −0.84  Underestimation  Puntillo [152]  2003  Nurses  37  NRS/Likert  −1.33***  0.19  −1.70  −0.96  Underestimation  Redinbaugh [99]  2002  Caregivers  31  VAS  .88***  0.21  0.47  1.29  Overestimation  Riemsma [153]  2000  Caregivers  177  VAS  .49***  0.08  0.33  0.65  Overestimation  Robinson [154]  2003  Other observers  29  VAS  −1.81***  0.42  −2.63  −0.98  Underestimation  Ruben [7]  2013  Other observers  55  NRS/Likert  −2.27***  0.34  −2.94  −1.60  Underestimation    2013  Other observers  13  NRS/Likert  −1.34***  0.41  −2.15  −0.53  Underestimation    2013  Other observers  21  VAS  −1.19***  0.36  −1.89  −0.48  Underestimation  Ruben [102]  2016  Other observers  172  VAS  −.26***  0.08  −0.41  −0.10  Underestimation  Santos [155]  2014  Caregivers  75  Faces  −4.66***  0.31  −5.27  −4.04  Underestimation    2014  Caregivers  63  Faces  −.43*  0.18  −0.79  −0.08  Underestimation  Schneider [104]  1992  Caregivers  40  NRS/Likert  −.16  0.16  −0.48  0.15  Underestimation    1992  Nurses  40  NRS/Likert  −.43**  0.17  −0.75  −0.10  Underestimation  Shugarman [156]  2010  Nurses  94  VAS  −.29**  0.11  −0.51  −0.06  Underestimation  Silveria [106]  2010  Caregivers  142  VAS  .20*  0.08  0.03  0.37  Overestimation  Sjöström [109]  1997  Other healthcare  60  Other  −1.07***  0.16  −1.37  −0.76  Underestimation  Sloman [157]  2005  Nurses  95  Faces  −.23*  0.10  −0.43  −0.03  Underestimation  Sneeuw [110]  1997  Caregivers  103  Faces  .11  0.10  −0.08  0.31  Overestimation  Sneeuw [111]  1998  Caregivers  307  Faces  .11*  0.06  0.00  0.22  Overestimation  Sneeuw [112]  1999  Caregivers  90  VAS  .25  0.15  −0.04  0.54  Overestimation    1999  Nurses  35  VAS  −.09  0.20  −0.48  0.31  Underestimation    1999  Physicians  15  VAS  −.34  0.28  −0.89  0.21  Underestimation  Stalnikowicz [158]  2005  Nurses  70  VAS  −.82***  0.18  −1.16  −0.47  Underestimation    2005  Physicians  70  VAS  −1.21***  0.18  −1.57  −0.85  Underestimation  Stephenson [113]  1994  Nurses  23  VAS  −.31  0.21  −0.73  0.10  Underestimation  Suarez-Almazor [115]  2001  Physicians  5  VAS  −1.08*  0.46  −1.99  −0.17  Underestimation  Sullivan [159]  2006  Other observers  60  NRS/Likert  −1.96***  0.37  −2.68  −1.24  Underestimation  Sullivan [160]  2006  Other observers  20  NRS/Likert  −.90**  0.29  −1.46  −0.34  Underestimation  Sutherland [116]  1988  Physicians  22  VAS  −.73***  0.22  −1.16  −0.29  Underestimation  Thomas [161]  1999  Physicians  30  VAS  −.75**  0.27  −1.28  −0.23  Underestimation  Todd [162]  1994  Physicians  65  VAS  −.28*  0.14  −0.56  0.00  Underestimation  Van der Does [118]  1989  Nurses  145  VAS  .32**  0.12  0.08  0.56  Overestimation  Vervoort [119]  2009  Caregivers  62  NRS/Likert  −.01  0.13  −0.26  0.24  Underestimation  Walkenstein [120]  1982  Nurses  44  Other  −.49**  0.16  −0.80  −0.17  Underestimation  Wennman-Larsen [122]  2007  Caregivers  54  NRS/Likert  .20  0.14  −0.07  0.47  Overestimation  Yeager [163]  1995  Caregivers  86  VAS  .24*  0.11  0.02  0.45  Overestimation  Yesilbalkan [123]  2010  Caregivers  80  NRS/Likert  .10  0.11  −0.12&nbs