Don’t Judge a Book by its Cover: Examiner Expectancy Effects Predict Neuropsychological Performance for Individuals Judged as Chronic Cannabis Users

Don’t Judge a Book by its Cover: Examiner Expectancy Effects Predict Neuropsychological... Abstract Objective The experimenter expectancy effect confound remains largely unexplored in neuropsychological research and has never been investigated among cannabis users. This study investigated whether examiner expectancies of cannabis user status affected examinees’ neuropsychological performance. Method Participants included 41 cannabis users and 20 non-users. Before testing, examiners who were blind to participant user status privately rated whether they believed the examinee was a cannabis user or non-user. Examiners then administered a battery of neuropsychological and performance validity measures. Multiple regression analyses compared performance between examinees judged as cannabis users (n = 37) and those judged as non-users (n = 24). Results Examiners’ judgments of cannabis users were 75% accurate; judgments of non-users were at chance. After controlling for age, gender, and actual user status, examiner judgments of cannabis user status predicted performance on two measures (California Verbal Learning Test-II, and Trail Making Test B; p < .05), as individuals judged as cannabis users obtained lower scores than those judged as non-users. Conclusions Examiners’ judgments of cannabis user status predicted performance even after controlling for actual user status, indicating vulnerability to examiner expectancy effects. These findings have important implications for both research and clinical settings, as scores may partially reflect examiners’ expectations regarding cannabis effects rather than participants’ cognitive abilities. These results demonstrate the need for expectancy effect research in the neuropsychological assessment of all populations, not just cannabis users. Malingering/symptom validity testing, Learning and memory, Assessment, Drug and alcohol abuse Introduction There is increasing importance in understanding the effects of cannabis use on neurocognitive functioning. Cannabis use has increased globally in adult populations, both medically and recreationally, and is currently one of the most widely used drugs worldwide, with an estimated 183 million people using it (United Nations Office on Drugs and Crime, 2016). Previous research investigating the effect of cannabis on cognition has produced mixed results regarding long-term use outcomes and the potential detrimental effects on cognition. Although some studies have suggested that cannabis negatively affects cognitive functioning in various domains among long-term users (Grant, Gonzalez, Carey, Natarajan, & Wolfson, 2003; Solowij et al., 2002), others found no cognitive differences between users and non-users (Lyketsos, Garrett, Liang, & Anthony, 1999; Schreiner & Dunn, 2012). Further, even in those studies that find cognitive effects, there is inconsistency across studies regarding which cognitive domains are affected by cannabis use. Studies have found poorer performance among cannabis users on tasks of attention, processing speed (Medina et al., 2007), learning and memory (Bolla, Brown, Eldreth, Tate, & Cadet, 2002; Grant et al., 2003; Meier et al., 2012; Solowij et al., 2002), and executive functioning (Bolla et al., 2002; Gruber, Sagar, Dahlgren, Racine, & Lukas, 2012). Therefore, neuropsychological performance differences of cannabis users relative to non-users vary widely from study to study. Researchers have suggested that the variability in findings may be due to the wide variety of research designs implemented (e.g., variability in what criteria constitutes chronic cannabis use, age of onset of cannabis use, or length of abstinence periods in participants), as well as methodological limitations, including failure to control for alcohol use, lack of data gathered regarding premorbid neurocognitive functioning, and failure to report cannabis abstinence periods (Schreiner & Dunn, 2012). One major potential confound that has yet to be explored in cannabis research is the expectancy effect. Expectancy effects occur when experimenters’ personal expectations about study outcomes influence participants’ performance (Rosenthal, 2002). Experimenters’ expectations in research studies can lead to subtle differences in interactions with the participant, resulting in a shift in the participant’s behavior in a manner that confirms the researchers’ hypotheses (Rosenthal, Kohn, Greenfield, & Carota, 1966). The expectancy effect confound subsequently increases the probability of invalid or inaccurate findings. For example, researchers who expect cannabis use to be associated with memory deficits may unconsciously act in subtly different ways towards cannabis user participants relative to non-user participants, such as allowing less time for responding during memory tests or offering less encouragement. These differences could influence rapport and thus alter participants’ performance on the testing (Karver, Handelsman, Fields, & Bickman, 2005). To minimize the potential for expectancy effects, Rosenthal (1991) strongly recommends utilizing blind research designs, i.e.,keeping experimenters unaware of participants’ group status. Despite the extensive literature on the expectancy effect confound, cannabis researchers often fail to implement blind research designs when examining the cognitive effects of cannabis use (e.g., Battisti et al., 2010; Dougherty et al., 2013; Mahmood, Jacobus, Bava, Scarlett, & Tapert, 2010; Rodgers, 2000; Solowij, 1995). It is important to note that even studies utilizing blind research designs have yielded mixed results regarding the effects of cannabis use on cognition (Lyons et al., 2004; Pope, Gruber, Hudson, Huestis, & Yurgelun-Todd, 2001). One possible reason for these mixed findings may be due to researchers guessing, either consciously or unconsciously, which user group participants belong to, thus still leaving the studies vulnerable to expectancy effects. Studies have shown that individuals can guess qualities like personality and behavioral traits based simply upon appearance alone, such as by examining a photograph (Fink, Neave, Manning, & Grammer, 2006; Todorov, Pakrashi, & Oosterhof, 2009; Willis & Todorov, 2006). For example, research shows individuals are able to discriminate substance users versus non-users based on a photograph alone, with near 60% accuracy (Olivola & Todorov, 2010). Further, research by Hirst and colleagues (Hirst et al., 2016; 2017) supports the phenomenon of the “jay-dar” – the ability to distinguish whether an individual uses marijuana (i.e., smoking joints or “jays”), similar to Shelp’s (2003) research on “gay-dar.” In these studies, researchers asked both undergraduates (Hirst et al., 2017) and neuropsychologists (i.e., the same profession of those who might conduct research on cannabis users; Hirst et al., 2016) to rate the likelihood that a person was a cannabis user based on photographs alone. Photos of actual cannabis users (those who had used cannabis at least 400 times in their life, wearing their typical clothing and hairstyle) received higher ratings than photos of non-users on the Marijuana Use Likelihood Index, a 7-point Likert scale range from “least likely” to “most likely.” Effect size estimates revealed a medium effect of cannabis use status on both the undergraduates’ (Cohen’s d = 0.46) and the neuropsychologists’ (Cohen’s d = 0.61) ratings (Hirst et al., 2016, 2017). These results suggest that even research studies using examiners who are blind to participants’ group status may be vulnerable to the expectancy effect confound if they are able to guess cannabis user status. Further, there is precedent for anticipating that examiners’ expectancy effects may contribute to variability in participants’ neuropsychological performance. Researchers have found that examiners’ beliefs about the effects of caffeine administration affected examinee physiological response and cognitive performance (Walach, Schmidt, Bihr, & Wiesch, 2001). In this study, experimenters administered placebo “caffeine” to participants and measured their blood pressure, heart rate, well-being, and performance on a cognitive task. When experimenters were told that caffeine administration alters examinees’ physiological and cognitive functions, their participants demonstrated higher systolic blood pressure and worse performance on cognitive testing following placebo administration, relative to the participants of experimenters who were told the opposite. These findings support the possibility that examiner expectations of cannabis user performance on cognitive testing could have acted as a confounding variable in many published research designs to date. The plethora of findings supporting the existence of expectancy effects in research experiments (Rosenthal, 2002) and the results of a study identifying expectancy effects influencing participants’ cognitive performance (Walach et al., 2001) suggest that this confound could be contributing to the variability in the literature on the cognitive effects of chronic cannabis use. Further, research indicates that neuropsychologists can discriminate between cannabis users and non-users, suggesting that even blind research designs are vulnerable to this confound (Hirst et al., 2016, 2017). Therefore, the present study sought to investigate whether examiners’ perceptions of examinees’ cannabis use status influenced participant performance on neuropsychological assessment. Specifically, the authors hypothesized that examiners would be able to judge participant cannabis user status based upon appearance alone with accuracy greater than chance, consistent with Hirst and colleagues (2016, 2017) findings. Further, we hypothesized that participants judged as cannabis users would perform worse on neuropsychological assessment, relative to those judged as non-users (irrespective of true user status). To this aim, examiners privately noted whether they believed the participant was a cannabis user or a non-user, allowing the measurement of whether these perceptions resulted in differential performance by examinees on neuropsychological testing. Finally, because expectancy effects are likely related to the examiners’ personal beliefs regarding the effects of cannabis use, the authors also sought to determine whether the experiences and beliefs of the examiners administering the neuropsychological assessment battery related to examinees’ performance. Method Participants Recruitment consisted of flyers posted in the community, online postings, and advertisements emailed to local colleges. To qualify for inclusion, participants must have been between 18 and 30 years old, to reduce the likelihood of confounding factors influencing cognition (e.g., age-related cognitive decline). In order to meet eligibility criteria as a chronic cannabis user, individuals must have been current cannabis users, using cannabis at least 4 days per week for the past year. To be eligible as a non-user control, the individual must have tried cannabis at least once but no more than five times in his or her lifetime, and not within the past 30 days. Cannabis researchers have previously recommended this approach because individuals who would never try cannabis may be conceivably different from individuals who would try cannabis in ways that may influence cognition, such as decision-making or impulsivity (Pope et al., 2001). The inclusion of individuals who would be willing to try cannabis allows for a non-user control group that is cognitively similar in premorbid functioning to cannabis users. Exclusionary criteria consisted of self-report of: (a) use of any other class of drugs of abuse (e.g., opiates) more than five times; (b) current alcohol use of a frequency that could interfere with neurocognitive performance (defined as consuming two or more drinks on four or more days per week, for the past month or longer); (c) current Diagnostic and Statistical Manual of Mental Disorders – Fourth Edition-Text Revision (DSM–IV-TR; American Psychiatric Association, 2000) diagnosis of Axis I disorder other than simple phobia; (d) history of head injury with loss of consciousness requiring medical intervention within the past six months; (e) current use of psychoactive medication; and (f) a medical, psychiatric, or neurological condition that might affect cognitive function (e.g., epilepsy, multiple sclerosis, brain tumor, etc.). These exclusion criteria were selected to eliminate potential confounds which might affect cognitive performance, and have been recommended by experts in the cannabis literature (Gonzalez, Carey, & Grant, 2002). A total of 61 individuals participated in the study; demographic information for the whole sample is presented in Table 1. Participants’ ages ranged from 18 to 30 years. Thirty-seven (60.7%) participants in the study sample were male, whereas 24 (39.3%) participants were female. Nineteen of the participants identified as Caucasian (31.1%), 16 identified as Hispanic or Latino (26.2%), 10 identified as Asian or Asian-American (16.4%), four identified as Black or African American (6.6%), four identified as Middle Eastern (6.6%), and eight participants identified as Other or Mixed ethnicity (13.1%). Total years of education for the entire sample ranged from 12 to 20 years. The mean estimated premorbid IQ for the entire sample, estimated from the National Adult Reading Test-Revised (NART-R; Wiens, Bryon, & Crossen, 1993), was 102.60 (SD = 15.63). Approximately 90% (n = 55) of the participants were right-handed, whereas about 10% (n = 6) were left-handed. Table 1. Demographic information: total sample, cannabis users, and non-users   Total sample  Cannabis users  Non-users  Age (years)  21.89 (3.32)  21.37 (2.91)  22.95 (3.90)  Percent male  60.7%  73.2%*  35.0%*  Years of education  14.27 (1.81)  13.96 (1.38)  14.90 (2.38)  Estimated premorbid IQ  102.60 (15.63)  100.58 (17.27)  106.76 (10.81)    Total sample  Cannabis users  Non-users  Age (years)  21.89 (3.32)  21.37 (2.91)  22.95 (3.90)  Percent male  60.7%  73.2%*  35.0%*  Years of education  14.27 (1.81)  13.96 (1.38)  14.90 (2.38)  Estimated premorbid IQ  102.60 (15.63)  100.58 (17.27)  106.76 (10.81)  Note: * Group difference is significant at the p < .05 level. Table 1. Demographic information: total sample, cannabis users, and non-users   Total sample  Cannabis users  Non-users  Age (years)  21.89 (3.32)  21.37 (2.91)  22.95 (3.90)  Percent male  60.7%  73.2%*  35.0%*  Years of education  14.27 (1.81)  13.96 (1.38)  14.90 (2.38)  Estimated premorbid IQ  102.60 (15.63)  100.58 (17.27)  106.76 (10.81)    Total sample  Cannabis users  Non-users  Age (years)  21.89 (3.32)  21.37 (2.91)  22.95 (3.90)  Percent male  60.7%  73.2%*  35.0%*  Years of education  14.27 (1.81)  13.96 (1.38)  14.90 (2.38)  Estimated premorbid IQ  102.60 (15.63)  100.58 (17.27)  106.76 (10.81)  Note: * Group difference is significant at the p < .05 level. The sample included 41 chronic cannabis users and 20 non-users. Demographic information of the user and non-user group is also included in Table 1. There were no significant differences between users and non-users in age, ethnicity, years of education, or estimated premorbid IQ based upon performance on the NART-R. There were significantly more males in the user group compared to the non-user group (p = .004); however, the gender distribution of chronic cannabis users in the present study is representative of documented gender differences among marijuana users (Johnston, O’Malley, Bachman, Schulenberg, & Miech, 2015). Within the non-user group, five participants identified as Caucasian (25%), three identified as Hispanic or Latino (15%), six identified as Asian or Asian-American (30%), three identified as Middle Eastern (15%), and three participants identified as Other or Mixed ethnicity (15%). Within the user group, 14 participants identified as Caucasian (34.1%), 13 identified as Hispanic or Latino (31.7%), four identified as Asian or Asian-American (9.8%), four identified as Black or African American (9.8%), one identified as Middle Eastern (2.4%), and five participants identified as Other or Mixed ethnicity (12.2%). Approximately 85% (n = 17) of the non-users were right-handed, whereas about 92% (n = 38) of users were right-handed. Because cannabis use can co-occur with alcohol use, independent samples t-tests examined whether cannabis users and non-users differed in alcohol use. Most respondents (72%; n = 44) reported using alcohol two times or less per month, and most (60.6%; n = 37) reported drinking two or fewer drinks per sitting. Analyses revealed that the two groups (cannabis users and non-users) did not significantly differ in frequency (p = .876) or amount (p = .987) of alcohol use. Procedure The researchers gave interested participants a brief preliminary eligibility screen (approximately 5–10 min) via online surveys and over the phone to ensure they met eligibility criteria. If met, an appointment for the neuropsychological testing was scheduled. Participants agreed to abstain from alcohol, marijuana, and other substances for at least 24 hrs prior to the appointment. They also agreed not to disclose their cannabis use status (user or non-user) to the examiner who completed the neuropsychological testing, so that examiners would remain blind to user status. Examiners for the study were comprised of doctoral-level graduate students with at least 1 year of training in neuropsychology and in the administration of the test battery. All examiners were familiar with research examining the impact of cannabis use on neurocognitive functioning. During the testing session, participants first completed informed consent. At the same time, examiners privately noted whether they believed the participant was a cannabis user or non-user, so that data could be analyzed for the expectancy effect confound. Examiners also administered a brief field sobriety test (i.e., standing on one foot for 30 s) to ensure participants were not under the influence of any substance at the time of testing. All participants passed this field sobriety test prior to completing the experiment. As part of a separate research study examining the effect of motivational statements on cognitive performance, examiners then administered one of two statements to the participant, randomly assigned prior to the beginning of the assessment session and identical to those previously described by Macher and Earleywine (2012). These motivational statement conditions were unrelated to the present study; therefore, we controlled for motivational condition in the statistical analyses. Participants then provided basic demographic information to the examiners, including age, date of birth, sex, ethnic identification, years of education, and handedness. Next, the examiners administered the neuropsychological test battery, which consisted of tests assessing a variety of cognitive domains, as well as performance validity tests (PVTs) to ensure that participants put forth adequate effort to perform well on the assessment, so that the results are known to be valid. The neuropsychological battery consisted of the California Verbal Learning Test – 2nd edition (CVLT-II; Delis, Kramer, Kaplan, & Ober, 2000), the Block Design, Digit Span, and Digit-Symbol Coding subtests of the Wechsler Adult Intelligence Scale – Third Edition (WAIS-III; Wechsler, 1997), the Rey–Osterrieth Complex Figure Test (RCF; Meyers & Meyers, 1995), the Trail Making Test of the Halstead–Reitan Neuropsychological Battery (TMT Part A and Part B; Reitan & Wolfson, 1992), and the National Adult Reading Test-Revised (NART-R; Blair & Spreen, 1989). Performance validity testing consisted of the Forced-Choice subtest of the CVLT-II (Moore & Donders, 2004), the Word Memory Test (WMT; Green, Allen, & Astner, 1996), and the Test of Memory Malingering (TOMM; Tombaugh, 1996), as well as the embedded measures of the Reliable Digit Span (RDS; Etherton, Bianchini, Greve, & Heinly, 2005) and the Trail Making Test Ratio (TMT B:A Ratio; Ruffolo, Guilmette, & Willis, 2000). All participants passed objective validity measures, indicating that the evaluation results were valid. Following the neuropsychological test battery, participants completed a two question self-report questionnaire assessing initial motivation to do well on the assessment, as well as potential change in motivation over time. Upon completion of the appointment, the examiners gave participants a $50 gift card as compensation. Research assistants de-identified all data prior to entering it into a statistical database (SPSS) for analyses. Once the neuropsychological assessments of all examinees were complete, each research examiner (n = 6) completed a survey regarding their own experiences and beliefs about the effects of cannabis use. This survey sought to assess whether the examiners’ personal beliefs might have influenced the interaction with the participant and subsequent performance on the neuropsychological assessment. Each examiner was randomly assigned an identification number and completed an anonymous and confidential survey that asked: “Assuming all other variables are equal (e.g., average baseline functioning, no comorbid diagnoses, etc.), to what extent (standard deviations above/below premorbid functioning) do you believe an individual’s Full Scale IQ in general would be affected, were the individual to report regular marijuana use (i.e., >4 times per week for at least 1 year)?” Response options ranged from −3 standard deviations (SDs) below baseline to +3 SDs above baseline, in 0.5 SDincrements. The same question was asked regarding the examiner’s beliefs about effects on individuals’ attention, memory, executive function, language, visuospatial/constructional skills, motor skills, processing speed, and performance validity/effort. Additionally, examiners answered questions about whether they believed their personal beliefs about cannabis user status might have influenced their interactions with participants and the participants’ subsequent performance on the neuropsychological assessment. Specifically, examiners answered whether they had ever used marijuana, whether they believed marijuana should be legalized recreationally, and whether they believed that their perceptions of the participants’ user status affected interactions with them. They also rated their confidence in their ability to guess whether participants were cannabis users or non-users. Results Accuracy of examiner judgment Analyses calculated the accuracy of examiners’ perceptions regarding participants’ cannabis use (i.e., cannabis user or non-user). The overall concordance of results was 63.9% (39/61: exact, 95% CI [50.6% –75.87%]), revealing that examiners accurately predicted cannabis use status at a rate better than chance (p = .04, as determined by an exact binomial test). Table 2 presents the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of examiners’ perceptions of participants’ cannabis use, along with 95% confidence intervals. Sensitivity was 68.29% (28/41: exact, 95% CI [51.91% –81.92%]), indicating that cannabis users were accurately identified as users at rates greater than chance (p = .028). However, the specificity was 55.00% (11/20: exact, 95% CI [31.53% –76.94%]), indicating that prediction of non-users did not differ from chance (p > .05). The PPV was 75.68% (28/37: exact, 95% CI [58.80% –88.23%]), indicating that about three-quarters of examiners’ cannabis user judgments were accurate, significantly above chance (p = .003). In contrast, the NPV was 45.83% (11/24: exact, 95% CI [25.55% –67.18%]), suggesting that less than half of examiners’ non-user judgments were accurate, which was not significantly more predictive than chance alone (p > .05). Within the user participants, there were no significant differences between examiner-judged users and examiner-judged non-users in age of onset of cannabis use, days per week of use, length of cannabis use, or amount of cannabis used per sitting (see Table 3). Table 2. Positive and negative predictive value, sensitivity, and specificity of examiners’ ratings of participants   User  Non-user  Total    Rated as user  28†  9  37  PPV = 75.68%  Rated as non-user  13‡  11  24  NPV = 45.83%  Total  41  20  61      Sens = 68.29%  Spec = 55.00%        User  Non-user  Total    Rated as user  28†  9  37  PPV = 75.68%  Rated as non-user  13‡  11  24  NPV = 45.83%  Total  41  20  61      Sens = 68.29%  Spec = 55.00%      Note: † = True Positives, ‡ = False Negatives, PPV = Positive Predictive Value, NPV = Negative Predictive Value, Sens = Sensitivity, Spec = Specificity. Table 2. Positive and negative predictive value, sensitivity, and specificity of examiners’ ratings of participants   User  Non-user  Total    Rated as user  28†  9  37  PPV = 75.68%  Rated as non-user  13‡  11  24  NPV = 45.83%  Total  41  20  61      Sens = 68.29%  Spec = 55.00%        User  Non-user  Total    Rated as user  28†  9  37  PPV = 75.68%  Rated as non-user  13‡  11  24  NPV = 45.83%  Total  41  20  61      Sens = 68.29%  Spec = 55.00%      Note: † = True Positives, ‡ = False Negatives, PPV = Positive Predictive Value, NPV = Negative Predictive Value, Sens = Sensitivity, Spec = Specificity. Table 3. Cannabis use data by user status and examiner-judged user status   Cannabis users  Examiner-judged users  Examiner-judged non-users  Age of onset of cannabis use (years)  16.35 (1.96)  16.19 (1.90)  16.69 (2.10)  Days per week of current cannabis (for true cannabis users only)  5.38 (1.53)  5.50 (1.42)  5.12 (1.78)  Length of cannabis use (years; for true cannabis users only)  2.36 (1.82)  2.35 (1.81)  2.38 (1.92)  Cannabis use per sitting (grams; for true cannabis users only)  1.76 (1.75)  1.96 (1.94)  1.37 (1.31)    Cannabis users  Examiner-judged users  Examiner-judged non-users  Age of onset of cannabis use (years)  16.35 (1.96)  16.19 (1.90)  16.69 (2.10)  Days per week of current cannabis (for true cannabis users only)  5.38 (1.53)  5.50 (1.42)  5.12 (1.78)  Length of cannabis use (years; for true cannabis users only)  2.36 (1.82)  2.35 (1.81)  2.38 (1.92)  Cannabis use per sitting (grams; for true cannabis users only)  1.76 (1.75)  1.96 (1.94)  1.37 (1.31)  Note: Means (standard deviations in parentheses). No group differences are significant at the p < .05 level. Table 3. Cannabis use data by user status and examiner-judged user status   Cannabis users  Examiner-judged users  Examiner-judged non-users  Age of onset of cannabis use (years)  16.35 (1.96)  16.19 (1.90)  16.69 (2.10)  Days per week of current cannabis (for true cannabis users only)  5.38 (1.53)  5.50 (1.42)  5.12 (1.78)  Length of cannabis use (years; for true cannabis users only)  2.36 (1.82)  2.35 (1.81)  2.38 (1.92)  Cannabis use per sitting (grams; for true cannabis users only)  1.76 (1.75)  1.96 (1.94)  1.37 (1.31)    Cannabis users  Examiner-judged users  Examiner-judged non-users  Age of onset of cannabis use (years)  16.35 (1.96)  16.19 (1.90)  16.69 (2.10)  Days per week of current cannabis (for true cannabis users only)  5.38 (1.53)  5.50 (1.42)  5.12 (1.78)  Length of cannabis use (years; for true cannabis users only)  2.36 (1.82)  2.35 (1.81)  2.38 (1.92)  Cannabis use per sitting (grams; for true cannabis users only)  1.76 (1.75)  1.96 (1.94)  1.37 (1.31)  Note: Means (standard deviations in parentheses). No group differences are significant at the p < .05 level. Differences in neuropsychological and PVT performance between examiner-judged users and examiner-judged non-users Table 4 presents a comparison of neuropsychological performance across four groups: those accurately judged as users, those accurately judged as non-users, those inaccurately judged as users, and those inaccurately judged as non-users. Importantly, direct comparisons are difficult to interpret given the known confounds that may have affected cognitive performance. Therefore, a multiple regression analyzed whether examiner belief of participants’ user status affected performance on neuropsychological and validity tests. Because demographic variables could act as confounds, age and gender were entered into the analysis. While independent samples t-tests revealed no significant differences between users and non-users or examiner-judged users and examiner-judged non-users in age (p > .05), there was a significantly (p < .01) larger representation of male cannabis users (n = 30 of 41 users; 73%) relative to female users (n = 11 of 41 users; 27%). This is consistent with expectation, as cannabis users are predominantly male (Johnston et al., 2015). Furthermore, bivariate correlational analysis revealed a significant (p < .02) association between gender and examiner expectancies. This association was anticipated given the examiners’ ability to accurately predict cannabis use and the larger prevalence of male cannabis users in the study. Therefore, gender was also included in the regression analyses. As described in the Method, motivational condition was included in the analysis to control for the potential effects of the motivational statement administered prior to testing, as this statement was unrelated to the present study. Finally, to isolate the effect of examiner belief of cannabis user status from actual cannabis user status, actual cannabis user status was included in the analysis. This permits the analysis of the effect of examiner belief of user status, above and beyond the effect of actual user status alone. Table 4. Neuropsychological and performance validity test performance by cannabis user status and examiner judgments   Cannabis users  Non-users    Examiner judged: user (accurate)  Examiner judged: non-user (inaccurate)  Examiner judged: non-user (accurate)  Examiner judged: user (inaccurate)  Neuropsychological measure   WMT Free Recall  23.75 (6.82)  25.38 (5.56)  26.73 (6.39)  25.56 (6.17)   RCF Copy  32.77 (3.41)  33.35 (2.77)  33.82 (1.66)  33.33 (2.41)   RCF Immediate Recall  18.57 (6.75)  17.54 (7.67)  21.14 (4.96)  22.17 (5.83)   RCF Delayed Recall  17.91 (7.16)  18.15 (7.78)  21.14 (4.23)  23.00 (5.44)   RCF Recognition  20.25 (1.97)  20.54 (1.81)  20.73 (1.62)  21.11 (2.42)   CVLT-II Trials 1-5 Free Recall  48.21 (9.60)  52.46 (8.50)  57.82 (7.47)  51.44 (8.82)   CVLT-II Trial B Free Recall  4.96 (1.53)  5.62 (1.66)  6.91 (1.81)  5.67 (2.24)   CVLT-II Short Delay Free Recall  9.86 (3.18)  11.69 (1.84)  12.27 (2.57)  11.56 (3.05)   CVLT-II Short Delay Cued Recall  10.71 (3.34)  11.77 (2.17)  13.45 (2.46)  11.56 (3.32)   CVLT-II Long Delay Free Recall  9.93 (3.22)  12.23 (2.39)  13.55 (2.46)  11.89 (3.30)   CVLT-II Long Delay Cued Recall  10.50 (3.31)  12.69 (2.46)  13.73 (2.20)  11.44 (3.47)   WAIS-III Digit Span  17.54 (2.73)  18.23 (4.21)  17.73 (4.45)  15.56 (2.88)   WAIS-III Block Design  45.96 (14.03)  46.08 (9.59)  46.73 (9.34)  48.78 (9.85)   WAIS-III Digit Symbol Coding  77.57 (15.40)  78.92 (16.77)  91.00 (8.40)  67.44 (9.03)   TMT A (in seconds; reverse scored)  24.79 (6.07)  24.31 (7.63)  21.82 (6.46)  33.00 (13.07)   TMT B (in seconds; reverse scored)  67.14 (20.29)  58.31 (16.43)  58.00 (12.30)  87.00 (34.66)  Performance Validity Measure   WMT IR  38.75 (1.35)  39.54 (.78)  39.55 (.93)  39.22 (.97)   WMT DR  39.00 (1.66)  39.31 (.75)  39.09 (1.45)  39.00 (1.32)   WMT CNS  38.32 (1.79)  38.85 (1.28)  38.64 (1.91)  38.67 (1.41)   RCF Combination Score  57.30 (6.96)  58.96 (6.76)  60.00 (5.87)  60.67 (8.74)   CVLT-II FC  15.96 (.19)  15.92 (.28)  16.00 (.00)  15.78 (.45)   RDS  11.75 (1.60)  12.08 (2.22)  12.00 (2.57)  10.67 (1.73)   TMT B/A Ratio  2.75 (.84)  2.52 (.82)  2.80 (.82)  2.70 (.67)   TOMM Trial 2  49.93 (.26)  49.85 (.38)  49.82 (.60)  50.00 (.00)   TOMM Retention Trial  49.71 (.85)  49.85 (.55)  49.91 (.30)  49.89 (.33)    Cannabis users  Non-users    Examiner judged: user (accurate)  Examiner judged: non-user (inaccurate)  Examiner judged: non-user (accurate)  Examiner judged: user (inaccurate)  Neuropsychological measure   WMT Free Recall  23.75 (6.82)  25.38 (5.56)  26.73 (6.39)  25.56 (6.17)   RCF Copy  32.77 (3.41)  33.35 (2.77)  33.82 (1.66)  33.33 (2.41)   RCF Immediate Recall  18.57 (6.75)  17.54 (7.67)  21.14 (4.96)  22.17 (5.83)   RCF Delayed Recall  17.91 (7.16)  18.15 (7.78)  21.14 (4.23)  23.00 (5.44)   RCF Recognition  20.25 (1.97)  20.54 (1.81)  20.73 (1.62)  21.11 (2.42)   CVLT-II Trials 1-5 Free Recall  48.21 (9.60)  52.46 (8.50)  57.82 (7.47)  51.44 (8.82)   CVLT-II Trial B Free Recall  4.96 (1.53)  5.62 (1.66)  6.91 (1.81)  5.67 (2.24)   CVLT-II Short Delay Free Recall  9.86 (3.18)  11.69 (1.84)  12.27 (2.57)  11.56 (3.05)   CVLT-II Short Delay Cued Recall  10.71 (3.34)  11.77 (2.17)  13.45 (2.46)  11.56 (3.32)   CVLT-II Long Delay Free Recall  9.93 (3.22)  12.23 (2.39)  13.55 (2.46)  11.89 (3.30)   CVLT-II Long Delay Cued Recall  10.50 (3.31)  12.69 (2.46)  13.73 (2.20)  11.44 (3.47)   WAIS-III Digit Span  17.54 (2.73)  18.23 (4.21)  17.73 (4.45)  15.56 (2.88)   WAIS-III Block Design  45.96 (14.03)  46.08 (9.59)  46.73 (9.34)  48.78 (9.85)   WAIS-III Digit Symbol Coding  77.57 (15.40)  78.92 (16.77)  91.00 (8.40)  67.44 (9.03)   TMT A (in seconds; reverse scored)  24.79 (6.07)  24.31 (7.63)  21.82 (6.46)  33.00 (13.07)   TMT B (in seconds; reverse scored)  67.14 (20.29)  58.31 (16.43)  58.00 (12.30)  87.00 (34.66)  Performance Validity Measure   WMT IR  38.75 (1.35)  39.54 (.78)  39.55 (.93)  39.22 (.97)   WMT DR  39.00 (1.66)  39.31 (.75)  39.09 (1.45)  39.00 (1.32)   WMT CNS  38.32 (1.79)  38.85 (1.28)  38.64 (1.91)  38.67 (1.41)   RCF Combination Score  57.30 (6.96)  58.96 (6.76)  60.00 (5.87)  60.67 (8.74)   CVLT-II FC  15.96 (.19)  15.92 (.28)  16.00 (.00)  15.78 (.45)   RDS  11.75 (1.60)  12.08 (2.22)  12.00 (2.57)  10.67 (1.73)   TMT B/A Ratio  2.75 (.84)  2.52 (.82)  2.80 (.82)  2.70 (.67)   TOMM Trial 2  49.93 (.26)  49.85 (.38)  49.82 (.60)  50.00 (.00)   TOMM Retention Trial  49.71 (.85)  49.85 (.55)  49.91 (.30)  49.89 (.33)  Note: Means (standard deviations in parentheses). Table 4. Neuropsychological and performance validity test performance by cannabis user status and examiner judgments   Cannabis users  Non-users    Examiner judged: user (accurate)  Examiner judged: non-user (inaccurate)  Examiner judged: non-user (accurate)  Examiner judged: user (inaccurate)  Neuropsychological measure   WMT Free Recall  23.75 (6.82)  25.38 (5.56)  26.73 (6.39)  25.56 (6.17)   RCF Copy  32.77 (3.41)  33.35 (2.77)  33.82 (1.66)  33.33 (2.41)   RCF Immediate Recall  18.57 (6.75)  17.54 (7.67)  21.14 (4.96)  22.17 (5.83)   RCF Delayed Recall  17.91 (7.16)  18.15 (7.78)  21.14 (4.23)  23.00 (5.44)   RCF Recognition  20.25 (1.97)  20.54 (1.81)  20.73 (1.62)  21.11 (2.42)   CVLT-II Trials 1-5 Free Recall  48.21 (9.60)  52.46 (8.50)  57.82 (7.47)  51.44 (8.82)   CVLT-II Trial B Free Recall  4.96 (1.53)  5.62 (1.66)  6.91 (1.81)  5.67 (2.24)   CVLT-II Short Delay Free Recall  9.86 (3.18)  11.69 (1.84)  12.27 (2.57)  11.56 (3.05)   CVLT-II Short Delay Cued Recall  10.71 (3.34)  11.77 (2.17)  13.45 (2.46)  11.56 (3.32)   CVLT-II Long Delay Free Recall  9.93 (3.22)  12.23 (2.39)  13.55 (2.46)  11.89 (3.30)   CVLT-II Long Delay Cued Recall  10.50 (3.31)  12.69 (2.46)  13.73 (2.20)  11.44 (3.47)   WAIS-III Digit Span  17.54 (2.73)  18.23 (4.21)  17.73 (4.45)  15.56 (2.88)   WAIS-III Block Design  45.96 (14.03)  46.08 (9.59)  46.73 (9.34)  48.78 (9.85)   WAIS-III Digit Symbol Coding  77.57 (15.40)  78.92 (16.77)  91.00 (8.40)  67.44 (9.03)   TMT A (in seconds; reverse scored)  24.79 (6.07)  24.31 (7.63)  21.82 (6.46)  33.00 (13.07)   TMT B (in seconds; reverse scored)  67.14 (20.29)  58.31 (16.43)  58.00 (12.30)  87.00 (34.66)  Performance Validity Measure   WMT IR  38.75 (1.35)  39.54 (.78)  39.55 (.93)  39.22 (.97)   WMT DR  39.00 (1.66)  39.31 (.75)  39.09 (1.45)  39.00 (1.32)   WMT CNS  38.32 (1.79)  38.85 (1.28)  38.64 (1.91)  38.67 (1.41)   RCF Combination Score  57.30 (6.96)  58.96 (6.76)  60.00 (5.87)  60.67 (8.74)   CVLT-II FC  15.96 (.19)  15.92 (.28)  16.00 (.00)  15.78 (.45)   RDS  11.75 (1.60)  12.08 (2.22)  12.00 (2.57)  10.67 (1.73)   TMT B/A Ratio  2.75 (.84)  2.52 (.82)  2.80 (.82)  2.70 (.67)   TOMM Trial 2  49.93 (.26)  49.85 (.38)  49.82 (.60)  50.00 (.00)   TOMM Retention Trial  49.71 (.85)  49.85 (.55)  49.91 (.30)  49.89 (.33)    Cannabis users  Non-users    Examiner judged: user (accurate)  Examiner judged: non-user (inaccurate)  Examiner judged: non-user (accurate)  Examiner judged: user (inaccurate)  Neuropsychological measure   WMT Free Recall  23.75 (6.82)  25.38 (5.56)  26.73 (6.39)  25.56 (6.17)   RCF Copy  32.77 (3.41)  33.35 (2.77)  33.82 (1.66)  33.33 (2.41)   RCF Immediate Recall  18.57 (6.75)  17.54 (7.67)  21.14 (4.96)  22.17 (5.83)   RCF Delayed Recall  17.91 (7.16)  18.15 (7.78)  21.14 (4.23)  23.00 (5.44)   RCF Recognition  20.25 (1.97)  20.54 (1.81)  20.73 (1.62)  21.11 (2.42)   CVLT-II Trials 1-5 Free Recall  48.21 (9.60)  52.46 (8.50)  57.82 (7.47)  51.44 (8.82)   CVLT-II Trial B Free Recall  4.96 (1.53)  5.62 (1.66)  6.91 (1.81)  5.67 (2.24)   CVLT-II Short Delay Free Recall  9.86 (3.18)  11.69 (1.84)  12.27 (2.57)  11.56 (3.05)   CVLT-II Short Delay Cued Recall  10.71 (3.34)  11.77 (2.17)  13.45 (2.46)  11.56 (3.32)   CVLT-II Long Delay Free Recall  9.93 (3.22)  12.23 (2.39)  13.55 (2.46)  11.89 (3.30)   CVLT-II Long Delay Cued Recall  10.50 (3.31)  12.69 (2.46)  13.73 (2.20)  11.44 (3.47)   WAIS-III Digit Span  17.54 (2.73)  18.23 (4.21)  17.73 (4.45)  15.56 (2.88)   WAIS-III Block Design  45.96 (14.03)  46.08 (9.59)  46.73 (9.34)  48.78 (9.85)   WAIS-III Digit Symbol Coding  77.57 (15.40)  78.92 (16.77)  91.00 (8.40)  67.44 (9.03)   TMT A (in seconds; reverse scored)  24.79 (6.07)  24.31 (7.63)  21.82 (6.46)  33.00 (13.07)   TMT B (in seconds; reverse scored)  67.14 (20.29)  58.31 (16.43)  58.00 (12.30)  87.00 (34.66)  Performance Validity Measure   WMT IR  38.75 (1.35)  39.54 (.78)  39.55 (.93)  39.22 (.97)   WMT DR  39.00 (1.66)  39.31 (.75)  39.09 (1.45)  39.00 (1.32)   WMT CNS  38.32 (1.79)  38.85 (1.28)  38.64 (1.91)  38.67 (1.41)   RCF Combination Score  57.30 (6.96)  58.96 (6.76)  60.00 (5.87)  60.67 (8.74)   CVLT-II FC  15.96 (.19)  15.92 (.28)  16.00 (.00)  15.78 (.45)   RDS  11.75 (1.60)  12.08 (2.22)  12.00 (2.57)  10.67 (1.73)   TMT B/A Ratio  2.75 (.84)  2.52 (.82)  2.80 (.82)  2.70 (.67)   TOMM Trial 2  49.93 (.26)  49.85 (.38)  49.82 (.60)  50.00 (.00)   TOMM Retention Trial  49.71 (.85)  49.85 (.55)  49.91 (.30)  49.89 (.33)  Note: Means (standard deviations in parentheses). Table 5 provides results of multiple regression analysis of neuropsychological and validity measures as a function of examiner judgments of cannabis user status, after adjusting for the participants’ age, gender, motivational condition, and actual cannabis user status. Examiner expectancies remained a significant predictor of several neuropsychological performance measures, including multiple subtests of the CVLT-II and the TMT B (p < .05), with effect sizes in the medium range (Cohen’s d = 0.75 and 0.61, respectively). These findings suggest that, even after controlling for potential confounds including user status, participants judged as cannabis users performed worse on two of the eight neuropsychological tests, compared to participants judged as non-users. Examiner expectancies were not a significant predictor on RCF, any of the three included WAIS-III subtests, TMT A, TOMM, or WMT (ps > .05). Table 5. Adjusted examiner expectancy effects on neuropsychological performance: controlling for age, gender, actual user status, and motivational condition Measure (Raw)  Term  Estimate  Standard error  t  p-Value  CVLT-II Trials 1-5 Free Recall  Examiner Belief  2.958  1.179  2.510  .015*  CVLT-II Trial B Free Recall  Examiner Belief  0.431  0.230  1.870  .066    User Status [0]  0.489  0.240  2.040  .046*  CVLT-II Short Delay Free Recall  Examiner Belief  0.559  0.377  1.480  .143    Gender [0]  −0.909  0.377  −2.410  .019*  CVLT-II Short Delay Cued Recall  Examiner Belief  0.811  0.392  2.070  .043*  CVLT-II Long Delay Free Recall  Examiner Belief  1.036  0.394  2.630  .011*    User Status [0]  0.830  0.410  2.030  .048*  CVLT-II Long Delay Cued Recall  Examiner Belief  1.218  0.391  3.120  .003**  WAIS-III Digit Symbol Coding  Examiner Belief  2.097  1.926  1.090  .281    Gender [0]  −5.562  1.890  −2.940  .005*    Motivation [0]  −4.038  1.797  −2.250  .0290*  TMT A (in seconds)  Examiner Belief  −1.809  1.075  −1.680  .098  TMT B (in seconds)  Examiner Belief  −6.693  2.766  −2.420  .019*    Age  1.888  0.822  2.300  .025*  WMT IR  Examiner Belief  0.277  0.146  1.890  .063    Motivation [0]  −0.297  0.143  −2.070  .043*  Measure (Raw)  Term  Estimate  Standard error  t  p-Value  CVLT-II Trials 1-5 Free Recall  Examiner Belief  2.958  1.179  2.510  .015*  CVLT-II Trial B Free Recall  Examiner Belief  0.431  0.230  1.870  .066    User Status [0]  0.489  0.240  2.040  .046*  CVLT-II Short Delay Free Recall  Examiner Belief  0.559  0.377  1.480  .143    Gender [0]  −0.909  0.377  −2.410  .019*  CVLT-II Short Delay Cued Recall  Examiner Belief  0.811  0.392  2.070  .043*  CVLT-II Long Delay Free Recall  Examiner Belief  1.036  0.394  2.630  .011*    User Status [0]  0.830  0.410  2.030  .048*  CVLT-II Long Delay Cued Recall  Examiner Belief  1.218  0.391  3.120  .003**  WAIS-III Digit Symbol Coding  Examiner Belief  2.097  1.926  1.090  .281    Gender [0]  −5.562  1.890  −2.940  .005*    Motivation [0]  −4.038  1.797  −2.250  .0290*  TMT A (in seconds)  Examiner Belief  −1.809  1.075  −1.680  .098  TMT B (in seconds)  Examiner Belief  −6.693  2.766  −2.420  .019*    Age  1.888  0.822  2.300  .025*  WMT IR  Examiner Belief  0.277  0.146  1.890  .063    Motivation [0]  −0.297  0.143  −2.070  .043*  Note: Significance level *.05, **.01. Bold p-values indicate significance for the Examiner Belief variable. Gender was coded as males = 0, females = 1. User status was coded as non-user = 0, user = 1. Motivational condition was coded as neutral statement = 0, motivational statement = 1. Table 5. Adjusted examiner expectancy effects on neuropsychological performance: controlling for age, gender, actual user status, and motivational condition Measure (Raw)  Term  Estimate  Standard error  t  p-Value  CVLT-II Trials 1-5 Free Recall  Examiner Belief  2.958  1.179  2.510  .015*  CVLT-II Trial B Free Recall  Examiner Belief  0.431  0.230  1.870  .066    User Status [0]  0.489  0.240  2.040  .046*  CVLT-II Short Delay Free Recall  Examiner Belief  0.559  0.377  1.480  .143    Gender [0]  −0.909  0.377  −2.410  .019*  CVLT-II Short Delay Cued Recall  Examiner Belief  0.811  0.392  2.070  .043*  CVLT-II Long Delay Free Recall  Examiner Belief  1.036  0.394  2.630  .011*    User Status [0]  0.830  0.410  2.030  .048*  CVLT-II Long Delay Cued Recall  Examiner Belief  1.218  0.391  3.120  .003**  WAIS-III Digit Symbol Coding  Examiner Belief  2.097  1.926  1.090  .281    Gender [0]  −5.562  1.890  −2.940  .005*    Motivation [0]  −4.038  1.797  −2.250  .0290*  TMT A (in seconds)  Examiner Belief  −1.809  1.075  −1.680  .098  TMT B (in seconds)  Examiner Belief  −6.693  2.766  −2.420  .019*    Age  1.888  0.822  2.300  .025*  WMT IR  Examiner Belief  0.277  0.146  1.890  .063    Motivation [0]  −0.297  0.143  −2.070  .043*  Measure (Raw)  Term  Estimate  Standard error  t  p-Value  CVLT-II Trials 1-5 Free Recall  Examiner Belief  2.958  1.179  2.510  .015*  CVLT-II Trial B Free Recall  Examiner Belief  0.431  0.230  1.870  .066    User Status [0]  0.489  0.240  2.040  .046*  CVLT-II Short Delay Free Recall  Examiner Belief  0.559  0.377  1.480  .143    Gender [0]  −0.909  0.377  −2.410  .019*  CVLT-II Short Delay Cued Recall  Examiner Belief  0.811  0.392  2.070  .043*  CVLT-II Long Delay Free Recall  Examiner Belief  1.036  0.394  2.630  .011*    User Status [0]  0.830  0.410  2.030  .048*  CVLT-II Long Delay Cued Recall  Examiner Belief  1.218  0.391  3.120  .003**  WAIS-III Digit Symbol Coding  Examiner Belief  2.097  1.926  1.090  .281    Gender [0]  −5.562  1.890  −2.940  .005*    Motivation [0]  −4.038  1.797  −2.250  .0290*  TMT A (in seconds)  Examiner Belief  −1.809  1.075  −1.680  .098  TMT B (in seconds)  Examiner Belief  −6.693  2.766  −2.420  .019*    Age  1.888  0.822  2.300  .025*  WMT IR  Examiner Belief  0.277  0.146  1.890  .063    Motivation [0]  −0.297  0.143  −2.070  .043*  Note: Significance level *.05, **.01. Bold p-values indicate significance for the Examiner Belief variable. Gender was coded as males = 0, females = 1. User status was coded as non-user = 0, user = 1. Motivational condition was coded as neutral statement = 0, motivational statement = 1. Differences in examiners’ judgments based upon prior cannabis experience To examine whether examiners’ judgments varied depending upon their own experience with cannabis, examiners (n = 6) completed a secured anonymous survey disclosing their own experiences with cannabis use. Two of the six examiners disclosed that they had previous experience using cannabis. Therefore, the authors analyzed whether examiners with cannabis experience differed in the accuracy of their judgments of participants’ user status, relative to examiners with no cannabis experience. Exclusion of examiners with cannabis experience from statistical analyses did not alter study conclusions: overall concordance remained significantly better than chance alone at 64.91% (37/57: exact 95%, CI [51.13% –77.09%]). Similarly, sensitivity excluding these two examiners was 68.42% (26/38: exact, 95% CI [51.35% –82.50%]), and PPV was 76.47% (26/34: exact, 95% CI [58.83% –89.25%]), indicating that even cannabis-naïve examiners accurately predicted cannabis use at rates better than chance (ps < .05). Both specificity at 57.89% (11/19: exact, 95% CI [33.50% –79.75%]) and NPV at 47.83% (11/23: exact, 95% CI [26.82% –69.41%]) remained non-significant, consistent with previous results. A larger sample of examiners would be required to achieve sufficient power to conduct subgroup analyses of examiners who have and have not previously used cannabis. However, the current results suggest that there were minimal differences in accuracy between these groups. Examiners’ beliefs about the effects of cannabis use To examine what factors may have contributed to examiners’ expectancies of neuropsychological performance, examiners also answered questions about their personal beliefs regarding the effect of chronic cannabis use on cognitive functioning. Examiners rated the extent to which they believed regular marijuana use (here defined as >4 times per week for at least 1 year; Gonzalez, Carey, & Grant, 2002) would affect cognitive functioning, based upon the expected number of standard deviations’ difference above or below premorbid (baseline) functioning. For example, a mean examiner rating of −1.0 suggests that examiners believed cannabis users would perform about one standard deviation below baseline on neuropsychological testing due to the effects of regular cannabis use. Examiners’ ratings illustrated that they believed cannabis would have the greatest effect on processing speed (x− = −0.67, SD = 0.26), attention (x− = −0.50, SD = 0.0), memory (x− = −0.42, SD = 0.20), and motor skills (x− = −0.33, SD = 0.41), with negligible effects on executive functioning (x− = −0.25, SD = 0.42), Full Scale IQ (x− = −0.17, SD = 0.26), language (x− = −0.08, SD = 0.20), visuospatial construction (x− = –0.08, SD = 0.20), and effort to perform well on testing (x− = 0.17, SD = 0.41). It is interesting to note that examiners did not anticipate that cannabis would have more of an effect than about a half a standard deviation below baseline functioning on any cognitive domains, indicating that they anticipated a relatively small effect on cognition. Examiners also reported whether they support the legalization of recreational cannabis use. Half (3/6) reported “Yes,” whereas two examiners (33.3%) stated “Maybe, under certain regulations” and one reported “No” (16.7%). Therefore, most of the examiners viewed cannabis legalization favorably. Examiners also rated how confident they felt about the accuracy of their judgments regarding participant cannabis user status. The mean confidence level across examiners’ ratings of participants was 47.33% (SD = 35.34%), indicating that most examiners did not believe their judgments of cannabis user status were more accurate than chance. Further, examiners answered whether they believed their perceptions of the participants’ cannabis user status affected interactions with participants. Out of the six examiners, one (16.7%) reported “Maybe,” whereas three (50.0%) reported “Probably not,” one (16.7%) reported “Definitely not,” and one (16.7%) reported “Almost certainly not.” Thus, most examiners did not believe they were accurate in judging user status and did not believe their perceptions affected interactions with the participants, despite the fact that our results suggest both judgment accuracy and an effect on participant performance. Additional exploratory analyses analyzed whether examiners’ personal expectations of chronic cannabis effects on neuropsychological performance was correlated with actual examinee performance. While most correlations were not significant, there was a positive correlation between examiners’ expectations that memory would be affected by chronic cannabis use and examinees’ performance on the CVLT-II Delayed Free Recall (r = 0.299, p = .02). This relationship indicates that examiners’ expectations of lower memory performance were in fact associated with poorer performance on a memory measure. This correlation aligns with the previously described finding that examiner judgment of user status predicted CVLT-II performance. However, this finding should be interpreted with caution, as most correlations were not statistically significant. Discussion Results from the current study provide evidence that examiners’ judgments of participant cannabis user status may in fact influence examinees’ performance on neuropsychological assessment. Even after controlling for confounding variables (age, gender, actual user status, and motivational condition), individuals judged to be cannabis users performed worse on two of eight neuropsychological tests, relative to individuals judged as non-users. These results suggest that examiners’ perceptions of cannabis user status represent a potential confound that can significantly affect the results of examinee neuropsychological performance. Interestingly, there was a trend across several other neuropsychological subtests following the same pattern, with differences between those judged as users and those as non-users reaching marginal significance (p < .10) after accounting for actual user status and other confounds. Indeed, there was a medium effect of examiner belief on the CVLT-II Trial B Free Recall (Cohen’s d = 0.59), TMT A (Cohen’s d = 0.43), and the WMT Immediate Recall (Cohen’s d = 0.59). Thus, this trend suggests that even after controlling for actual user status, participants judged as users performed worse on many of the neuropsychological tests compared to those judged as non-users. While the trend should be interpreted with caution, it does suggest that greater power (e.g., a larger sample size) could result in statistically significant group differences between examiner-judged users and examiner-judged non-users on even more tests in the battery. Within the present study, examiners accurately judged participant cannabis user status at rates better than chance (sensitivity = 68%), and about three-quarters of examiners’ cannabis user judgments were accurate (PPV = 76%). These results are consistent with previous findings suggesting that laymen in general, as well as neuropsychologists specifically, can discriminate between cannabis users and non-users based upon appearance alone (Hirst et al., 2016, 2017). Importantly, characteristics of the examiners in this study are consistent with those of researchers analyzing the effects of cannabis on cognition: they had at least one year of training in neuropsychological assessment specifically, and were familiar with empirical literature examining the cognitive effects of cannabis. Thus, their accuracy rates are consistent with those of other cannabis researchers, as well as neuropsychologists in general (Hirst et al., 2016). Conversely, non-users’ statuses were not judged at a rate above chance levels (specificity = 55%) and only about half of the non-user predictions were accurate (NPV = 46% accuracy). Due to the comparatively smaller number of non-users (n = 20, vs. n = 41 cannabis users) and individuals judged as non-users (n = 24, vs. n = 37 judged as users), it is possible that greater statistical power would permit more reliable estimations of specificity and NPV. Studies including larger samples of both non-users and individuals judged to be non-users might have greater power to determine whether non-users can be judged accurately. Regardless, in this sample, judgment accuracy rates were clearly much higher in the user group relative to the non-user group. Therefore, the present findings suggest that examiner expectancy effects are more likely to negatively affect cannabis users because they are more accurately judged as users. It is important to note that although examiners were more accurate in judging users than non-users, controlling for actual user status in the regression permits analysis of only the expectancy effect and its impact on cognitive performance. Thus, the expectancy effect shown was above and beyond the impact of actual cannabis user status on neuropsychological functioning. While several previous studies examining the cognitive effects of cannabis utilize a research paradigm in which examiners are blind to participants’ user status, the present findings suggest that these studies remain vulnerable to the expectancy effect confound if examiners are, consciously or unconsciously, guessing participant user status. Although the sample of examiners was too small for formal statistical comparisons, exploratory analyses examined the examiners’ expectations about the effects of cannabis use. The examiners in this study reported that they expected users to perform only about 0.5 standard deviations below baseline in the domains of memory, attention, and processing speed. It is particularly intriguing that the present findings revealed an expectancy effect confound, given that examiners anticipated only small effects of cannabis users on neuropsychological performance and generally viewed cannabis legalization favorably. Further exploratory analyses revealed that examiners’ expectations of memory effects correlated with examinees’ performance on one of the memory measures (CVLT-II Delayed Free Recall). This relationship is consistent with the finding that examiner judgment of user status predicted performance on the CVLT-II. While these results should be interpreted with caution, as most correlations were not significant, this finding does raise the possibility that examiners’ personal beliefs regarding cannabis effects could contribute to the expectancy effect confound on measures of memory. In addition, most examiners did not believe their accuracy in judging user status was better than chance, and also did not believe that their perceptions about user status affected their interactions with participants. Again, these responses make it even more interesting that statistically significant expectancy effects were identified. The present findings are consistent with a similar line of research that investigated the impact of examiner expectancies on examinee cognitive performance following caffeine administration. Results from that study showed that examiners’ expectations regarding the effects of caffeine administration influenced the examinees’ physiological response (i.e., heart rate), as well as performance on neuropsychological assessment (Walach et al., 2001). Taken together with the current results, it is apparent that examiner expectations can influence examinees’ performance, even with the standardized administration utilized in neuropsychological assessment. Furthermore, these findings allude to important clinical implications as well. Clinicians have access to information regarding patient cannabis use history; therefore, it is possible that neuropsychologists’ personal beliefs regarding the effects of cannabis on cognitive functioning could affect patients’ performance on clinical neuropsychological evaluations. Clinicians’ expectancies may, consciously or unconsciously, influence test administration, thus affecting the interpretation of test results, recommendations, or clinical decision-making. Expectancy effects are vastly under-studied in the field of neuropsychology, as the authors were unable to identify any other studies measuring examiner expectancy effects on examinee neuropsychological performance. The current findings suggest that the field of neuropsychology would benefit from further exploration of examiner expectancy effects in all populations, including individuals with other types of substance use, psychiatric disorders, or neurological disorders, as expectancy effects could occur in any study using neuropsychological assessment. One interpretation of the present findings may be to suggest minimizing expectancy effects with computerized administration of neuropsychological testing, as has become increasingly prevalent. In this study, there was no statistically significant expectancy effect shown on the WMT (a computer-administered test). However, the medium effect (Cohen’s d = 0.59) of examiner belief on WMT performance suggests that even computerized tests may not be immune to experimenter expectancy effects. Further, research suggests that computerized administration is not as sensitive to subtle cognitive deficits as clinician-administered, paper-and-pencil testing (e.g., Register-Mihalik et al., 2013). In clinical settings, the benefit of clinical judgment throughout the evaluation process cannot be understated in determining neuropsychological functioning (Bauer et al., 2012). Given that the present findings on computerized testing showed a medium effect of examiner expectancies but no statistically significant group differences, future studies with greater power are needed to directly compare participants’ performance on computer-administered tests to examiner-administered tests to determine whether expectancy effects are truly reduced in participants completing cognitive tests on a computer. However, the possibility that this was a spurious result cannot be ruled out. There are several limitations to note within the present study. This study sought to recruit “pure” cannabis users (i.e., those with no other significant substance use or comorbid medical/psychiatric disorders) in an attempt to limit other confounding variables that may potentially affect cognitive functioning. Thus, the “pure” cannabis user sample in this study may not be representative of the “typical” cannabis user, who may have comorbid substance use or other diagnoses. Additionally, a larger sample size in the present study may have revealed expectancy effects on more assessments within the battery, due to greater power. Another limitation of the present study may be that the authors did not objectively verify the accuracy of self-reported cannabis use or other eligibility criteria. However, as interested individuals were unaware of eligibility criteria during the phone screen, they had little reason to purposely misrepresent their substance use or other history. Additionally, data regarding examiners’ personal history of cannabis use was anonymously collected and de-identified after all examinees’ data were collected, to reduce the threat to integrity of accurate examiner self-reporting. In these ways, we attempted to minimize the impact of potential inaccurate estimations of cannabis use. Another limitation of the present study was its correlational analysis, with no direct manipulation of examiner expectancies. However, the authors purposely selected this correlational design in order to more closely approximate the authentic researcher/clinician and participant/patient relationship that occurs in research studies and clinical contexts. In real-world situations, examiners indeed present with their own inherent personal beliefs about cannabis effects and thus are not directly told how to perceive participants/patients. In addition, directly manipulating examiners’ expectancies of examinee performance could have ethical implications that are not desirable. Further, given that examiners’ expectancies were not manipulated, it is even more interesting that significant expectancy effects were found. It is possible that the act of guessing whether participants were cannabis users or non-users prior to testing increased examiners’ vulnerability to the expectancy effect confound by making their expectancies more salient during the testing process. We chose this design to ensure that examiners’ judgments were made based upon participants’ appearance rather than test performance; however, future studies could examine whether rating participants after testing results in larger or smaller expectancy effects. A larger sample of examiners in future studies would also allow for greater confidence in the conclusions of the exploratory analyses run in this study. The present study is also limited by a lack of measurement of the physical factors examiners used to make their cannabis use judgments, and to the authors’ knowledge, this has not been examined in any prior studies. Future studies should assess these factors in more detail to determine what visual cues are considered when deciding individuals’ likelihood of cannabis use. Finally, future studies would also benefit from measuring examiners’ other perceptions about participants (e.g., mood, intellect, other substance use) to determine how it relates to cannabis use perceptions as well as participants’ cognitive performance. The present findings shed light on the impact of examiner expectancy effects on the neuropsychological performance of chronic cannabis users. Examiners who were blind to participants’ user status accurately identified cannabis users at rates better than chance, and individuals judged as cannabis users performed worse on several neuropsychological tests even after controlling for potential confounding variables. While previous studies have utilized a blind research design in an attempt to control for examiner bias, it is important to realize that this approach is inadequate if examiners are able to guess participant user status. The results of the current study suggest that even studies using blind examiners are vulnerable to the expectancy effect confound, and that these expectancies can influence examinee performance and thus, the validity of study outcomes. Future research should assess for examiner expectancy effects by measuring examiners’ ratings of participant group status and including those ratings as a covariate in analyses to determine whether examiner expectancies did in fact influence study participants’ neuropsychological performance. This is a vastly under-studied topic in the field of neuropsychology, and the present results emphasize the need for further exploration of this topic in additional populations. Conflict of interest None declared. References American Psychiatric Association. ( 2000). Diagnostic and Statistical Manual of Mental Disorders–Fourth Edition-Text Revision (DSM–IV-TR) . Washington D.C.: American Psychiatric Association. Battisti, R. A., Roodenrys, S., Johnstone, S. J., Respondek, C., Hermens, D. F., & Solowij, N. ( 2010). Chronic use of cannabis and poor neural efficiency in verbal memory ability. Psychpharmacology , 209, 319– 330. doi:10.1007/s00213-010-1800-4. Google Scholar CrossRef Search ADS   Bauer, R. M., Iverson, G. L., Cernich, A. N., Binder, L. M., Ruff, R. M., & Naugle, R. I. ( 2012). Computerized neuropsychological assessment devices: Joint position paper of the American Academy of Clinical Neuropsychology and the National Academy of Neuropsychology. Archives of Clinical Neuropsychology , 27, 362– 373. doi:10.1093/arclin/acs027. Google Scholar CrossRef Search ADS PubMed  Blair, J. R., & Spreen, O. ( 1989). Predicting premorbid IQ: A revision of the National Adult Reading Test. The Clinical Neuropsychologist , 3, 129– 136. doi:10.1080/13854048908403285. Google Scholar CrossRef Search ADS   Bolla, K. I., Brown, K., Eldreth, D., Tate, K., & Cadet, J. L. ( 2002). Dose-related neurocognitive effects of marijuana use. Neurology , 59, 1337– 1343. doi:10.1212/01.WNL.0000031422.66442.49. Google Scholar CrossRef Search ADS PubMed  Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. ( 2000). California Verbal Learning Test  ( 2nd ed.). San Antonio, TX: Psychological Corporation. Dougherty, D. M., Mathias, C. W., Dawes, M. A., Furr, R. M., Charles, N. E., Liguori, A., et al.  . ( 2013). Impulsivity, attention, memory, and decision-making among adolescent marijuana users. Psychopharmacology , 226, 307– 319. doi:10.1007/s00213-012-2908-5. Google Scholar CrossRef Search ADS PubMed  Etherton, J. L., Bianchini, K. J., Greve, K. W., & Heinly, M. T. ( 2005). Sensitivity and specificity of Reliable Digit Span in malingered pain-related disability. Assessment , 12, 130– 136. doi:10.1177/1073191105274859. Google Scholar CrossRef Search ADS PubMed  Fink, B., Neave, N., Manning, J. T., & Grammer, K. ( 2006). Facial symmetry and judgments of attractiveness, health and personality. Personality and Individual Differences , 41, 491– 499. doi:10.1016/j.paid.2006.01.017. Google Scholar CrossRef Search ADS   Gonzalez, R., Carey, C., & Grant, I. ( 2002). Nonacute (residual) neuropsychological effects of cannabis use: A qualitative analysis and systematic review. Journal of Clinical Pharmacology , 42, 48S– 57S. doi:10.1002/j.1552-4604.2002.tb06003.x. Google Scholar CrossRef Search ADS PubMed  Grant, I., Gonzalez, R., Carey, C. I., Natarajan, L., & Wolfson, T. ( 2003). Nonacute (residual) neurocognitive effects of cannabis use: A meta analytic study. Journal of the International Neuropsychological Society , 9, 679– 689. doi:10.1017/S1355617703950016. Google Scholar CrossRef Search ADS PubMed  Green, P., Allen, L. M., & Astner, K. ( 1996). The Word Memory Test: A manual for the oral and computerized forms. . Durham, NC: Cognisyst, Inc. Gruber, S. A., Sagar, K. A., Dahlgren, M. K., Racine, M., & Lukas, S. E. ( 2012). Age of onset of marijuana use and executive function. Psychology of Addictive Behaviors , 26, 496– 506. doi:10.1037/a0026269. Google Scholar CrossRef Search ADS PubMed  Hirst, R. B., Teague, A., Sodos, L., Wickham, R. E., Whittington, L. T., Mills, B., et al.  . ( 2016). Determining cannabis use status from a photograph: An assessment of the “jay-dar” in neuropsychologists. Substance Use and Misuse , 52, 401– 410. doi:10.1080/10826084.2016.1233564. Google Scholar CrossRef Search ADS PubMed  Hirst, R. B., Enriquez, R., Wickham, R. E., Gretler, J., Sodos, L. M., Gade, S. A., et al.  . ( 2017). Marijuana stereotypes and the “jay-dar”: Perceptions of cannabis use and memory abilities based upon appearance. Personality and Individual Differences , 110, 131– 138. doi:10.1016/j.paid.2016.12.056. Google Scholar CrossRef Search ADS   Johnston, L. D., O’Malley, P. M., Bachman, J. G., Schulenberg, J. E., & Miech, R. A. ( 2015). Monitoring the future national survey results on drug use, 1975–2014: Volume II, college students and adults ages 19-55. Ann Arbor: Institute for Social research, The University of Michigan. Retrieved from. http://www.monitoringthefuture.org/pubs/monographs/mtf-vol2_2014.pdf. Karver, M. S., Handelsman, J. B., Fields, S., & Bickman, L. ( 2005). A theoretical model of common process factors in youth and family therapy. Mental Health Services Research , 7, 35– 51. doi:10.1007/s11020-005-1964-4. Google Scholar CrossRef Search ADS PubMed  Lyketsos, C. G., Garrett, E., Liang, K. Y., & Anthony, J. C. ( 1999). Cannabis use and cognitive decline in persons under 65 years of age. American Journal of Epidemiology , 149, 794– 800. doi:10.1093/oxfordjournals.aje.a009894. Google Scholar CrossRef Search ADS PubMed  Lyons, M. J., Bar, J. L., Panizzon, M. S., Toomey, R., Eisen, S., Xian, H., et al.  . ( 2004). Neuropsychological consequences of regular marijuana use: A twin study. Psychological Medicine , 34, 1239– 1250. doi:10.1017/S0033291704002260. Google Scholar CrossRef Search ADS PubMed  Macher, R. B., & Earleywine, M. ( 2012). Enhancing neuropsychological performance in chronic cannabis users: The role of motivation. Journal of Clinical and Experimental Neuropsychology , 34, 405– 415. doi:10.1080/13803395.2011.646957. Google Scholar CrossRef Search ADS PubMed  Mahmood, O. M., Jacobus, J., Bava, S., Scarlett, A., & Tapert, S. F. ( 2010). Learning and memory performances in adolescent users of alcohol and marijuana: Interactive effects. Journal of Studies on Alcohol and Drugs , 71, 885– 894. doi:10.15288/jsad.2010.71.885. Google Scholar CrossRef Search ADS PubMed  Medina, K. L., Hanson, K. L., Schweinsburg, A. D., Cohen-Zion, M., Nagel, B. J., & Tapert, S. F. ( 2007). Neuropsychological functioning in adolescent marijuana users: Subtle deficits detectable after a month of abstinence. Journal of the International Neuropsychological Society , 13, 807– 820. doi:10.1017/S1355617707071032. Google Scholar CrossRef Search ADS PubMed  Meier, M. H., Caspi, A., Ambler, A., Harrington, H., Houts, R., Keefe, R. S. E., et al.  . ( 2012). Persistent cannabis users show neuropsychological decline from childhood to midlife. Proceedings of the National Academy of Sciences , 109, E2657– E2664. doi:10.1073/pnas.1206820109. Google Scholar CrossRef Search ADS   Meyers, J. E., & Meyers, K. R. ( 1995). Rey Complex Figure Test and Recognition Trial, professional manual. . Odessa, FL: Psychological Assessment Resources (PAR). Moore, B. A., & Donders, J. ( 2004). Predictors of invalid neuropsychological test performance after traumatic brain injury. Brain Injury , 18, 975– 984. doi:10.1080/02699050410001672350. Google Scholar CrossRef Search ADS PubMed  Olivola, C. Y., & Todorov, A. ( 2010). Fooled by first impressions? Reexamining the diagnostic value of appearance-based inferences. Journal of Experimental Social Psychology , 46, 315– 324. doi:10.1016/j.jesp.2009.12.002. Google Scholar CrossRef Search ADS   Pope, H. G., Gruber, A. J., Hudson, J. I., Huestis, M. A., & Yurgelun-Todd, D. ( 2001). Neuropsychological performance in long-term cannabis users. Archives of General Psychiatry , 58, 909– 915. doi:10.1001/archpsyc.58.10.909. Google Scholar CrossRef Search ADS PubMed  Register-Mihalik, J. K., Guskiewicz, K. M., Mihalik, J. P., Schmidt, J. D., Kerr, Z. Y., & McCrea, M. A. ( 2013). Reliable change, sensitivity, and specificity of a multidimensional concussion assessment battery: Implications for caution in clinical practice. Journal of Head Trauma Rehabilitation , 28, 274– 283. doi:10.1097/HTR.0b013e3182585d37. Google Scholar CrossRef Search ADS PubMed  Rodgers, J. ( 2000). Cognitive performance among recreational users of “ecstasy. Psychopharmacology , 151, 19– 24. doi:10.1007/s002130000467. Google Scholar CrossRef Search ADS PubMed  Rosenthal, R. ( 1991). Meta-analytic procedures for social research . Beverly Hills, CA: Sage. Google Scholar CrossRef Search ADS   Rosenthal, R. ( 2002). Experimenter and clinician effects in scientific inquiry and clinic practice. Prevention and Treatment , 5. doi.org/10.1037/1522-3736.5.1.538c. Rosenthal, R., Kohn, P., Greenfield, P. M., & Carota, N. ( 1966). Data desirability, experimenter expectancy, and the results of psychological research. Journal of Personality and Social Psychology , 3, 20– 27. doi:10.1037/h0022604. Google Scholar CrossRef Search ADS PubMed  Reitan, R. M., & Wolfson, D. ( 1992). The Halstead-Reitan Neuropsychological Battery: Theory and clinical interpretation  ( (2nd ed.)). South Tucson, AZ: Neuropsychology Press. Ruffolo, L. F., Guilmette, T. J., & Willis, G. W. ( 2000). Comparison of time and error rates on the Trail Making Test among patients with head injuries, experimental malingerers, patients with suspect effort on testing, and normal controls. The Clinical Neuropsychologist , 14, 223– 230. doi:10.1076/1385-4046(200005)14:2;1-Z;FT223. Google Scholar CrossRef Search ADS PubMed  Schreiner, A. M., & Dunn, M. E. ( 2012). Residual effects of cannabis use on neurocognitive performance after prolonged abstinence: A meta-analysis. Experimental and Clinical Psychopharmacology , 20, 420– 429. doi:10.1037/a0029117. Google Scholar CrossRef Search ADS PubMed  Shelp, S. G. ( 2003). Gaydar. Journal of Homosexuality , 44 ( 1), 1– 14. doi:10.1300/J082v44n01_01. Google Scholar CrossRef Search ADS PubMed  Solowij, N. ( 1995). Do cognitive impairments recover following cessation of cannabis use? Life Sciences , 56, 2119– 2126. doi:10.1016/0024-3205(95)00197-E. Google Scholar CrossRef Search ADS PubMed  Solowij, N., Stephens, R. S., Roffman, R. A., Babor, T., Kadden, R., Miller, M., et al.  . ( 2002). Cognitive functioning of long term heavy cannabis users seeking treatment. Journal of the American Medical Association , 287, 1123– 1131. doi:10.1001/jama.287.9.1123. Google Scholar CrossRef Search ADS PubMed  Todorov, A., Pakrashi, M., & Oosterhof, N. N. ( 2009). Evaluating faces on trustworthiness after minimal time exposure. Social Cognition , 27, 813– 833. doi:10.1521/soco.2009.27.6.813. Google Scholar CrossRef Search ADS   Tombaugh, T. N. ( 1996). Test of Memory Malingering . Toronto, Ont: Multihealth Systems. United Nations Office on Drugs and Crime. ( 2016). World Drug Report 2016 (United Nations publication, Sales No. E.16.XI.7) . Washington, DC: U.S. Government Printing Office. Walach, H., Schimdt, S., Bihr, Y.-M., & Wiesch, S. ( 2001). The effects of a caffeine placebo and experimenter expectation on blood pressure, heart rate, well-being, and cognitive performance. European Psychologist , 6, 15– 25. doi:10.1027//1016-9040.6.1.15. Google Scholar CrossRef Search ADS   Wechsler, D. ( 1997). Wechsler Adult Intelligence Scale-Third Edition . San Antonio, TX: The Psychological Corporation. Wiens, A. N., Bryan, J. E., & Crossen, J. R. ( 1993). Estimating WAIS-R FSIQ from National Adult Reading Test-Revised in normal subjects. The Clinical Neuropsychologist , 7, 70– 84. doi:10.1080/13854049308401889. Google Scholar CrossRef Search ADS   Willis, J., & Todorov, A. ( 2006). First impressions: Making up your mind after a 100-ms exposure to a face. Psychological Science , 17, 592– 598. doi:10.1111/j.1467-9280.2006.01750.x. Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Archives of Clinical Neuropsychology Oxford University Press

Don’t Judge a Book by its Cover: Examiner Expectancy Effects Predict Neuropsychological Performance for Individuals Judged as Chronic Cannabis Users

Loading next page...
 
/lp/ou_press/don-t-judge-a-book-by-its-cover-examiner-expectancy-effects-predict-10YtcxgbpP
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.
ISSN
0887-6177
eISSN
1873-5843
D.O.I.
10.1093/arclin/acx114
Publisher site
See Article on Publisher Site

Abstract

Abstract Objective The experimenter expectancy effect confound remains largely unexplored in neuropsychological research and has never been investigated among cannabis users. This study investigated whether examiner expectancies of cannabis user status affected examinees’ neuropsychological performance. Method Participants included 41 cannabis users and 20 non-users. Before testing, examiners who were blind to participant user status privately rated whether they believed the examinee was a cannabis user or non-user. Examiners then administered a battery of neuropsychological and performance validity measures. Multiple regression analyses compared performance between examinees judged as cannabis users (n = 37) and those judged as non-users (n = 24). Results Examiners’ judgments of cannabis users were 75% accurate; judgments of non-users were at chance. After controlling for age, gender, and actual user status, examiner judgments of cannabis user status predicted performance on two measures (California Verbal Learning Test-II, and Trail Making Test B; p < .05), as individuals judged as cannabis users obtained lower scores than those judged as non-users. Conclusions Examiners’ judgments of cannabis user status predicted performance even after controlling for actual user status, indicating vulnerability to examiner expectancy effects. These findings have important implications for both research and clinical settings, as scores may partially reflect examiners’ expectations regarding cannabis effects rather than participants’ cognitive abilities. These results demonstrate the need for expectancy effect research in the neuropsychological assessment of all populations, not just cannabis users. Malingering/symptom validity testing, Learning and memory, Assessment, Drug and alcohol abuse Introduction There is increasing importance in understanding the effects of cannabis use on neurocognitive functioning. Cannabis use has increased globally in adult populations, both medically and recreationally, and is currently one of the most widely used drugs worldwide, with an estimated 183 million people using it (United Nations Office on Drugs and Crime, 2016). Previous research investigating the effect of cannabis on cognition has produced mixed results regarding long-term use outcomes and the potential detrimental effects on cognition. Although some studies have suggested that cannabis negatively affects cognitive functioning in various domains among long-term users (Grant, Gonzalez, Carey, Natarajan, & Wolfson, 2003; Solowij et al., 2002), others found no cognitive differences between users and non-users (Lyketsos, Garrett, Liang, & Anthony, 1999; Schreiner & Dunn, 2012). Further, even in those studies that find cognitive effects, there is inconsistency across studies regarding which cognitive domains are affected by cannabis use. Studies have found poorer performance among cannabis users on tasks of attention, processing speed (Medina et al., 2007), learning and memory (Bolla, Brown, Eldreth, Tate, & Cadet, 2002; Grant et al., 2003; Meier et al., 2012; Solowij et al., 2002), and executive functioning (Bolla et al., 2002; Gruber, Sagar, Dahlgren, Racine, & Lukas, 2012). Therefore, neuropsychological performance differences of cannabis users relative to non-users vary widely from study to study. Researchers have suggested that the variability in findings may be due to the wide variety of research designs implemented (e.g., variability in what criteria constitutes chronic cannabis use, age of onset of cannabis use, or length of abstinence periods in participants), as well as methodological limitations, including failure to control for alcohol use, lack of data gathered regarding premorbid neurocognitive functioning, and failure to report cannabis abstinence periods (Schreiner & Dunn, 2012). One major potential confound that has yet to be explored in cannabis research is the expectancy effect. Expectancy effects occur when experimenters’ personal expectations about study outcomes influence participants’ performance (Rosenthal, 2002). Experimenters’ expectations in research studies can lead to subtle differences in interactions with the participant, resulting in a shift in the participant’s behavior in a manner that confirms the researchers’ hypotheses (Rosenthal, Kohn, Greenfield, & Carota, 1966). The expectancy effect confound subsequently increases the probability of invalid or inaccurate findings. For example, researchers who expect cannabis use to be associated with memory deficits may unconsciously act in subtly different ways towards cannabis user participants relative to non-user participants, such as allowing less time for responding during memory tests or offering less encouragement. These differences could influence rapport and thus alter participants’ performance on the testing (Karver, Handelsman, Fields, & Bickman, 2005). To minimize the potential for expectancy effects, Rosenthal (1991) strongly recommends utilizing blind research designs, i.e.,keeping experimenters unaware of participants’ group status. Despite the extensive literature on the expectancy effect confound, cannabis researchers often fail to implement blind research designs when examining the cognitive effects of cannabis use (e.g., Battisti et al., 2010; Dougherty et al., 2013; Mahmood, Jacobus, Bava, Scarlett, & Tapert, 2010; Rodgers, 2000; Solowij, 1995). It is important to note that even studies utilizing blind research designs have yielded mixed results regarding the effects of cannabis use on cognition (Lyons et al., 2004; Pope, Gruber, Hudson, Huestis, & Yurgelun-Todd, 2001). One possible reason for these mixed findings may be due to researchers guessing, either consciously or unconsciously, which user group participants belong to, thus still leaving the studies vulnerable to expectancy effects. Studies have shown that individuals can guess qualities like personality and behavioral traits based simply upon appearance alone, such as by examining a photograph (Fink, Neave, Manning, & Grammer, 2006; Todorov, Pakrashi, & Oosterhof, 2009; Willis & Todorov, 2006). For example, research shows individuals are able to discriminate substance users versus non-users based on a photograph alone, with near 60% accuracy (Olivola & Todorov, 2010). Further, research by Hirst and colleagues (Hirst et al., 2016; 2017) supports the phenomenon of the “jay-dar” – the ability to distinguish whether an individual uses marijuana (i.e., smoking joints or “jays”), similar to Shelp’s (2003) research on “gay-dar.” In these studies, researchers asked both undergraduates (Hirst et al., 2017) and neuropsychologists (i.e., the same profession of those who might conduct research on cannabis users; Hirst et al., 2016) to rate the likelihood that a person was a cannabis user based on photographs alone. Photos of actual cannabis users (those who had used cannabis at least 400 times in their life, wearing their typical clothing and hairstyle) received higher ratings than photos of non-users on the Marijuana Use Likelihood Index, a 7-point Likert scale range from “least likely” to “most likely.” Effect size estimates revealed a medium effect of cannabis use status on both the undergraduates’ (Cohen’s d = 0.46) and the neuropsychologists’ (Cohen’s d = 0.61) ratings (Hirst et al., 2016, 2017). These results suggest that even research studies using examiners who are blind to participants’ group status may be vulnerable to the expectancy effect confound if they are able to guess cannabis user status. Further, there is precedent for anticipating that examiners’ expectancy effects may contribute to variability in participants’ neuropsychological performance. Researchers have found that examiners’ beliefs about the effects of caffeine administration affected examinee physiological response and cognitive performance (Walach, Schmidt, Bihr, & Wiesch, 2001). In this study, experimenters administered placebo “caffeine” to participants and measured their blood pressure, heart rate, well-being, and performance on a cognitive task. When experimenters were told that caffeine administration alters examinees’ physiological and cognitive functions, their participants demonstrated higher systolic blood pressure and worse performance on cognitive testing following placebo administration, relative to the participants of experimenters who were told the opposite. These findings support the possibility that examiner expectations of cannabis user performance on cognitive testing could have acted as a confounding variable in many published research designs to date. The plethora of findings supporting the existence of expectancy effects in research experiments (Rosenthal, 2002) and the results of a study identifying expectancy effects influencing participants’ cognitive performance (Walach et al., 2001) suggest that this confound could be contributing to the variability in the literature on the cognitive effects of chronic cannabis use. Further, research indicates that neuropsychologists can discriminate between cannabis users and non-users, suggesting that even blind research designs are vulnerable to this confound (Hirst et al., 2016, 2017). Therefore, the present study sought to investigate whether examiners’ perceptions of examinees’ cannabis use status influenced participant performance on neuropsychological assessment. Specifically, the authors hypothesized that examiners would be able to judge participant cannabis user status based upon appearance alone with accuracy greater than chance, consistent with Hirst and colleagues (2016, 2017) findings. Further, we hypothesized that participants judged as cannabis users would perform worse on neuropsychological assessment, relative to those judged as non-users (irrespective of true user status). To this aim, examiners privately noted whether they believed the participant was a cannabis user or a non-user, allowing the measurement of whether these perceptions resulted in differential performance by examinees on neuropsychological testing. Finally, because expectancy effects are likely related to the examiners’ personal beliefs regarding the effects of cannabis use, the authors also sought to determine whether the experiences and beliefs of the examiners administering the neuropsychological assessment battery related to examinees’ performance. Method Participants Recruitment consisted of flyers posted in the community, online postings, and advertisements emailed to local colleges. To qualify for inclusion, participants must have been between 18 and 30 years old, to reduce the likelihood of confounding factors influencing cognition (e.g., age-related cognitive decline). In order to meet eligibility criteria as a chronic cannabis user, individuals must have been current cannabis users, using cannabis at least 4 days per week for the past year. To be eligible as a non-user control, the individual must have tried cannabis at least once but no more than five times in his or her lifetime, and not within the past 30 days. Cannabis researchers have previously recommended this approach because individuals who would never try cannabis may be conceivably different from individuals who would try cannabis in ways that may influence cognition, such as decision-making or impulsivity (Pope et al., 2001). The inclusion of individuals who would be willing to try cannabis allows for a non-user control group that is cognitively similar in premorbid functioning to cannabis users. Exclusionary criteria consisted of self-report of: (a) use of any other class of drugs of abuse (e.g., opiates) more than five times; (b) current alcohol use of a frequency that could interfere with neurocognitive performance (defined as consuming two or more drinks on four or more days per week, for the past month or longer); (c) current Diagnostic and Statistical Manual of Mental Disorders – Fourth Edition-Text Revision (DSM–IV-TR; American Psychiatric Association, 2000) diagnosis of Axis I disorder other than simple phobia; (d) history of head injury with loss of consciousness requiring medical intervention within the past six months; (e) current use of psychoactive medication; and (f) a medical, psychiatric, or neurological condition that might affect cognitive function (e.g., epilepsy, multiple sclerosis, brain tumor, etc.). These exclusion criteria were selected to eliminate potential confounds which might affect cognitive performance, and have been recommended by experts in the cannabis literature (Gonzalez, Carey, & Grant, 2002). A total of 61 individuals participated in the study; demographic information for the whole sample is presented in Table 1. Participants’ ages ranged from 18 to 30 years. Thirty-seven (60.7%) participants in the study sample were male, whereas 24 (39.3%) participants were female. Nineteen of the participants identified as Caucasian (31.1%), 16 identified as Hispanic or Latino (26.2%), 10 identified as Asian or Asian-American (16.4%), four identified as Black or African American (6.6%), four identified as Middle Eastern (6.6%), and eight participants identified as Other or Mixed ethnicity (13.1%). Total years of education for the entire sample ranged from 12 to 20 years. The mean estimated premorbid IQ for the entire sample, estimated from the National Adult Reading Test-Revised (NART-R; Wiens, Bryon, & Crossen, 1993), was 102.60 (SD = 15.63). Approximately 90% (n = 55) of the participants were right-handed, whereas about 10% (n = 6) were left-handed. Table 1. Demographic information: total sample, cannabis users, and non-users   Total sample  Cannabis users  Non-users  Age (years)  21.89 (3.32)  21.37 (2.91)  22.95 (3.90)  Percent male  60.7%  73.2%*  35.0%*  Years of education  14.27 (1.81)  13.96 (1.38)  14.90 (2.38)  Estimated premorbid IQ  102.60 (15.63)  100.58 (17.27)  106.76 (10.81)    Total sample  Cannabis users  Non-users  Age (years)  21.89 (3.32)  21.37 (2.91)  22.95 (3.90)  Percent male  60.7%  73.2%*  35.0%*  Years of education  14.27 (1.81)  13.96 (1.38)  14.90 (2.38)  Estimated premorbid IQ  102.60 (15.63)  100.58 (17.27)  106.76 (10.81)  Note: * Group difference is significant at the p < .05 level. Table 1. Demographic information: total sample, cannabis users, and non-users   Total sample  Cannabis users  Non-users  Age (years)  21.89 (3.32)  21.37 (2.91)  22.95 (3.90)  Percent male  60.7%  73.2%*  35.0%*  Years of education  14.27 (1.81)  13.96 (1.38)  14.90 (2.38)  Estimated premorbid IQ  102.60 (15.63)  100.58 (17.27)  106.76 (10.81)    Total sample  Cannabis users  Non-users  Age (years)  21.89 (3.32)  21.37 (2.91)  22.95 (3.90)  Percent male  60.7%  73.2%*  35.0%*  Years of education  14.27 (1.81)  13.96 (1.38)  14.90 (2.38)  Estimated premorbid IQ  102.60 (15.63)  100.58 (17.27)  106.76 (10.81)  Note: * Group difference is significant at the p < .05 level. The sample included 41 chronic cannabis users and 20 non-users. Demographic information of the user and non-user group is also included in Table 1. There were no significant differences between users and non-users in age, ethnicity, years of education, or estimated premorbid IQ based upon performance on the NART-R. There were significantly more males in the user group compared to the non-user group (p = .004); however, the gender distribution of chronic cannabis users in the present study is representative of documented gender differences among marijuana users (Johnston, O’Malley, Bachman, Schulenberg, & Miech, 2015). Within the non-user group, five participants identified as Caucasian (25%), three identified as Hispanic or Latino (15%), six identified as Asian or Asian-American (30%), three identified as Middle Eastern (15%), and three participants identified as Other or Mixed ethnicity (15%). Within the user group, 14 participants identified as Caucasian (34.1%), 13 identified as Hispanic or Latino (31.7%), four identified as Asian or Asian-American (9.8%), four identified as Black or African American (9.8%), one identified as Middle Eastern (2.4%), and five participants identified as Other or Mixed ethnicity (12.2%). Approximately 85% (n = 17) of the non-users were right-handed, whereas about 92% (n = 38) of users were right-handed. Because cannabis use can co-occur with alcohol use, independent samples t-tests examined whether cannabis users and non-users differed in alcohol use. Most respondents (72%; n = 44) reported using alcohol two times or less per month, and most (60.6%; n = 37) reported drinking two or fewer drinks per sitting. Analyses revealed that the two groups (cannabis users and non-users) did not significantly differ in frequency (p = .876) or amount (p = .987) of alcohol use. Procedure The researchers gave interested participants a brief preliminary eligibility screen (approximately 5–10 min) via online surveys and over the phone to ensure they met eligibility criteria. If met, an appointment for the neuropsychological testing was scheduled. Participants agreed to abstain from alcohol, marijuana, and other substances for at least 24 hrs prior to the appointment. They also agreed not to disclose their cannabis use status (user or non-user) to the examiner who completed the neuropsychological testing, so that examiners would remain blind to user status. Examiners for the study were comprised of doctoral-level graduate students with at least 1 year of training in neuropsychology and in the administration of the test battery. All examiners were familiar with research examining the impact of cannabis use on neurocognitive functioning. During the testing session, participants first completed informed consent. At the same time, examiners privately noted whether they believed the participant was a cannabis user or non-user, so that data could be analyzed for the expectancy effect confound. Examiners also administered a brief field sobriety test (i.e., standing on one foot for 30 s) to ensure participants were not under the influence of any substance at the time of testing. All participants passed this field sobriety test prior to completing the experiment. As part of a separate research study examining the effect of motivational statements on cognitive performance, examiners then administered one of two statements to the participant, randomly assigned prior to the beginning of the assessment session and identical to those previously described by Macher and Earleywine (2012). These motivational statement conditions were unrelated to the present study; therefore, we controlled for motivational condition in the statistical analyses. Participants then provided basic demographic information to the examiners, including age, date of birth, sex, ethnic identification, years of education, and handedness. Next, the examiners administered the neuropsychological test battery, which consisted of tests assessing a variety of cognitive domains, as well as performance validity tests (PVTs) to ensure that participants put forth adequate effort to perform well on the assessment, so that the results are known to be valid. The neuropsychological battery consisted of the California Verbal Learning Test – 2nd edition (CVLT-II; Delis, Kramer, Kaplan, & Ober, 2000), the Block Design, Digit Span, and Digit-Symbol Coding subtests of the Wechsler Adult Intelligence Scale – Third Edition (WAIS-III; Wechsler, 1997), the Rey–Osterrieth Complex Figure Test (RCF; Meyers & Meyers, 1995), the Trail Making Test of the Halstead–Reitan Neuropsychological Battery (TMT Part A and Part B; Reitan & Wolfson, 1992), and the National Adult Reading Test-Revised (NART-R; Blair & Spreen, 1989). Performance validity testing consisted of the Forced-Choice subtest of the CVLT-II (Moore & Donders, 2004), the Word Memory Test (WMT; Green, Allen, & Astner, 1996), and the Test of Memory Malingering (TOMM; Tombaugh, 1996), as well as the embedded measures of the Reliable Digit Span (RDS; Etherton, Bianchini, Greve, & Heinly, 2005) and the Trail Making Test Ratio (TMT B:A Ratio; Ruffolo, Guilmette, & Willis, 2000). All participants passed objective validity measures, indicating that the evaluation results were valid. Following the neuropsychological test battery, participants completed a two question self-report questionnaire assessing initial motivation to do well on the assessment, as well as potential change in motivation over time. Upon completion of the appointment, the examiners gave participants a $50 gift card as compensation. Research assistants de-identified all data prior to entering it into a statistical database (SPSS) for analyses. Once the neuropsychological assessments of all examinees were complete, each research examiner (n = 6) completed a survey regarding their own experiences and beliefs about the effects of cannabis use. This survey sought to assess whether the examiners’ personal beliefs might have influenced the interaction with the participant and subsequent performance on the neuropsychological assessment. Each examiner was randomly assigned an identification number and completed an anonymous and confidential survey that asked: “Assuming all other variables are equal (e.g., average baseline functioning, no comorbid diagnoses, etc.), to what extent (standard deviations above/below premorbid functioning) do you believe an individual’s Full Scale IQ in general would be affected, were the individual to report regular marijuana use (i.e., >4 times per week for at least 1 year)?” Response options ranged from −3 standard deviations (SDs) below baseline to +3 SDs above baseline, in 0.5 SDincrements. The same question was asked regarding the examiner’s beliefs about effects on individuals’ attention, memory, executive function, language, visuospatial/constructional skills, motor skills, processing speed, and performance validity/effort. Additionally, examiners answered questions about whether they believed their personal beliefs about cannabis user status might have influenced their interactions with participants and the participants’ subsequent performance on the neuropsychological assessment. Specifically, examiners answered whether they had ever used marijuana, whether they believed marijuana should be legalized recreationally, and whether they believed that their perceptions of the participants’ user status affected interactions with them. They also rated their confidence in their ability to guess whether participants were cannabis users or non-users. Results Accuracy of examiner judgment Analyses calculated the accuracy of examiners’ perceptions regarding participants’ cannabis use (i.e., cannabis user or non-user). The overall concordance of results was 63.9% (39/61: exact, 95% CI [50.6% –75.87%]), revealing that examiners accurately predicted cannabis use status at a rate better than chance (p = .04, as determined by an exact binomial test). Table 2 presents the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of examiners’ perceptions of participants’ cannabis use, along with 95% confidence intervals. Sensitivity was 68.29% (28/41: exact, 95% CI [51.91% –81.92%]), indicating that cannabis users were accurately identified as users at rates greater than chance (p = .028). However, the specificity was 55.00% (11/20: exact, 95% CI [31.53% –76.94%]), indicating that prediction of non-users did not differ from chance (p > .05). The PPV was 75.68% (28/37: exact, 95% CI [58.80% –88.23%]), indicating that about three-quarters of examiners’ cannabis user judgments were accurate, significantly above chance (p = .003). In contrast, the NPV was 45.83% (11/24: exact, 95% CI [25.55% –67.18%]), suggesting that less than half of examiners’ non-user judgments were accurate, which was not significantly more predictive than chance alone (p > .05). Within the user participants, there were no significant differences between examiner-judged users and examiner-judged non-users in age of onset of cannabis use, days per week of use, length of cannabis use, or amount of cannabis used per sitting (see Table 3). Table 2. Positive and negative predictive value, sensitivity, and specificity of examiners’ ratings of participants   User  Non-user  Total    Rated as user  28†  9  37  PPV = 75.68%  Rated as non-user  13‡  11  24  NPV = 45.83%  Total  41  20  61      Sens = 68.29%  Spec = 55.00%        User  Non-user  Total    Rated as user  28†  9  37  PPV = 75.68%  Rated as non-user  13‡  11  24  NPV = 45.83%  Total  41  20  61      Sens = 68.29%  Spec = 55.00%      Note: † = True Positives, ‡ = False Negatives, PPV = Positive Predictive Value, NPV = Negative Predictive Value, Sens = Sensitivity, Spec = Specificity. Table 2. Positive and negative predictive value, sensitivity, and specificity of examiners’ ratings of participants   User  Non-user  Total    Rated as user  28†  9  37  PPV = 75.68%  Rated as non-user  13‡  11  24  NPV = 45.83%  Total  41  20  61      Sens = 68.29%  Spec = 55.00%        User  Non-user  Total    Rated as user  28†  9  37  PPV = 75.68%  Rated as non-user  13‡  11  24  NPV = 45.83%  Total  41  20  61      Sens = 68.29%  Spec = 55.00%      Note: † = True Positives, ‡ = False Negatives, PPV = Positive Predictive Value, NPV = Negative Predictive Value, Sens = Sensitivity, Spec = Specificity. Table 3. Cannabis use data by user status and examiner-judged user status   Cannabis users  Examiner-judged users  Examiner-judged non-users  Age of onset of cannabis use (years)  16.35 (1.96)  16.19 (1.90)  16.69 (2.10)  Days per week of current cannabis (for true cannabis users only)  5.38 (1.53)  5.50 (1.42)  5.12 (1.78)  Length of cannabis use (years; for true cannabis users only)  2.36 (1.82)  2.35 (1.81)  2.38 (1.92)  Cannabis use per sitting (grams; for true cannabis users only)  1.76 (1.75)  1.96 (1.94)  1.37 (1.31)    Cannabis users  Examiner-judged users  Examiner-judged non-users  Age of onset of cannabis use (years)  16.35 (1.96)  16.19 (1.90)  16.69 (2.10)  Days per week of current cannabis (for true cannabis users only)  5.38 (1.53)  5.50 (1.42)  5.12 (1.78)  Length of cannabis use (years; for true cannabis users only)  2.36 (1.82)  2.35 (1.81)  2.38 (1.92)  Cannabis use per sitting (grams; for true cannabis users only)  1.76 (1.75)  1.96 (1.94)  1.37 (1.31)  Note: Means (standard deviations in parentheses). No group differences are significant at the p < .05 level. Table 3. Cannabis use data by user status and examiner-judged user status   Cannabis users  Examiner-judged users  Examiner-judged non-users  Age of onset of cannabis use (years)  16.35 (1.96)  16.19 (1.90)  16.69 (2.10)  Days per week of current cannabis (for true cannabis users only)  5.38 (1.53)  5.50 (1.42)  5.12 (1.78)  Length of cannabis use (years; for true cannabis users only)  2.36 (1.82)  2.35 (1.81)  2.38 (1.92)  Cannabis use per sitting (grams; for true cannabis users only)  1.76 (1.75)  1.96 (1.94)  1.37 (1.31)    Cannabis users  Examiner-judged users  Examiner-judged non-users  Age of onset of cannabis use (years)  16.35 (1.96)  16.19 (1.90)  16.69 (2.10)  Days per week of current cannabis (for true cannabis users only)  5.38 (1.53)  5.50 (1.42)  5.12 (1.78)  Length of cannabis use (years; for true cannabis users only)  2.36 (1.82)  2.35 (1.81)  2.38 (1.92)  Cannabis use per sitting (grams; for true cannabis users only)  1.76 (1.75)  1.96 (1.94)  1.37 (1.31)  Note: Means (standard deviations in parentheses). No group differences are significant at the p < .05 level. Differences in neuropsychological and PVT performance between examiner-judged users and examiner-judged non-users Table 4 presents a comparison of neuropsychological performance across four groups: those accurately judged as users, those accurately judged as non-users, those inaccurately judged as users, and those inaccurately judged as non-users. Importantly, direct comparisons are difficult to interpret given the known confounds that may have affected cognitive performance. Therefore, a multiple regression analyzed whether examiner belief of participants’ user status affected performance on neuropsychological and validity tests. Because demographic variables could act as confounds, age and gender were entered into the analysis. While independent samples t-tests revealed no significant differences between users and non-users or examiner-judged users and examiner-judged non-users in age (p > .05), there was a significantly (p < .01) larger representation of male cannabis users (n = 30 of 41 users; 73%) relative to female users (n = 11 of 41 users; 27%). This is consistent with expectation, as cannabis users are predominantly male (Johnston et al., 2015). Furthermore, bivariate correlational analysis revealed a significant (p < .02) association between gender and examiner expectancies. This association was anticipated given the examiners’ ability to accurately predict cannabis use and the larger prevalence of male cannabis users in the study. Therefore, gender was also included in the regression analyses. As described in the Method, motivational condition was included in the analysis to control for the potential effects of the motivational statement administered prior to testing, as this statement was unrelated to the present study. Finally, to isolate the effect of examiner belief of cannabis user status from actual cannabis user status, actual cannabis user status was included in the analysis. This permits the analysis of the effect of examiner belief of user status, above and beyond the effect of actual user status alone. Table 4. Neuropsychological and performance validity test performance by cannabis user status and examiner judgments   Cannabis users  Non-users    Examiner judged: user (accurate)  Examiner judged: non-user (inaccurate)  Examiner judged: non-user (accurate)  Examiner judged: user (inaccurate)  Neuropsychological measure   WMT Free Recall  23.75 (6.82)  25.38 (5.56)  26.73 (6.39)  25.56 (6.17)   RCF Copy  32.77 (3.41)  33.35 (2.77)  33.82 (1.66)  33.33 (2.41)   RCF Immediate Recall  18.57 (6.75)  17.54 (7.67)  21.14 (4.96)  22.17 (5.83)   RCF Delayed Recall  17.91 (7.16)  18.15 (7.78)  21.14 (4.23)  23.00 (5.44)   RCF Recognition  20.25 (1.97)  20.54 (1.81)  20.73 (1.62)  21.11 (2.42)   CVLT-II Trials 1-5 Free Recall  48.21 (9.60)  52.46 (8.50)  57.82 (7.47)  51.44 (8.82)   CVLT-II Trial B Free Recall  4.96 (1.53)  5.62 (1.66)  6.91 (1.81)  5.67 (2.24)   CVLT-II Short Delay Free Recall  9.86 (3.18)  11.69 (1.84)  12.27 (2.57)  11.56 (3.05)   CVLT-II Short Delay Cued Recall  10.71 (3.34)  11.77 (2.17)  13.45 (2.46)  11.56 (3.32)   CVLT-II Long Delay Free Recall  9.93 (3.22)  12.23 (2.39)  13.55 (2.46)  11.89 (3.30)   CVLT-II Long Delay Cued Recall  10.50 (3.31)  12.69 (2.46)  13.73 (2.20)  11.44 (3.47)   WAIS-III Digit Span  17.54 (2.73)  18.23 (4.21)  17.73 (4.45)  15.56 (2.88)   WAIS-III Block Design  45.96 (14.03)  46.08 (9.59)  46.73 (9.34)  48.78 (9.85)   WAIS-III Digit Symbol Coding  77.57 (15.40)  78.92 (16.77)  91.00 (8.40)  67.44 (9.03)   TMT A (in seconds; reverse scored)  24.79 (6.07)  24.31 (7.63)  21.82 (6.46)  33.00 (13.07)   TMT B (in seconds; reverse scored)  67.14 (20.29)  58.31 (16.43)  58.00 (12.30)  87.00 (34.66)  Performance Validity Measure   WMT IR  38.75 (1.35)  39.54 (.78)  39.55 (.93)  39.22 (.97)   WMT DR  39.00 (1.66)  39.31 (.75)  39.09 (1.45)  39.00 (1.32)   WMT CNS  38.32 (1.79)  38.85 (1.28)  38.64 (1.91)  38.67 (1.41)   RCF Combination Score  57.30 (6.96)  58.96 (6.76)  60.00 (5.87)  60.67 (8.74)   CVLT-II FC  15.96 (.19)  15.92 (.28)  16.00 (.00)  15.78 (.45)   RDS  11.75 (1.60)  12.08 (2.22)  12.00 (2.57)  10.67 (1.73)   TMT B/A Ratio  2.75 (.84)  2.52 (.82)  2.80 (.82)  2.70 (.67)   TOMM Trial 2  49.93 (.26)  49.85 (.38)  49.82 (.60)  50.00 (.00)   TOMM Retention Trial  49.71 (.85)  49.85 (.55)  49.91 (.30)  49.89 (.33)    Cannabis users  Non-users    Examiner judged: user (accurate)  Examiner judged: non-user (inaccurate)  Examiner judged: non-user (accurate)  Examiner judged: user (inaccurate)  Neuropsychological measure   WMT Free Recall  23.75 (6.82)  25.38 (5.56)  26.73 (6.39)  25.56 (6.17)   RCF Copy  32.77 (3.41)  33.35 (2.77)  33.82 (1.66)  33.33 (2.41)   RCF Immediate Recall  18.57 (6.75)  17.54 (7.67)  21.14 (4.96)  22.17 (5.83)   RCF Delayed Recall  17.91 (7.16)  18.15 (7.78)  21.14 (4.23)  23.00 (5.44)   RCF Recognition  20.25 (1.97)  20.54 (1.81)  20.73 (1.62)  21.11 (2.42)   CVLT-II Trials 1-5 Free Recall  48.21 (9.60)  52.46 (8.50)  57.82 (7.47)  51.44 (8.82)   CVLT-II Trial B Free Recall  4.96 (1.53)  5.62 (1.66)  6.91 (1.81)  5.67 (2.24)   CVLT-II Short Delay Free Recall  9.86 (3.18)  11.69 (1.84)  12.27 (2.57)  11.56 (3.05)   CVLT-II Short Delay Cued Recall  10.71 (3.34)  11.77 (2.17)  13.45 (2.46)  11.56 (3.32)   CVLT-II Long Delay Free Recall  9.93 (3.22)  12.23 (2.39)  13.55 (2.46)  11.89 (3.30)   CVLT-II Long Delay Cued Recall  10.50 (3.31)  12.69 (2.46)  13.73 (2.20)  11.44 (3.47)   WAIS-III Digit Span  17.54 (2.73)  18.23 (4.21)  17.73 (4.45)  15.56 (2.88)   WAIS-III Block Design  45.96 (14.03)  46.08 (9.59)  46.73 (9.34)  48.78 (9.85)   WAIS-III Digit Symbol Coding  77.57 (15.40)  78.92 (16.77)  91.00 (8.40)  67.44 (9.03)   TMT A (in seconds; reverse scored)  24.79 (6.07)  24.31 (7.63)  21.82 (6.46)  33.00 (13.07)   TMT B (in seconds; reverse scored)  67.14 (20.29)  58.31 (16.43)  58.00 (12.30)  87.00 (34.66)  Performance Validity Measure   WMT IR  38.75 (1.35)  39.54 (.78)  39.55 (.93)  39.22 (.97)   WMT DR  39.00 (1.66)  39.31 (.75)  39.09 (1.45)  39.00 (1.32)   WMT CNS  38.32 (1.79)  38.85 (1.28)  38.64 (1.91)  38.67 (1.41)   RCF Combination Score  57.30 (6.96)  58.96 (6.76)  60.00 (5.87)  60.67 (8.74)   CVLT-II FC  15.96 (.19)  15.92 (.28)  16.00 (.00)  15.78 (.45)   RDS  11.75 (1.60)  12.08 (2.22)  12.00 (2.57)  10.67 (1.73)   TMT B/A Ratio  2.75 (.84)  2.52 (.82)  2.80 (.82)  2.70 (.67)   TOMM Trial 2  49.93 (.26)  49.85 (.38)  49.82 (.60)  50.00 (.00)   TOMM Retention Trial  49.71 (.85)  49.85 (.55)  49.91 (.30)  49.89 (.33)  Note: Means (standard deviations in parentheses). Table 4. Neuropsychological and performance validity test performance by cannabis user status and examiner judgments   Cannabis users  Non-users    Examiner judged: user (accurate)  Examiner judged: non-user (inaccurate)  Examiner judged: non-user (accurate)  Examiner judged: user (inaccurate)  Neuropsychological measure   WMT Free Recall  23.75 (6.82)  25.38 (5.56)  26.73 (6.39)  25.56 (6.17)   RCF Copy  32.77 (3.41)  33.35 (2.77)  33.82 (1.66)  33.33 (2.41)   RCF Immediate Recall  18.57 (6.75)  17.54 (7.67)  21.14 (4.96)  22.17 (5.83)   RCF Delayed Recall  17.91 (7.16)  18.15 (7.78)  21.14 (4.23)  23.00 (5.44)   RCF Recognition  20.25 (1.97)  20.54 (1.81)  20.73 (1.62)  21.11 (2.42)   CVLT-II Trials 1-5 Free Recall  48.21 (9.60)  52.46 (8.50)  57.82 (7.47)  51.44 (8.82)   CVLT-II Trial B Free Recall  4.96 (1.53)  5.62 (1.66)  6.91 (1.81)  5.67 (2.24)   CVLT-II Short Delay Free Recall  9.86 (3.18)  11.69 (1.84)  12.27 (2.57)  11.56 (3.05)   CVLT-II Short Delay Cued Recall  10.71 (3.34)  11.77 (2.17)  13.45 (2.46)  11.56 (3.32)   CVLT-II Long Delay Free Recall  9.93 (3.22)  12.23 (2.39)  13.55 (2.46)  11.89 (3.30)   CVLT-II Long Delay Cued Recall  10.50 (3.31)  12.69 (2.46)  13.73 (2.20)  11.44 (3.47)   WAIS-III Digit Span  17.54 (2.73)  18.23 (4.21)  17.73 (4.45)  15.56 (2.88)   WAIS-III Block Design  45.96 (14.03)  46.08 (9.59)  46.73 (9.34)  48.78 (9.85)   WAIS-III Digit Symbol Coding  77.57 (15.40)  78.92 (16.77)  91.00 (8.40)  67.44 (9.03)   TMT A (in seconds; reverse scored)  24.79 (6.07)  24.31 (7.63)  21.82 (6.46)  33.00 (13.07)   TMT B (in seconds; reverse scored)  67.14 (20.29)  58.31 (16.43)  58.00 (12.30)  87.00 (34.66)  Performance Validity Measure   WMT IR  38.75 (1.35)  39.54 (.78)  39.55 (.93)  39.22 (.97)   WMT DR  39.00 (1.66)  39.31 (.75)  39.09 (1.45)  39.00 (1.32)   WMT CNS  38.32 (1.79)  38.85 (1.28)  38.64 (1.91)  38.67 (1.41)   RCF Combination Score  57.30 (6.96)  58.96 (6.76)  60.00 (5.87)  60.67 (8.74)   CVLT-II FC  15.96 (.19)  15.92 (.28)  16.00 (.00)  15.78 (.45)   RDS  11.75 (1.60)  12.08 (2.22)  12.00 (2.57)  10.67 (1.73)   TMT B/A Ratio  2.75 (.84)  2.52 (.82)  2.80 (.82)  2.70 (.67)   TOMM Trial 2  49.93 (.26)  49.85 (.38)  49.82 (.60)  50.00 (.00)   TOMM Retention Trial  49.71 (.85)  49.85 (.55)  49.91 (.30)  49.89 (.33)    Cannabis users  Non-users    Examiner judged: user (accurate)  Examiner judged: non-user (inaccurate)  Examiner judged: non-user (accurate)  Examiner judged: user (inaccurate)  Neuropsychological measure   WMT Free Recall  23.75 (6.82)  25.38 (5.56)  26.73 (6.39)  25.56 (6.17)   RCF Copy  32.77 (3.41)  33.35 (2.77)  33.82 (1.66)  33.33 (2.41)   RCF Immediate Recall  18.57 (6.75)  17.54 (7.67)  21.14 (4.96)  22.17 (5.83)   RCF Delayed Recall  17.91 (7.16)  18.15 (7.78)  21.14 (4.23)  23.00 (5.44)   RCF Recognition  20.25 (1.97)  20.54 (1.81)  20.73 (1.62)  21.11 (2.42)   CVLT-II Trials 1-5 Free Recall  48.21 (9.60)  52.46 (8.50)  57.82 (7.47)  51.44 (8.82)   CVLT-II Trial B Free Recall  4.96 (1.53)  5.62 (1.66)  6.91 (1.81)  5.67 (2.24)   CVLT-II Short Delay Free Recall  9.86 (3.18)  11.69 (1.84)  12.27 (2.57)  11.56 (3.05)   CVLT-II Short Delay Cued Recall  10.71 (3.34)  11.77 (2.17)  13.45 (2.46)  11.56 (3.32)   CVLT-II Long Delay Free Recall  9.93 (3.22)  12.23 (2.39)  13.55 (2.46)  11.89 (3.30)   CVLT-II Long Delay Cued Recall  10.50 (3.31)  12.69 (2.46)  13.73 (2.20)  11.44 (3.47)   WAIS-III Digit Span  17.54 (2.73)  18.23 (4.21)  17.73 (4.45)  15.56 (2.88)   WAIS-III Block Design  45.96 (14.03)  46.08 (9.59)  46.73 (9.34)  48.78 (9.85)   WAIS-III Digit Symbol Coding  77.57 (15.40)  78.92 (16.77)  91.00 (8.40)  67.44 (9.03)   TMT A (in seconds; reverse scored)  24.79 (6.07)  24.31 (7.63)  21.82 (6.46)  33.00 (13.07)   TMT B (in seconds; reverse scored)  67.14 (20.29)  58.31 (16.43)  58.00 (12.30)  87.00 (34.66)  Performance Validity Measure   WMT IR  38.75 (1.35)  39.54 (.78)  39.55 (.93)  39.22 (.97)   WMT DR  39.00 (1.66)  39.31 (.75)  39.09 (1.45)  39.00 (1.32)   WMT CNS  38.32 (1.79)  38.85 (1.28)  38.64 (1.91)  38.67 (1.41)   RCF Combination Score  57.30 (6.96)  58.96 (6.76)  60.00 (5.87)  60.67 (8.74)   CVLT-II FC  15.96 (.19)  15.92 (.28)  16.00 (.00)  15.78 (.45)   RDS  11.75 (1.60)  12.08 (2.22)  12.00 (2.57)  10.67 (1.73)   TMT B/A Ratio  2.75 (.84)  2.52 (.82)  2.80 (.82)  2.70 (.67)   TOMM Trial 2  49.93 (.26)  49.85 (.38)  49.82 (.60)  50.00 (.00)   TOMM Retention Trial  49.71 (.85)  49.85 (.55)  49.91 (.30)  49.89 (.33)  Note: Means (standard deviations in parentheses). Table 5 provides results of multiple regression analysis of neuropsychological and validity measures as a function of examiner judgments of cannabis user status, after adjusting for the participants’ age, gender, motivational condition, and actual cannabis user status. Examiner expectancies remained a significant predictor of several neuropsychological performance measures, including multiple subtests of the CVLT-II and the TMT B (p < .05), with effect sizes in the medium range (Cohen’s d = 0.75 and 0.61, respectively). These findings suggest that, even after controlling for potential confounds including user status, participants judged as cannabis users performed worse on two of the eight neuropsychological tests, compared to participants judged as non-users. Examiner expectancies were not a significant predictor on RCF, any of the three included WAIS-III subtests, TMT A, TOMM, or WMT (ps > .05). Table 5. Adjusted examiner expectancy effects on neuropsychological performance: controlling for age, gender, actual user status, and motivational condition Measure (Raw)  Term  Estimate  Standard error  t  p-Value  CVLT-II Trials 1-5 Free Recall  Examiner Belief  2.958  1.179  2.510  .015*  CVLT-II Trial B Free Recall  Examiner Belief  0.431  0.230  1.870  .066    User Status [0]  0.489  0.240  2.040  .046*  CVLT-II Short Delay Free Recall  Examiner Belief  0.559  0.377  1.480  .143    Gender [0]  −0.909  0.377  −2.410  .019*  CVLT-II Short Delay Cued Recall  Examiner Belief  0.811  0.392  2.070  .043*  CVLT-II Long Delay Free Recall  Examiner Belief  1.036  0.394  2.630  .011*    User Status [0]  0.830  0.410  2.030  .048*  CVLT-II Long Delay Cued Recall  Examiner Belief  1.218  0.391  3.120  .003**  WAIS-III Digit Symbol Coding  Examiner Belief  2.097  1.926  1.090  .281    Gender [0]  −5.562  1.890  −2.940  .005*    Motivation [0]  −4.038  1.797  −2.250  .0290*  TMT A (in seconds)  Examiner Belief  −1.809  1.075  −1.680  .098  TMT B (in seconds)  Examiner Belief  −6.693  2.766  −2.420  .019*    Age  1.888  0.822  2.300  .025*  WMT IR  Examiner Belief  0.277  0.146  1.890  .063    Motivation [0]  −0.297  0.143  −2.070  .043*  Measure (Raw)  Term  Estimate  Standard error  t  p-Value  CVLT-II Trials 1-5 Free Recall  Examiner Belief  2.958  1.179  2.510  .015*  CVLT-II Trial B Free Recall  Examiner Belief  0.431  0.230  1.870  .066    User Status [0]  0.489  0.240  2.040  .046*  CVLT-II Short Delay Free Recall  Examiner Belief  0.559  0.377  1.480  .143    Gender [0]  −0.909  0.377  −2.410  .019*  CVLT-II Short Delay Cued Recall  Examiner Belief  0.811  0.392  2.070  .043*  CVLT-II Long Delay Free Recall  Examiner Belief  1.036  0.394  2.630  .011*    User Status [0]  0.830  0.410  2.030  .048*  CVLT-II Long Delay Cued Recall  Examiner Belief  1.218  0.391  3.120  .003**  WAIS-III Digit Symbol Coding  Examiner Belief  2.097  1.926  1.090  .281    Gender [0]  −5.562  1.890  −2.940  .005*    Motivation [0]  −4.038  1.797  −2.250  .0290*  TMT A (in seconds)  Examiner Belief  −1.809  1.075  −1.680  .098  TMT B (in seconds)  Examiner Belief  −6.693  2.766  −2.420  .019*    Age  1.888  0.822  2.300  .025*  WMT IR  Examiner Belief  0.277  0.146  1.890  .063    Motivation [0]  −0.297  0.143  −2.070  .043*  Note: Significance level *.05, **.01. Bold p-values indicate significance for the Examiner Belief variable. Gender was coded as males = 0, females = 1. User status was coded as non-user = 0, user = 1. Motivational condition was coded as neutral statement = 0, motivational statement = 1. Table 5. Adjusted examiner expectancy effects on neuropsychological performance: controlling for age, gender, actual user status, and motivational condition Measure (Raw)  Term  Estimate  Standard error  t  p-Value  CVLT-II Trials 1-5 Free Recall  Examiner Belief  2.958  1.179  2.510  .015*  CVLT-II Trial B Free Recall  Examiner Belief  0.431  0.230  1.870  .066    User Status [0]  0.489  0.240  2.040  .046*  CVLT-II Short Delay Free Recall  Examiner Belief  0.559  0.377  1.480  .143    Gender [0]  −0.909  0.377  −2.410  .019*  CVLT-II Short Delay Cued Recall  Examiner Belief  0.811  0.392  2.070  .043*  CVLT-II Long Delay Free Recall  Examiner Belief  1.036  0.394  2.630  .011*    User Status [0]  0.830  0.410  2.030  .048*  CVLT-II Long Delay Cued Recall  Examiner Belief  1.218  0.391  3.120  .003**  WAIS-III Digit Symbol Coding  Examiner Belief  2.097  1.926  1.090  .281    Gender [0]  −5.562  1.890  −2.940  .005*    Motivation [0]  −4.038  1.797  −2.250  .0290*  TMT A (in seconds)  Examiner Belief  −1.809  1.075  −1.680  .098  TMT B (in seconds)  Examiner Belief  −6.693  2.766  −2.420  .019*    Age  1.888  0.822  2.300  .025*  WMT IR  Examiner Belief  0.277  0.146  1.890  .063    Motivation [0]  −0.297  0.143  −2.070  .043*  Measure (Raw)  Term  Estimate  Standard error  t  p-Value  CVLT-II Trials 1-5 Free Recall  Examiner Belief  2.958  1.179  2.510  .015*  CVLT-II Trial B Free Recall  Examiner Belief  0.431  0.230  1.870  .066    User Status [0]  0.489  0.240  2.040  .046*  CVLT-II Short Delay Free Recall  Examiner Belief  0.559  0.377  1.480  .143    Gender [0]  −0.909  0.377  −2.410  .019*  CVLT-II Short Delay Cued Recall  Examiner Belief  0.811  0.392  2.070  .043*  CVLT-II Long Delay Free Recall  Examiner Belief  1.036  0.394  2.630  .011*    User Status [0]  0.830  0.410  2.030  .048*  CVLT-II Long Delay Cued Recall  Examiner Belief  1.218  0.391  3.120  .003**  WAIS-III Digit Symbol Coding  Examiner Belief  2.097  1.926  1.090  .281    Gender [0]  −5.562  1.890  −2.940  .005*    Motivation [0]  −4.038  1.797  −2.250  .0290*  TMT A (in seconds)  Examiner Belief  −1.809  1.075  −1.680  .098  TMT B (in seconds)  Examiner Belief  −6.693  2.766  −2.420  .019*    Age  1.888  0.822  2.300  .025*  WMT IR  Examiner Belief  0.277  0.146  1.890  .063    Motivation [0]  −0.297  0.143  −2.070  .043*  Note: Significance level *.05, **.01. Bold p-values indicate significance for the Examiner Belief variable. Gender was coded as males = 0, females = 1. User status was coded as non-user = 0, user = 1. Motivational condition was coded as neutral statement = 0, motivational statement = 1. Differences in examiners’ judgments based upon prior cannabis experience To examine whether examiners’ judgments varied depending upon their own experience with cannabis, examiners (n = 6) completed a secured anonymous survey disclosing their own experiences with cannabis use. Two of the six examiners disclosed that they had previous experience using cannabis. Therefore, the authors analyzed whether examiners with cannabis experience differed in the accuracy of their judgments of participants’ user status, relative to examiners with no cannabis experience. Exclusion of examiners with cannabis experience from statistical analyses did not alter study conclusions: overall concordance remained significantly better than chance alone at 64.91% (37/57: exact 95%, CI [51.13% –77.09%]). Similarly, sensitivity excluding these two examiners was 68.42% (26/38: exact, 95% CI [51.35% –82.50%]), and PPV was 76.47% (26/34: exact, 95% CI [58.83% –89.25%]), indicating that even cannabis-naïve examiners accurately predicted cannabis use at rates better than chance (ps < .05). Both specificity at 57.89% (11/19: exact, 95% CI [33.50% –79.75%]) and NPV at 47.83% (11/23: exact, 95% CI [26.82% –69.41%]) remained non-significant, consistent with previous results. A larger sample of examiners would be required to achieve sufficient power to conduct subgroup analyses of examiners who have and have not previously used cannabis. However, the current results suggest that there were minimal differences in accuracy between these groups. Examiners’ beliefs about the effects of cannabis use To examine what factors may have contributed to examiners’ expectancies of neuropsychological performance, examiners also answered questions about their personal beliefs regarding the effect of chronic cannabis use on cognitive functioning. Examiners rated the extent to which they believed regular marijuana use (here defined as >4 times per week for at least 1 year; Gonzalez, Carey, & Grant, 2002) would affect cognitive functioning, based upon the expected number of standard deviations’ difference above or below premorbid (baseline) functioning. For example, a mean examiner rating of −1.0 suggests that examiners believed cannabis users would perform about one standard deviation below baseline on neuropsychological testing due to the effects of regular cannabis use. Examiners’ ratings illustrated that they believed cannabis would have the greatest effect on processing speed (x− = −0.67, SD = 0.26), attention (x− = −0.50, SD = 0.0), memory (x− = −0.42, SD = 0.20), and motor skills (x− = −0.33, SD = 0.41), with negligible effects on executive functioning (x− = −0.25, SD = 0.42), Full Scale IQ (x− = −0.17, SD = 0.26), language (x− = −0.08, SD = 0.20), visuospatial construction (x− = –0.08, SD = 0.20), and effort to perform well on testing (x− = 0.17, SD = 0.41). It is interesting to note that examiners did not anticipate that cannabis would have more of an effect than about a half a standard deviation below baseline functioning on any cognitive domains, indicating that they anticipated a relatively small effect on cognition. Examiners also reported whether they support the legalization of recreational cannabis use. Half (3/6) reported “Yes,” whereas two examiners (33.3%) stated “Maybe, under certain regulations” and one reported “No” (16.7%). Therefore, most of the examiners viewed cannabis legalization favorably. Examiners also rated how confident they felt about the accuracy of their judgments regarding participant cannabis user status. The mean confidence level across examiners’ ratings of participants was 47.33% (SD = 35.34%), indicating that most examiners did not believe their judgments of cannabis user status were more accurate than chance. Further, examiners answered whether they believed their perceptions of the participants’ cannabis user status affected interactions with participants. Out of the six examiners, one (16.7%) reported “Maybe,” whereas three (50.0%) reported “Probably not,” one (16.7%) reported “Definitely not,” and one (16.7%) reported “Almost certainly not.” Thus, most examiners did not believe they were accurate in judging user status and did not believe their perceptions affected interactions with the participants, despite the fact that our results suggest both judgment accuracy and an effect on participant performance. Additional exploratory analyses analyzed whether examiners’ personal expectations of chronic cannabis effects on neuropsychological performance was correlated with actual examinee performance. While most correlations were not significant, there was a positive correlation between examiners’ expectations that memory would be affected by chronic cannabis use and examinees’ performance on the CVLT-II Delayed Free Recall (r = 0.299, p = .02). This relationship indicates that examiners’ expectations of lower memory performance were in fact associated with poorer performance on a memory measure. This correlation aligns with the previously described finding that examiner judgment of user status predicted CVLT-II performance. However, this finding should be interpreted with caution, as most correlations were not statistically significant. Discussion Results from the current study provide evidence that examiners’ judgments of participant cannabis user status may in fact influence examinees’ performance on neuropsychological assessment. Even after controlling for confounding variables (age, gender, actual user status, and motivational condition), individuals judged to be cannabis users performed worse on two of eight neuropsychological tests, relative to individuals judged as non-users. These results suggest that examiners’ perceptions of cannabis user status represent a potential confound that can significantly affect the results of examinee neuropsychological performance. Interestingly, there was a trend across several other neuropsychological subtests following the same pattern, with differences between those judged as users and those as non-users reaching marginal significance (p < .10) after accounting for actual user status and other confounds. Indeed, there was a medium effect of examiner belief on the CVLT-II Trial B Free Recall (Cohen’s d = 0.59), TMT A (Cohen’s d = 0.43), and the WMT Immediate Recall (Cohen’s d = 0.59). Thus, this trend suggests that even after controlling for actual user status, participants judged as users performed worse on many of the neuropsychological tests compared to those judged as non-users. While the trend should be interpreted with caution, it does suggest that greater power (e.g., a larger sample size) could result in statistically significant group differences between examiner-judged users and examiner-judged non-users on even more tests in the battery. Within the present study, examiners accurately judged participant cannabis user status at rates better than chance (sensitivity = 68%), and about three-quarters of examiners’ cannabis user judgments were accurate (PPV = 76%). These results are consistent with previous findings suggesting that laymen in general, as well as neuropsychologists specifically, can discriminate between cannabis users and non-users based upon appearance alone (Hirst et al., 2016, 2017). Importantly, characteristics of the examiners in this study are consistent with those of researchers analyzing the effects of cannabis on cognition: they had at least one year of training in neuropsychological assessment specifically, and were familiar with empirical literature examining the cognitive effects of cannabis. Thus, their accuracy rates are consistent with those of other cannabis researchers, as well as neuropsychologists in general (Hirst et al., 2016). Conversely, non-users’ statuses were not judged at a rate above chance levels (specificity = 55%) and only about half of the non-user predictions were accurate (NPV = 46% accuracy). Due to the comparatively smaller number of non-users (n = 20, vs. n = 41 cannabis users) and individuals judged as non-users (n = 24, vs. n = 37 judged as users), it is possible that greater statistical power would permit more reliable estimations of specificity and NPV. Studies including larger samples of both non-users and individuals judged to be non-users might have greater power to determine whether non-users can be judged accurately. Regardless, in this sample, judgment accuracy rates were clearly much higher in the user group relative to the non-user group. Therefore, the present findings suggest that examiner expectancy effects are more likely to negatively affect cannabis users because they are more accurately judged as users. It is important to note that although examiners were more accurate in judging users than non-users, controlling for actual user status in the regression permits analysis of only the expectancy effect and its impact on cognitive performance. Thus, the expectancy effect shown was above and beyond the impact of actual cannabis user status on neuropsychological functioning. While several previous studies examining the cognitive effects of cannabis utilize a research paradigm in which examiners are blind to participants’ user status, the present findings suggest that these studies remain vulnerable to the expectancy effect confound if examiners are, consciously or unconsciously, guessing participant user status. Although the sample of examiners was too small for formal statistical comparisons, exploratory analyses examined the examiners’ expectations about the effects of cannabis use. The examiners in this study reported that they expected users to perform only about 0.5 standard deviations below baseline in the domains of memory, attention, and processing speed. It is particularly intriguing that the present findings revealed an expectancy effect confound, given that examiners anticipated only small effects of cannabis users on neuropsychological performance and generally viewed cannabis legalization favorably. Further exploratory analyses revealed that examiners’ expectations of memory effects correlated with examinees’ performance on one of the memory measures (CVLT-II Delayed Free Recall). This relationship is consistent with the finding that examiner judgment of user status predicted performance on the CVLT-II. While these results should be interpreted with caution, as most correlations were not significant, this finding does raise the possibility that examiners’ personal beliefs regarding cannabis effects could contribute to the expectancy effect confound on measures of memory. In addition, most examiners did not believe their accuracy in judging user status was better than chance, and also did not believe that their perceptions about user status affected their interactions with participants. Again, these responses make it even more interesting that statistically significant expectancy effects were identified. The present findings are consistent with a similar line of research that investigated the impact of examiner expectancies on examinee cognitive performance following caffeine administration. Results from that study showed that examiners’ expectations regarding the effects of caffeine administration influenced the examinees’ physiological response (i.e., heart rate), as well as performance on neuropsychological assessment (Walach et al., 2001). Taken together with the current results, it is apparent that examiner expectations can influence examinees’ performance, even with the standardized administration utilized in neuropsychological assessment. Furthermore, these findings allude to important clinical implications as well. Clinicians have access to information regarding patient cannabis use history; therefore, it is possible that neuropsychologists’ personal beliefs regarding the effects of cannabis on cognitive functioning could affect patients’ performance on clinical neuropsychological evaluations. Clinicians’ expectancies may, consciously or unconsciously, influence test administration, thus affecting the interpretation of test results, recommendations, or clinical decision-making. Expectancy effects are vastly under-studied in the field of neuropsychology, as the authors were unable to identify any other studies measuring examiner expectancy effects on examinee neuropsychological performance. The current findings suggest that the field of neuropsychology would benefit from further exploration of examiner expectancy effects in all populations, including individuals with other types of substance use, psychiatric disorders, or neurological disorders, as expectancy effects could occur in any study using neuropsychological assessment. One interpretation of the present findings may be to suggest minimizing expectancy effects with computerized administration of neuropsychological testing, as has become increasingly prevalent. In this study, there was no statistically significant expectancy effect shown on the WMT (a computer-administered test). However, the medium effect (Cohen’s d = 0.59) of examiner belief on WMT performance suggests that even computerized tests may not be immune to experimenter expectancy effects. Further, research suggests that computerized administration is not as sensitive to subtle cognitive deficits as clinician-administered, paper-and-pencil testing (e.g., Register-Mihalik et al., 2013). In clinical settings, the benefit of clinical judgment throughout the evaluation process cannot be understated in determining neuropsychological functioning (Bauer et al., 2012). Given that the present findings on computerized testing showed a medium effect of examiner expectancies but no statistically significant group differences, future studies with greater power are needed to directly compare participants’ performance on computer-administered tests to examiner-administered tests to determine whether expectancy effects are truly reduced in participants completing cognitive tests on a computer. However, the possibility that this was a spurious result cannot be ruled out. There are several limitations to note within the present study. This study sought to recruit “pure” cannabis users (i.e., those with no other significant substance use or comorbid medical/psychiatric disorders) in an attempt to limit other confounding variables that may potentially affect cognitive functioning. Thus, the “pure” cannabis user sample in this study may not be representative of the “typical” cannabis user, who may have comorbid substance use or other diagnoses. Additionally, a larger sample size in the present study may have revealed expectancy effects on more assessments within the battery, due to greater power. Another limitation of the present study may be that the authors did not objectively verify the accuracy of self-reported cannabis use or other eligibility criteria. However, as interested individuals were unaware of eligibility criteria during the phone screen, they had little reason to purposely misrepresent their substance use or other history. Additionally, data regarding examiners’ personal history of cannabis use was anonymously collected and de-identified after all examinees’ data were collected, to reduce the threat to integrity of accurate examiner self-reporting. In these ways, we attempted to minimize the impact of potential inaccurate estimations of cannabis use. Another limitation of the present study was its correlational analysis, with no direct manipulation of examiner expectancies. However, the authors purposely selected this correlational design in order to more closely approximate the authentic researcher/clinician and participant/patient relationship that occurs in research studies and clinical contexts. In real-world situations, examiners indeed present with their own inherent personal beliefs about cannabis effects and thus are not directly told how to perceive participants/patients. In addition, directly manipulating examiners’ expectancies of examinee performance could have ethical implications that are not desirable. Further, given that examiners’ expectancies were not manipulated, it is even more interesting that significant expectancy effects were found. It is possible that the act of guessing whether participants were cannabis users or non-users prior to testing increased examiners’ vulnerability to the expectancy effect confound by making their expectancies more salient during the testing process. We chose this design to ensure that examiners’ judgments were made based upon participants’ appearance rather than test performance; however, future studies could examine whether rating participants after testing results in larger or smaller expectancy effects. A larger sample of examiners in future studies would also allow for greater confidence in the conclusions of the exploratory analyses run in this study. The present study is also limited by a lack of measurement of the physical factors examiners used to make their cannabis use judgments, and to the authors’ knowledge, this has not been examined in any prior studies. Future studies should assess these factors in more detail to determine what visual cues are considered when deciding individuals’ likelihood of cannabis use. Finally, future studies would also benefit from measuring examiners’ other perceptions about participants (e.g., mood, intellect, other substance use) to determine how it relates to cannabis use perceptions as well as participants’ cognitive performance. The present findings shed light on the impact of examiner expectancy effects on the neuropsychological performance of chronic cannabis users. Examiners who were blind to participants’ user status accurately identified cannabis users at rates better than chance, and individuals judged as cannabis users performed worse on several neuropsychological tests even after controlling for potential confounding variables. While previous studies have utilized a blind research design in an attempt to control for examiner bias, it is important to realize that this approach is inadequate if examiners are able to guess participant user status. The results of the current study suggest that even studies using blind examiners are vulnerable to the expectancy effect confound, and that these expectancies can influence examinee performance and thus, the validity of study outcomes. Future research should assess for examiner expectancy effects by measuring examiners’ ratings of participant group status and including those ratings as a covariate in analyses to determine whether examiner expectancies did in fact influence study participants’ neuropsychological performance. This is a vastly under-studied topic in the field of neuropsychology, and the present results emphasize the need for further exploration of this topic in additional populations. Conflict of interest None declared. References American Psychiatric Association. ( 2000). Diagnostic and Statistical Manual of Mental Disorders–Fourth Edition-Text Revision (DSM–IV-TR) . Washington D.C.: American Psychiatric Association. Battisti, R. A., Roodenrys, S., Johnstone, S. J., Respondek, C., Hermens, D. F., & Solowij, N. ( 2010). Chronic use of cannabis and poor neural efficiency in verbal memory ability. Psychpharmacology , 209, 319– 330. doi:10.1007/s00213-010-1800-4. Google Scholar CrossRef Search ADS   Bauer, R. M., Iverson, G. L., Cernich, A. N., Binder, L. M., Ruff, R. M., & Naugle, R. I. ( 2012). Computerized neuropsychological assessment devices: Joint position paper of the American Academy of Clinical Neuropsychology and the National Academy of Neuropsychology. Archives of Clinical Neuropsychology , 27, 362– 373. doi:10.1093/arclin/acs027. Google Scholar CrossRef Search ADS PubMed  Blair, J. R., & Spreen, O. ( 1989). Predicting premorbid IQ: A revision of the National Adult Reading Test. The Clinical Neuropsychologist , 3, 129– 136. doi:10.1080/13854048908403285. Google Scholar CrossRef Search ADS   Bolla, K. I., Brown, K., Eldreth, D., Tate, K., & Cadet, J. L. ( 2002). Dose-related neurocognitive effects of marijuana use. Neurology , 59, 1337– 1343. doi:10.1212/01.WNL.0000031422.66442.49. Google Scholar CrossRef Search ADS PubMed  Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. ( 2000). California Verbal Learning Test  ( 2nd ed.). San Antonio, TX: Psychological Corporation. Dougherty, D. M., Mathias, C. W., Dawes, M. A., Furr, R. M., Charles, N. E., Liguori, A., et al.  . ( 2013). Impulsivity, attention, memory, and decision-making among adolescent marijuana users. Psychopharmacology , 226, 307– 319. doi:10.1007/s00213-012-2908-5. Google Scholar CrossRef Search ADS PubMed  Etherton, J. L., Bianchini, K. J., Greve, K. W., & Heinly, M. T. ( 2005). Sensitivity and specificity of Reliable Digit Span in malingered pain-related disability. Assessment , 12, 130– 136. doi:10.1177/1073191105274859. Google Scholar CrossRef Search ADS PubMed  Fink, B., Neave, N., Manning, J. T., & Grammer, K. ( 2006). Facial symmetry and judgments of attractiveness, health and personality. Personality and Individual Differences , 41, 491– 499. doi:10.1016/j.paid.2006.01.017. Google Scholar CrossRef Search ADS   Gonzalez, R., Carey, C., & Grant, I. ( 2002). Nonacute (residual) neuropsychological effects of cannabis use: A qualitative analysis and systematic review. Journal of Clinical Pharmacology , 42, 48S– 57S. doi:10.1002/j.1552-4604.2002.tb06003.x. Google Scholar CrossRef Search ADS PubMed  Grant, I., Gonzalez, R., Carey, C. I., Natarajan, L., & Wolfson, T. ( 2003). Nonacute (residual) neurocognitive effects of cannabis use: A meta analytic study. Journal of the International Neuropsychological Society , 9, 679– 689. doi:10.1017/S1355617703950016. Google Scholar CrossRef Search ADS PubMed  Green, P., Allen, L. M., & Astner, K. ( 1996). The Word Memory Test: A manual for the oral and computerized forms. . Durham, NC: Cognisyst, Inc. Gruber, S. A., Sagar, K. A., Dahlgren, M. K., Racine, M., & Lukas, S. E. ( 2012). Age of onset of marijuana use and executive function. Psychology of Addictive Behaviors , 26, 496– 506. doi:10.1037/a0026269. Google Scholar CrossRef Search ADS PubMed  Hirst, R. B., Teague, A., Sodos, L., Wickham, R. E., Whittington, L. T., Mills, B., et al.  . ( 2016). Determining cannabis use status from a photograph: An assessment of the “jay-dar” in neuropsychologists. Substance Use and Misuse , 52, 401– 410. doi:10.1080/10826084.2016.1233564. Google Scholar CrossRef Search ADS PubMed  Hirst, R. B., Enriquez, R., Wickham, R. E., Gretler, J., Sodos, L. M., Gade, S. A., et al.  . ( 2017). Marijuana stereotypes and the “jay-dar”: Perceptions of cannabis use and memory abilities based upon appearance. Personality and Individual Differences , 110, 131– 138. doi:10.1016/j.paid.2016.12.056. Google Scholar CrossRef Search ADS   Johnston, L. D., O’Malley, P. M., Bachman, J. G., Schulenberg, J. E., & Miech, R. A. ( 2015). Monitoring the future national survey results on drug use, 1975–2014: Volume II, college students and adults ages 19-55. Ann Arbor: Institute for Social research, The University of Michigan. Retrieved from. http://www.monitoringthefuture.org/pubs/monographs/mtf-vol2_2014.pdf. Karver, M. S., Handelsman, J. B., Fields, S., & Bickman, L. ( 2005). A theoretical model of common process factors in youth and family therapy. Mental Health Services Research , 7, 35– 51. doi:10.1007/s11020-005-1964-4. Google Scholar CrossRef Search ADS PubMed  Lyketsos, C. G., Garrett, E., Liang, K. Y., & Anthony, J. C. ( 1999). Cannabis use and cognitive decline in persons under 65 years of age. American Journal of Epidemiology , 149, 794– 800. doi:10.1093/oxfordjournals.aje.a009894. Google Scholar CrossRef Search ADS PubMed  Lyons, M. J., Bar, J. L., Panizzon, M. S., Toomey, R., Eisen, S., Xian, H., et al.  . ( 2004). Neuropsychological consequences of regular marijuana use: A twin study. Psychological Medicine , 34, 1239– 1250. doi:10.1017/S0033291704002260. Google Scholar CrossRef Search ADS PubMed  Macher, R. B., & Earleywine, M. ( 2012). Enhancing neuropsychological performance in chronic cannabis users: The role of motivation. Journal of Clinical and Experimental Neuropsychology , 34, 405– 415. doi:10.1080/13803395.2011.646957. Google Scholar CrossRef Search ADS PubMed  Mahmood, O. M., Jacobus, J., Bava, S., Scarlett, A., & Tapert, S. F. ( 2010). Learning and memory performances in adolescent users of alcohol and marijuana: Interactive effects. Journal of Studies on Alcohol and Drugs , 71, 885– 894. doi:10.15288/jsad.2010.71.885. Google Scholar CrossRef Search ADS PubMed  Medina, K. L., Hanson, K. L., Schweinsburg, A. D., Cohen-Zion, M., Nagel, B. J., & Tapert, S. F. ( 2007). Neuropsychological functioning in adolescent marijuana users: Subtle deficits detectable after a month of abstinence. Journal of the International Neuropsychological Society , 13, 807– 820. doi:10.1017/S1355617707071032. Google Scholar CrossRef Search ADS PubMed  Meier, M. H., Caspi, A., Ambler, A., Harrington, H., Houts, R., Keefe, R. S. E., et al.  . ( 2012). Persistent cannabis users show neuropsychological decline from childhood to midlife. Proceedings of the National Academy of Sciences , 109, E2657– E2664. doi:10.1073/pnas.1206820109. Google Scholar CrossRef Search ADS   Meyers, J. E., & Meyers, K. R. ( 1995). Rey Complex Figure Test and Recognition Trial, professional manual. . Odessa, FL: Psychological Assessment Resources (PAR). Moore, B. A., & Donders, J. ( 2004). Predictors of invalid neuropsychological test performance after traumatic brain injury. Brain Injury , 18, 975– 984. doi:10.1080/02699050410001672350. Google Scholar CrossRef Search ADS PubMed  Olivola, C. Y., & Todorov, A. ( 2010). Fooled by first impressions? Reexamining the diagnostic value of appearance-based inferences. Journal of Experimental Social Psychology , 46, 315– 324. doi:10.1016/j.jesp.2009.12.002. Google Scholar CrossRef Search ADS   Pope, H. G., Gruber, A. J., Hudson, J. I., Huestis, M. A., & Yurgelun-Todd, D. ( 2001). Neuropsychological performance in long-term cannabis users. Archives of General Psychiatry , 58, 909– 915. doi:10.1001/archpsyc.58.10.909. Google Scholar CrossRef Search ADS PubMed  Register-Mihalik, J. K., Guskiewicz, K. M., Mihalik, J. P., Schmidt, J. D., Kerr, Z. Y., & McCrea, M. A. ( 2013). Reliable change, sensitivity, and specificity of a multidimensional concussion assessment battery: Implications for caution in clinical practice. Journal of Head Trauma Rehabilitation , 28, 274– 283. doi:10.1097/HTR.0b013e3182585d37. Google Scholar CrossRef Search ADS PubMed  Rodgers, J. ( 2000). Cognitive performance among recreational users of “ecstasy. Psychopharmacology , 151, 19– 24. doi:10.1007/s002130000467. Google Scholar CrossRef Search ADS PubMed  Rosenthal, R. ( 1991). Meta-analytic procedures for social research . Beverly Hills, CA: Sage. Google Scholar CrossRef Search ADS   Rosenthal, R. ( 2002). Experimenter and clinician effects in scientific inquiry and clinic practice. Prevention and Treatment , 5. doi.org/10.1037/1522-3736.5.1.538c. Rosenthal, R., Kohn, P., Greenfield, P. M., & Carota, N. ( 1966). Data desirability, experimenter expectancy, and the results of psychological research. Journal of Personality and Social Psychology , 3, 20– 27. doi:10.1037/h0022604. Google Scholar CrossRef Search ADS PubMed  Reitan, R. M., & Wolfson, D. ( 1992). The Halstead-Reitan Neuropsychological Battery: Theory and clinical interpretation  ( (2nd ed.)). South Tucson, AZ: Neuropsychology Press. Ruffolo, L. F., Guilmette, T. J., & Willis, G. W. ( 2000). Comparison of time and error rates on the Trail Making Test among patients with head injuries, experimental malingerers, patients with suspect effort on testing, and normal controls. The Clinical Neuropsychologist , 14, 223– 230. doi:10.1076/1385-4046(200005)14:2;1-Z;FT223. Google Scholar CrossRef Search ADS PubMed  Schreiner, A. M., & Dunn, M. E. ( 2012). Residual effects of cannabis use on neurocognitive performance after prolonged abstinence: A meta-analysis. Experimental and Clinical Psychopharmacology , 20, 420– 429. doi:10.1037/a0029117. Google Scholar CrossRef Search ADS PubMed  Shelp, S. G. ( 2003). Gaydar. Journal of Homosexuality , 44 ( 1), 1– 14. doi:10.1300/J082v44n01_01. Google Scholar CrossRef Search ADS PubMed  Solowij, N. ( 1995). Do cognitive impairments recover following cessation of cannabis use? Life Sciences , 56, 2119– 2126. doi:10.1016/0024-3205(95)00197-E. Google Scholar CrossRef Search ADS PubMed  Solowij, N., Stephens, R. S., Roffman, R. A., Babor, T., Kadden, R., Miller, M., et al.  . ( 2002). Cognitive functioning of long term heavy cannabis users seeking treatment. Journal of the American Medical Association , 287, 1123– 1131. doi:10.1001/jama.287.9.1123. Google Scholar CrossRef Search ADS PubMed  Todorov, A., Pakrashi, M., & Oosterhof, N. N. ( 2009). Evaluating faces on trustworthiness after minimal time exposure. Social Cognition , 27, 813– 833. doi:10.1521/soco.2009.27.6.813. Google Scholar CrossRef Search ADS   Tombaugh, T. N. ( 1996). Test of Memory Malingering . Toronto, Ont: Multihealth Systems. United Nations Office on Drugs and Crime. ( 2016). World Drug Report 2016 (United Nations publication, Sales No. E.16.XI.7) . Washington, DC: U.S. Government Printing Office. Walach, H., Schimdt, S., Bihr, Y.-M., & Wiesch, S. ( 2001). The effects of a caffeine placebo and experimenter expectation on blood pressure, heart rate, well-being, and cognitive performance. European Psychologist , 6, 15– 25. doi:10.1027//1016-9040.6.1.15. Google Scholar CrossRef Search ADS   Wechsler, D. ( 1997). Wechsler Adult Intelligence Scale-Third Edition . San Antonio, TX: The Psychological Corporation. Wiens, A. N., Bryan, J. E., & Crossen, J. R. ( 1993). Estimating WAIS-R FSIQ from National Adult Reading Test-Revised in normal subjects. The Clinical Neuropsychologist , 7, 70– 84. doi:10.1080/13854049308401889. Google Scholar CrossRef Search ADS   Willis, J., & Todorov, A. ( 2006). First impressions: Making up your mind after a 100-ms exposure to a face. Psychological Science , 17, 592– 598. doi:10.1111/j.1467-9280.2006.01750.x. Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Journal

Archives of Clinical NeuropsychologyOxford University Press

Published: Jan 12, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off