TY - JOUR
AU - Bosco, Francesca M.
AB - Objective Schizophrenia is associated with a severe impairment in the communicative-pragmatic domain. Recent research has tried to disentangle the relationship between communicative impairment and other domains usually impaired in schizophrenia, i.e. Theory of Mind (ToM) and cognitive functions. However, the results are inconclusive and this relationship is still unclear. Machine learning (ML) provides novel opportunities for studying complex relationships among phenomena and representing causality among multiple variables. The present research explored the potential of applying ML, specifically Bayesian network (BNs) analysis, to characterize the relationship between cognitive, ToM and pragmatic abilities in individuals with schizophrenia and healthy controls, and to identify the cognitive and pragmatic abilities that are most informative in discriminating between schizophrenia and controls. Methods We provided a comprehensive assessment of different aspects of pragmatic performance, i.e. linguistic, extralinguistic, paralinguistic, contextual and conversational, ToM and cognitive functions, i.e. Executive Functions (EF)—selective attention, planning, inhibition, cognitive flexibility, working memory and speed processing—and general intelligence, in a sample of 32 individuals with schizophrenia and 35 controls. Results The results showed that the BNs classifier discriminated well between patients with schizophrenia and healthy controls. The network structure revealed that only pragmatic Linguistic ability directly influenced the classification of patients and controls, while diagnosis determined performance on ToM, Extralinguistic, Paralinguistic, Selective Attention, Planning, Inhibition and Cognitive Flexibility tasks. The model identified pragmatic, ToM and cognitive abilities as three distinct domains independent of one another. Conclusion Taken together, our results confirmed the importance of considering pragmatic linguistic impairment as a core dysfunction in schizophrenia, and demonstrated the potential of applying BNs in investigating the relationship between pragmatic ability and cognition. Introduction Communicative impairment has been considered as a core feature of schizophrenia since the condition was first described [1,2], and in recent decades a large body of evidence has confirmed that individuals with schizophrenia (SCZ) show pervasive difficulties in the communicative-pragmatic domain [3–8]. Pragmatic ability can be defined as the use of language and other expressive means, such as gestures, facial expressions and prosody, to act communicatively in a certain context [9,10]. Communicative-pragmatic deficits are frequent in SCZ, resulting in difficulties in the comprehension of non-literal communicative acts, such as indirect speech acts [11] and irony [4,12], and figurative expressions, like metaphors and proverbs [13–15], in understanding deceit [7,16], in recognizing and recovering communicative failures [17], together with deficits in narrative production [18] and conversational skills [19,20]. Pragmatic deficits seem to encompass both comprehension and production, as well as different communicative modalities, such as linguistic, extralinguistic (e.g. gestures), and paralinguistic (e.g. prosody). Patients are, in fact, often impaired in the ability to use and comprehend gestures and prosodic cues to express and recognize communicative intentions and emotional content [8,21,22]. The ability to use language and other expressive means to communicate effectively in social contexts is a high-level process that relies on the complex interaction of different functions, like Executive Functions (EF) and Theory of Mind (ToM) [8,23,24]. However, it is still not completely clear which functions are responsible for communicative difficulties. Recent research has mainly focused on investigating the relationship between communicative-pragmatic impairment and two other domains that are usually impaired in schizophrenia, i.e. ToM and EF. Theory of Mind, i.e., the ability to infer others’ mental states, such as beliefs and intentions [25], has been considered a necessary prerequisite for communication [26–29], and several studies have reported an association between a deficit in ToM and pragmatic difficulties in patients with SCZ in various pragmatic tasks [4,5,14,17,20,30]. Some authors have proposed that a deficit in EF, i.e. the ability to manage goal-directed behavior in a flexible and adaptive way [31], is the primary impairment responsible for the communicative difficulties observed in SCZ, and provided empirical evidence supporting the relationship between EF deficit and pragmatic performance [18,32–34]. However, other studies have indicated no association between pragmatic ability and ToM [14,35], or EF [8,20,36]. A few studies have examined the role of both cognitive functions and ToM concomitantly [8,16,20,37,38]. In a recent study [8] the authors evaluated the role of EF and ToM in explaining the communicative-pragmatic performance of patients with schizophrenia. The results showed the primary role of ToM in explaining patients’ performance in linguistic, but not extralinguistic, comprehension and production tasks, while EFs did not show any significant role in explaining pragmatic performance. Other studies [8,20,37,39] reported an association between ToM and pragmatic ability, even when controlling for the role of EF. However, the nature and strength of this association differed greatly according to the pragmatic tasks used in the different studies. Considered as a whole, the results of previous studies are sometimes conflicting and not conclusive, thus making it difficult to draw firm conclusions about the relationship between pragmatic disorders and cognitive functions in schizophrenia. Different reasons can explain the variability observed in previous results, and may have prevented to establish robust and reliable findings across studies. Schizophrenia is a highly heterogeneous disorder, with a large natural variability in terms of communicative profiles, clinical features and disease severity. However, previous studies have rarely taken such heterogeneity into account, thus limiting out-of-sample generalizability. A second main issue concerns the very notion of pragmatic ability. Pragmatics involves the appropriate use of a wide range of expressive means, such as language, gestures, prosody, as well as the ability to flexibly adapt our communicative behavior to a specific context, to manage a conversation, to use inferential ability to reconstruct the speaker’s communicative intention. Given the complexity and variety of communicative behaviors, pragmatic ability relies upon different interacting cognitive systems. For this reason, the causal relationship between pragmatic ability and cognition is difficult to define, and the direction of this relationship often unclear. For example, several authors have claimed that ToM is a necessary requisite for developing social communication ability [40–42]; however, experience in and exposure to communicative exchanges may itself improve ToM ability [43]. As a result, cognitive, ToM and communicative factors interact in a complex and bidirectional way, thus often resulting in non-linear and multivariate patterns. Most of the previous studies adopted a classic statistical methodology (e.g., null-hypothesis testing) to explore the relationships between cognitive functions and pragmatic ability. However, this statistical framework has some limitations for representing causality among multiple variables interacting in a complex, non-linear and multivariate way [44,45]. In addition, by focusing more on explaining current measurements than on predicting new data, classical statistical methods may suffer from methodological limitations, such as multicollinearity and overfitting, which may have limited replicability and out-of-sample generalization of previous results. In contrast, machine learning (ML) methods provide some important advantages over classical statistical methods [44–46]. Indeed, ML uses multiple features to build models aimed at accurately discriminating among different classes—SCZ vs. HC -, and to represent multivariate and non-linear relationships among these variables. Furthermore, the focus on prediction and multivariate analysis offers ML methods greater sensitivity and higher generalization even in case of data heterogeneity. In recent years, several attempts have been made to apply ML techniques to support the diagnostic process in neuropsychiatric and neurological disorders [47,48], identifying the symptom network and predicting the polarity of symptoms [49,50], predicting outcome after stroke and neurological disorders [51,52], representing connectivity between cerebral networks [53], with promising results. These studies showed a higher accuracy in discriminating between patients and control, or in predicting symptomatology, compared to classical univariate techniques (see also [21,54]), thus suggesting a high potential for clinical translation (e.g. to enhance clinical diagnostic and monitoring processes, see also [55,56]). Among the ML methods, the Bayesian network (BNs) is particularly suitable for representing causality among multiple variables in conditions of high variability, as in the case of investigating the interplay between cognition, ToM and pragmatic ability in clinical conditions like schizophrenia. Aims of the study In the present paper we take a further step beyond previous research [8] to evaluate, for the first time in the current literature, the potential of applying Bayesian network analysis to characterize the functional relationship among cognitive variables, i.e. Theory of Mind and Executive Functions, and pragmatic ability in patients with schizophrenia and controls. We aimed to identify the most informative cognitive and pragmatic abilities in discriminating between patients with schizophrenia and controls, and to represent the interaction among these variables and diagnosis of schizophrenia. We provided a comprehensive assessment of participants’ pragmatic ability, using the Assessment Battery for Communication to identify how the ability to use different expressive means, e.g., linguistic, extralinguistic, paralinguistic, interacts with Theory of Mind and Executive functions. Aims of the study In the present paper we take a further step beyond previous research [8] to evaluate, for the first time in the current literature, the potential of applying Bayesian network analysis to characterize the functional relationship among cognitive variables, i.e. Theory of Mind and Executive Functions, and pragmatic ability in patients with schizophrenia and controls. We aimed to identify the most informative cognitive and pragmatic abilities in discriminating between patients with schizophrenia and controls, and to represent the interaction among these variables and diagnosis of schizophrenia. We provided a comprehensive assessment of participants’ pragmatic ability, using the Assessment Battery for Communication to identify how the ability to use different expressive means, e.g., linguistic, extralinguistic, paralinguistic, interacts with Theory of Mind and Executive functions. Methods Participants The sample of the present study included 32 individuals (25 males) with a diagnosis of schizophrenia according to the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV [57]). The age of the participants with schizophrenia ranged from 22 to 57 years (M = 40.17 years; SD = 10.19); their education level ranged from 8 to 14 years (M = 10.59 years, SD = 2.46). Participants with schizophrenia were all in a chronic phase of the illness and stable in clinical terms: none of them had been hospitalized during the 6 months prior to the study, nor had they undergone any changes to their antipsychotic therapy during the previous 3 months. Their symptomatology was evaluated by an expert clinician with the Positive and Negative Syndrome Scale (PANSS [58]), specifically using the PANSS total score (M = 45.64; SD = 19.02), negative (M = 20.28; SD = 9.65) and positive symptoms (M = 18.83; SD = 8.89) indexes. Cut-off scores were set, as inclusion criteria, on the following neuropsychological tests, in order to rule out any severe cognitive or linguistic deficits: (a) Mini-Mental State Examination (MMSE [59]). Cut-off 24/30; (b) Token Test [60]. Cut-off 5/6; (c) Denomination scale of the Aachener Aphasie test (AAT [61]). Cut-off: no deficit. A group of 35 healthy individuals matched for sex (29 males), age (M = 39.46; SD = 10.95) and education (M = 10.57; SD = 2.46) were recruited. Only Italian native speakers were included. Exclusion criteria for all the participants were: (a) current and/or prior neurological disorder, (b) history of head injury, (c) substance abuse, (d) impaired hearing or vision. This study was carried out in accordance with the recommendations of ‘A.S.L. To2 ethics committee’ with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the ‘A.S.L. To2 ethics committee. Materials Assessment of communicative-pragmatic abilities. Communicative-pragmatic skills were assessed with the Assessment Battery for Communication (ABaCo) [62–64], a tool designed to evaluate comprehension and production of a wide range of pragmatic phenomena (see Table 1 for a description of the items and Table 2 for mean performance raw scores). The ABaCo is composed of 176 items organized in five evaluation scales, i.e., linguistic, extralinguistic, paralinguistic, context and conversational. For each item, the participants are required to comprehend or produce pragmatically relevant communicative acts either in response to short videos in which two actors play out a communicative exchange, or to vis-à-vis interaction with the examiner. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Description of the tools administered for the pragmatic and neuropsychological and ToM assessment. https://doi.org/10.1371/journal.pone.0229603.t001 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. The participants’ mean performance raw scores at pragmatic, neuropsychological and ToM tasks. https://doi.org/10.1371/journal.pone.0229603.t002 Two independent raters coded the video-recording of the sessions off-line according to the scoring procedure reported in the ABaCo manual. More details about the administration, coding, and structure of the ABaCo are provided in [63] and [65]. The Intraclass Correlation Coefficient (ICC) was used to calculate raters’ agreement, with a result of 0.88, indicating a high value of inter-rater agreement [66]. Neuropsychological assessment. To evaluate the participants’ cognitive abilities, we employed a neuropsychological battery assessing the cognitive functions relevant for determining their communicative performance. See Table 1. General intelligence was assessed with Raven’s Colored Progressive Matrices (RCPM, [67]), speed processing with the Trail Making test (Part A [68]), and selective attention using the Attentive Matrices [69]. We evaluated the following EFs: working memory with the Disyllabic Word Repetition test and Corsi’s Block-Tapping test [69], inhibitory control by administering the Modified Card Sorting test[70]; cognitive flexibility with the Trail Making Test (Part B–Part A [68]). The following tests were used to evaluate ToM performance: First-order/third person ToM with the Sally and Ann [71] and the Smarties tasks [72]. More advanced ToM abilities were evaluated with a selection of the Strange Stories tasks [28]. A description of each task is provided in Table 1, and mean performance raw scores in neuropsychological and ToM tasks are provided in Table 2. Data analysis. To analyze the data we used the Bayesian network approach. Bayesian networks (BN) [73] are probabilistic models that represent a set of variables and their conditional dependencies through a directed acyclic graph. Each node represents an attribute of the domain, and arcs between nodes represent conditional dependencies. For each node it is possible to calculate its conditional probability given the values of their parents. Each variable is assumed to be independent of its non-descendants given its set of parents. Under this assumption the joint probability distribution of all variables can be calculated. A BN can be used for classification when one of the variables is selected as the class attribute, and all other variables are input attributes. For an observation to be classified, the model calculates the posterior probability for each class marginalizing the joint probability distribution and labels the observation with the class that maximizes it. In this work, the attribute Type of subject was defined as the class attribute, and the input attributes were Sex, Age, Education, Linguistic, Extralinguistic, Paralinguistic, Context, Conversation, General Intelligence, Speed Processing, Selective Attention, Working Memory, Planning, Inhibition, Cognitive Flexibility, and Theory of Mind. The BN model was generated by Weka framework, version 3.8.3 [74]. The Weka learning algorithm used a hill climbing strategy, where several configurations of structures are examined, adding, deleting and reversing arcs between the nodes, such that maximize the likelihood of the class. To generate a simpler and more interpretative model, we set the maximum number of parents of a node as one. The initial count of the simple estimator in order to avoid zero frequencies was set as 0.1. The score type, which determines the measure used to assess the quality of the network structure, was the Minimum Description Length (MDL) [75]. The numeric attributes were discretized by Weka using Fayyad and Irani’s method [76]. We estimated the generalization performance of the BN model using cross-validation, which is a technique for evaluating predictive models by partitioning the original sample into a training set to create the model, and a test set to evaluate it. In our experiments, we used stratified 10-fold cross-validation, where the data is randomly split into ten sub-samples, each one with the same proportion of observations of each class. The process is then repeated ten times and each time a different sub-sample (fold) is used as a test set. Metrics presented are averaged across all folds, and are related to the test sets. We reported the following performance metrics: Accuracy, Sensitivity, Precision and Specificity. Accuracy is the proportion of the total number of classifications that were correct in both classes (SCZ and HC), Sensitivity gives the proportion of cases of schizophrenia classified correctly, Precision gives the proportion of the cases classified as schizophrenia that were correct, and Specificity gives the proportion of cases of controls classified correctly. Participants The sample of the present study included 32 individuals (25 males) with a diagnosis of schizophrenia according to the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV [57]). The age of the participants with schizophrenia ranged from 22 to 57 years (M = 40.17 years; SD = 10.19); their education level ranged from 8 to 14 years (M = 10.59 years, SD = 2.46). Participants with schizophrenia were all in a chronic phase of the illness and stable in clinical terms: none of them had been hospitalized during the 6 months prior to the study, nor had they undergone any changes to their antipsychotic therapy during the previous 3 months. Their symptomatology was evaluated by an expert clinician with the Positive and Negative Syndrome Scale (PANSS [58]), specifically using the PANSS total score (M = 45.64; SD = 19.02), negative (M = 20.28; SD = 9.65) and positive symptoms (M = 18.83; SD = 8.89) indexes. Cut-off scores were set, as inclusion criteria, on the following neuropsychological tests, in order to rule out any severe cognitive or linguistic deficits: (a) Mini-Mental State Examination (MMSE [59]). Cut-off 24/30; (b) Token Test [60]. Cut-off 5/6; (c) Denomination scale of the Aachener Aphasie test (AAT [61]). Cut-off: no deficit. A group of 35 healthy individuals matched for sex (29 males), age (M = 39.46; SD = 10.95) and education (M = 10.57; SD = 2.46) were recruited. Only Italian native speakers were included. Exclusion criteria for all the participants were: (a) current and/or prior neurological disorder, (b) history of head injury, (c) substance abuse, (d) impaired hearing or vision. This study was carried out in accordance with the recommendations of ‘A.S.L. To2 ethics committee’ with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the ‘A.S.L. To2 ethics committee. Materials Assessment of communicative-pragmatic abilities. Communicative-pragmatic skills were assessed with the Assessment Battery for Communication (ABaCo) [62–64], a tool designed to evaluate comprehension and production of a wide range of pragmatic phenomena (see Table 1 for a description of the items and Table 2 for mean performance raw scores). The ABaCo is composed of 176 items organized in five evaluation scales, i.e., linguistic, extralinguistic, paralinguistic, context and conversational. For each item, the participants are required to comprehend or produce pragmatically relevant communicative acts either in response to short videos in which two actors play out a communicative exchange, or to vis-à-vis interaction with the examiner. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Description of the tools administered for the pragmatic and neuropsychological and ToM assessment. https://doi.org/10.1371/journal.pone.0229603.t001 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. The participants’ mean performance raw scores at pragmatic, neuropsychological and ToM tasks. https://doi.org/10.1371/journal.pone.0229603.t002 Two independent raters coded the video-recording of the sessions off-line according to the scoring procedure reported in the ABaCo manual. More details about the administration, coding, and structure of the ABaCo are provided in [63] and [65]. The Intraclass Correlation Coefficient (ICC) was used to calculate raters’ agreement, with a result of 0.88, indicating a high value of inter-rater agreement [66]. Neuropsychological assessment. To evaluate the participants’ cognitive abilities, we employed a neuropsychological battery assessing the cognitive functions relevant for determining their communicative performance. See Table 1. General intelligence was assessed with Raven’s Colored Progressive Matrices (RCPM, [67]), speed processing with the Trail Making test (Part A [68]), and selective attention using the Attentive Matrices [69]. We evaluated the following EFs: working memory with the Disyllabic Word Repetition test and Corsi’s Block-Tapping test [69], inhibitory control by administering the Modified Card Sorting test[70]; cognitive flexibility with the Trail Making Test (Part B–Part A [68]). The following tests were used to evaluate ToM performance: First-order/third person ToM with the Sally and Ann [71] and the Smarties tasks [72]. More advanced ToM abilities were evaluated with a selection of the Strange Stories tasks [28]. A description of each task is provided in Table 1, and mean performance raw scores in neuropsychological and ToM tasks are provided in Table 2. Data analysis. To analyze the data we used the Bayesian network approach. Bayesian networks (BN) [73] are probabilistic models that represent a set of variables and their conditional dependencies through a directed acyclic graph. Each node represents an attribute of the domain, and arcs between nodes represent conditional dependencies. For each node it is possible to calculate its conditional probability given the values of their parents. Each variable is assumed to be independent of its non-descendants given its set of parents. Under this assumption the joint probability distribution of all variables can be calculated. A BN can be used for classification when one of the variables is selected as the class attribute, and all other variables are input attributes. For an observation to be classified, the model calculates the posterior probability for each class marginalizing the joint probability distribution and labels the observation with the class that maximizes it. In this work, the attribute Type of subject was defined as the class attribute, and the input attributes were Sex, Age, Education, Linguistic, Extralinguistic, Paralinguistic, Context, Conversation, General Intelligence, Speed Processing, Selective Attention, Working Memory, Planning, Inhibition, Cognitive Flexibility, and Theory of Mind. The BN model was generated by Weka framework, version 3.8.3 [74]. The Weka learning algorithm used a hill climbing strategy, where several configurations of structures are examined, adding, deleting and reversing arcs between the nodes, such that maximize the likelihood of the class. To generate a simpler and more interpretative model, we set the maximum number of parents of a node as one. The initial count of the simple estimator in order to avoid zero frequencies was set as 0.1. The score type, which determines the measure used to assess the quality of the network structure, was the Minimum Description Length (MDL) [75]. The numeric attributes were discretized by Weka using Fayyad and Irani’s method [76]. We estimated the generalization performance of the BN model using cross-validation, which is a technique for evaluating predictive models by partitioning the original sample into a training set to create the model, and a test set to evaluate it. In our experiments, we used stratified 10-fold cross-validation, where the data is randomly split into ten sub-samples, each one with the same proportion of observations of each class. The process is then repeated ten times and each time a different sub-sample (fold) is used as a test set. Metrics presented are averaged across all folds, and are related to the test sets. We reported the following performance metrics: Accuracy, Sensitivity, Precision and Specificity. Accuracy is the proportion of the total number of classifications that were correct in both classes (SCZ and HC), Sensitivity gives the proportion of cases of schizophrenia classified correctly, Precision gives the proportion of the cases classified as schizophrenia that were correct, and Specificity gives the proportion of cases of controls classified correctly. Assessment of communicative-pragmatic abilities. Communicative-pragmatic skills were assessed with the Assessment Battery for Communication (ABaCo) [62–64], a tool designed to evaluate comprehension and production of a wide range of pragmatic phenomena (see Table 1 for a description of the items and Table 2 for mean performance raw scores). The ABaCo is composed of 176 items organized in five evaluation scales, i.e., linguistic, extralinguistic, paralinguistic, context and conversational. For each item, the participants are required to comprehend or produce pragmatically relevant communicative acts either in response to short videos in which two actors play out a communicative exchange, or to vis-à-vis interaction with the examiner. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Description of the tools administered for the pragmatic and neuropsychological and ToM assessment. https://doi.org/10.1371/journal.pone.0229603.t001 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. The participants’ mean performance raw scores at pragmatic, neuropsychological and ToM tasks. https://doi.org/10.1371/journal.pone.0229603.t002 Two independent raters coded the video-recording of the sessions off-line according to the scoring procedure reported in the ABaCo manual. More details about the administration, coding, and structure of the ABaCo are provided in [63] and [65]. The Intraclass Correlation Coefficient (ICC) was used to calculate raters’ agreement, with a result of 0.88, indicating a high value of inter-rater agreement [66]. Neuropsychological assessment. To evaluate the participants’ cognitive abilities, we employed a neuropsychological battery assessing the cognitive functions relevant for determining their communicative performance. See Table 1. General intelligence was assessed with Raven’s Colored Progressive Matrices (RCPM, [67]), speed processing with the Trail Making test (Part A [68]), and selective attention using the Attentive Matrices [69]. We evaluated the following EFs: working memory with the Disyllabic Word Repetition test and Corsi’s Block-Tapping test [69], inhibitory control by administering the Modified Card Sorting test[70]; cognitive flexibility with the Trail Making Test (Part B–Part A [68]). The following tests were used to evaluate ToM performance: First-order/third person ToM with the Sally and Ann [71] and the Smarties tasks [72]. More advanced ToM abilities were evaluated with a selection of the Strange Stories tasks [28]. A description of each task is provided in Table 1, and mean performance raw scores in neuropsychological and ToM tasks are provided in Table 2. Data analysis. To analyze the data we used the Bayesian network approach. Bayesian networks (BN) [73] are probabilistic models that represent a set of variables and their conditional dependencies through a directed acyclic graph. Each node represents an attribute of the domain, and arcs between nodes represent conditional dependencies. For each node it is possible to calculate its conditional probability given the values of their parents. Each variable is assumed to be independent of its non-descendants given its set of parents. Under this assumption the joint probability distribution of all variables can be calculated. A BN can be used for classification when one of the variables is selected as the class attribute, and all other variables are input attributes. For an observation to be classified, the model calculates the posterior probability for each class marginalizing the joint probability distribution and labels the observation with the class that maximizes it. In this work, the attribute Type of subject was defined as the class attribute, and the input attributes were Sex, Age, Education, Linguistic, Extralinguistic, Paralinguistic, Context, Conversation, General Intelligence, Speed Processing, Selective Attention, Working Memory, Planning, Inhibition, Cognitive Flexibility, and Theory of Mind. The BN model was generated by Weka framework, version 3.8.3 [74]. The Weka learning algorithm used a hill climbing strategy, where several configurations of structures are examined, adding, deleting and reversing arcs between the nodes, such that maximize the likelihood of the class. To generate a simpler and more interpretative model, we set the maximum number of parents of a node as one. The initial count of the simple estimator in order to avoid zero frequencies was set as 0.1. The score type, which determines the measure used to assess the quality of the network structure, was the Minimum Description Length (MDL) [75]. The numeric attributes were discretized by Weka using Fayyad and Irani’s method [76]. We estimated the generalization performance of the BN model using cross-validation, which is a technique for evaluating predictive models by partitioning the original sample into a training set to create the model, and a test set to evaluate it. In our experiments, we used stratified 10-fold cross-validation, where the data is randomly split into ten sub-samples, each one with the same proportion of observations of each class. The process is then repeated ten times and each time a different sub-sample (fold) is used as a test set. Metrics presented are averaged across all folds, and are related to the test sets. We reported the following performance metrics: Accuracy, Sensitivity, Precision and Specificity. Accuracy is the proportion of the total number of classifications that were correct in both classes (SCZ and HC), Sensitivity gives the proportion of cases of schizophrenia classified correctly, Precision gives the proportion of the cases classified as schizophrenia that were correct, and Specificity gives the proportion of cases of controls classified correctly. Results Bayesian network model Fig 1 presents the BN model structure generated by Weka from the data. It is important to note that not all conditional dependency relationships that may exist between attributes appear in the generated model. This is due to two reasons: the restriction that each node can have at most one parent, and the learning algorithm that sought the best network structure to maximize the likelihood of the class attribute Type of subject. When setting a value to Type of subject (schizophrenia or control), the attributes Linguistic, Theory of Mind, Extralinguistic, Paralinguistic, Selective Attention, Planning, Inhibition, and Cognitive Flexibility become independent of one another. Besides, these attributes are sufficient to calculate the class probabilities for a new individual. Table 3 shows the conditional probability tables of all attributes in the BN model. Thus, we can investigate how the class probabilities are affected by using different attributes, and how much each attribute value is relevant to the classification of a particular instance. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Bayesian network model of pragmatic ability, Theory of Mind and cognitive functions. Each node in the graph represents a variable, and each arc asserts the dependence relationship between the pair of variables. The arc direction tells us that the value of variable X influences the value of variable Y. https://doi.org/10.1371/journal.pone.0229603.g001 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 3. Conditional probability tables of all attributes in the BN model. https://doi.org/10.1371/journal.pone.0229603.t003 The generated model shows that Linguistic influences Type of subject, i.e., given that Linguistic ≤ 0.88, the probability that Type of subject is schizophrenia is 76.3%. The other attributes are conditional dependent on Type of subject, i.e., given that Type of subject is schizophrenia, 62.4% of the patients have Theory of Mind ≤ -3.39, 68.6% have Extralinguistic ≤ 0.76, 59.3% have Paralinguistic > 0.70, 71.7% have Planning ≤ −1.04, 90.4% have Selective Attention ≤ −0.22, 62.4% have Inhibition ≤ −1.03, and 53.1% have Cognitive Flexibility ≤ −1.94. The most influential attribute in classifying patients with SCZ is Selective Attention (about 90% of the patients have Selective Attention ≤ −0.22). The attributes Planning, Extralinguistic, Theory of Mind, and Inhibition also revealed to be important in classifying patients with SCZ. Patients with SCZ show a large variability in Cognitive Flexibility values (53.1% of the patients have values below, and 46.9% have values above −1.94), probably due to the larger range of values of this test and its complexity (Trail Making Test part B), and thus the attribute is less influential in classifying patients with SCZ. In Paralinguistic patients show a small tendency (of 59.3%) to have values above 0.70, but the attribute is less discriminant compared to Linguistic, Extralinguistic and Context. Other relationships between cognitive and pragmatic variables can be observed in the generated model (see Fig 1). Paralinguistic strongly influences Context. If Paralinguistic ≤ 0.70, the probability of Context ≤ 0.75 is 99.2%, and when Paralinguistic > 0.70, the probability of Context > 0.75 is 94.3%. Selective Attention influences Speed Processing and Working Memory. If Selective Attention ≤ −0.22, there is a high tendency (94.5%) that Speed Processing > −0.19, and a tendency of 84.0% that Working Memory ≤ −0.44. Furthermore, Working Memory influences General Intelligence. If Working Memory ≤ −0.44 is likely that General Intelligence ≤ −0.62, although its values tend to be more scattered when Working Memory > −0.44. Lastly, the probability of Sex is influenced by Planning. If Planning > 1.04, it is very likely that Sex = female. Bayesian network model generalization performance In order to assess the ability of the BN model to generalize beyond the derivation cohort, stratified 10-fold cross-validation was performed. The overall accuracy of the model was 95.5% (SD 7.3%), showing that the model fits well in both classes since they are well balanced (32 cases of SCZ and 35 cases of HC). As the model classified all cases of schizophrenia correctly, sensitivity was 100%. From all cases classified as schizophrenia, there were three false positives (cases of control classified as schizophrenia), and therefore the precision of the model was 91.4% (SD 12.1%) and the specificity of the model was 91.4% (SD 13.6%) with 32/35 controls classified correctly. Role of clinical feature and medications on cognitive and pragmatic performance. The Bayesian network model described above did not include relevant clinical features (e.g. symptomatology) and medications taken by patients with SCZ since these variables were available only for patient and not for healthy controls. For this reason, we performed ad hoc analysis in order to investigate the role of psychiatric symptoms and medications on pragmatic and cognitive performance. We split our sample in two subgroups on the basis of severity of symptomatology (low global symptomatology: total PANSS score ≤ 81, and high global symptomatology: total PANSS score > 81). We then built the same Bayesian network model used in main analysis (see Bayesian network model section) trying to predict the belonging of the patients to one of the subgroups (low vs high symptomatology) by using cognitive and pragmatics performance. The generated Bayesian model failed to classify the severity of symptomatology using cognitive and pragmatic variables. Furthermore, statistical analysis confirmed that there is no correlation (considering the significant level alpha = .05) between these variables according to the Spearman Correlation Test (-0.34 < r < 0.1, .052 < p < .85). We also evaluated the role of medications by testing whether cognitive and pragmatic performance was different in patients taking typical and atypical antipsychotic medications. A series of independent samples T-test (considering the significant level alpha = .05) showed no significant differences between the two subgroups of patients (i.e., those taking typical and atypical antipsychotic medications) in cognitive or pragmatic performance (0.43 < t < 1.52, .14 < p < .97). Bayesian network model Fig 1 presents the BN model structure generated by Weka from the data. It is important to note that not all conditional dependency relationships that may exist between attributes appear in the generated model. This is due to two reasons: the restriction that each node can have at most one parent, and the learning algorithm that sought the best network structure to maximize the likelihood of the class attribute Type of subject. When setting a value to Type of subject (schizophrenia or control), the attributes Linguistic, Theory of Mind, Extralinguistic, Paralinguistic, Selective Attention, Planning, Inhibition, and Cognitive Flexibility become independent of one another. Besides, these attributes are sufficient to calculate the class probabilities for a new individual. Table 3 shows the conditional probability tables of all attributes in the BN model. Thus, we can investigate how the class probabilities are affected by using different attributes, and how much each attribute value is relevant to the classification of a particular instance. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Bayesian network model of pragmatic ability, Theory of Mind and cognitive functions. Each node in the graph represents a variable, and each arc asserts the dependence relationship between the pair of variables. The arc direction tells us that the value of variable X influences the value of variable Y. https://doi.org/10.1371/journal.pone.0229603.g001 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 3. Conditional probability tables of all attributes in the BN model. https://doi.org/10.1371/journal.pone.0229603.t003 The generated model shows that Linguistic influences Type of subject, i.e., given that Linguistic ≤ 0.88, the probability that Type of subject is schizophrenia is 76.3%. The other attributes are conditional dependent on Type of subject, i.e., given that Type of subject is schizophrenia, 62.4% of the patients have Theory of Mind ≤ -3.39, 68.6% have Extralinguistic ≤ 0.76, 59.3% have Paralinguistic > 0.70, 71.7% have Planning ≤ −1.04, 90.4% have Selective Attention ≤ −0.22, 62.4% have Inhibition ≤ −1.03, and 53.1% have Cognitive Flexibility ≤ −1.94. The most influential attribute in classifying patients with SCZ is Selective Attention (about 90% of the patients have Selective Attention ≤ −0.22). The attributes Planning, Extralinguistic, Theory of Mind, and Inhibition also revealed to be important in classifying patients with SCZ. Patients with SCZ show a large variability in Cognitive Flexibility values (53.1% of the patients have values below, and 46.9% have values above −1.94), probably due to the larger range of values of this test and its complexity (Trail Making Test part B), and thus the attribute is less influential in classifying patients with SCZ. In Paralinguistic patients show a small tendency (of 59.3%) to have values above 0.70, but the attribute is less discriminant compared to Linguistic, Extralinguistic and Context. Other relationships between cognitive and pragmatic variables can be observed in the generated model (see Fig 1). Paralinguistic strongly influences Context. If Paralinguistic ≤ 0.70, the probability of Context ≤ 0.75 is 99.2%, and when Paralinguistic > 0.70, the probability of Context > 0.75 is 94.3%. Selective Attention influences Speed Processing and Working Memory. If Selective Attention ≤ −0.22, there is a high tendency (94.5%) that Speed Processing > −0.19, and a tendency of 84.0% that Working Memory ≤ −0.44. Furthermore, Working Memory influences General Intelligence. If Working Memory ≤ −0.44 is likely that General Intelligence ≤ −0.62, although its values tend to be more scattered when Working Memory > −0.44. Lastly, the probability of Sex is influenced by Planning. If Planning > 1.04, it is very likely that Sex = female. Bayesian network model generalization performance In order to assess the ability of the BN model to generalize beyond the derivation cohort, stratified 10-fold cross-validation was performed. The overall accuracy of the model was 95.5% (SD 7.3%), showing that the model fits well in both classes since they are well balanced (32 cases of SCZ and 35 cases of HC). As the model classified all cases of schizophrenia correctly, sensitivity was 100%. From all cases classified as schizophrenia, there were three false positives (cases of control classified as schizophrenia), and therefore the precision of the model was 91.4% (SD 12.1%) and the specificity of the model was 91.4% (SD 13.6%) with 32/35 controls classified correctly. Role of clinical feature and medications on cognitive and pragmatic performance. The Bayesian network model described above did not include relevant clinical features (e.g. symptomatology) and medications taken by patients with SCZ since these variables were available only for patient and not for healthy controls. For this reason, we performed ad hoc analysis in order to investigate the role of psychiatric symptoms and medications on pragmatic and cognitive performance. We split our sample in two subgroups on the basis of severity of symptomatology (low global symptomatology: total PANSS score ≤ 81, and high global symptomatology: total PANSS score > 81). We then built the same Bayesian network model used in main analysis (see Bayesian network model section) trying to predict the belonging of the patients to one of the subgroups (low vs high symptomatology) by using cognitive and pragmatics performance. The generated Bayesian model failed to classify the severity of symptomatology using cognitive and pragmatic variables. Furthermore, statistical analysis confirmed that there is no correlation (considering the significant level alpha = .05) between these variables according to the Spearman Correlation Test (-0.34 < r < 0.1, .052 < p < .85). We also evaluated the role of medications by testing whether cognitive and pragmatic performance was different in patients taking typical and atypical antipsychotic medications. A series of independent samples T-test (considering the significant level alpha = .05) showed no significant differences between the two subgroups of patients (i.e., those taking typical and atypical antipsychotic medications) in cognitive or pragmatic performance (0.43 < t < 1.52, .14 < p < .97). Role of clinical feature and medications on cognitive and pragmatic performance. The Bayesian network model described above did not include relevant clinical features (e.g. symptomatology) and medications taken by patients with SCZ since these variables were available only for patient and not for healthy controls. For this reason, we performed ad hoc analysis in order to investigate the role of psychiatric symptoms and medications on pragmatic and cognitive performance. We split our sample in two subgroups on the basis of severity of symptomatology (low global symptomatology: total PANSS score ≤ 81, and high global symptomatology: total PANSS score > 81). We then built the same Bayesian network model used in main analysis (see Bayesian network model section) trying to predict the belonging of the patients to one of the subgroups (low vs high symptomatology) by using cognitive and pragmatics performance. The generated Bayesian model failed to classify the severity of symptomatology using cognitive and pragmatic variables. Furthermore, statistical analysis confirmed that there is no correlation (considering the significant level alpha = .05) between these variables according to the Spearman Correlation Test (-0.34 < r < 0.1, .052 < p < .85). We also evaluated the role of medications by testing whether cognitive and pragmatic performance was different in patients taking typical and atypical antipsychotic medications. A series of independent samples T-test (considering the significant level alpha = .05) showed no significant differences between the two subgroups of patients (i.e., those taking typical and atypical antipsychotic medications) in cognitive or pragmatic performance (0.43 < t < 1.52, .14 < p < .97). Discussion A few previous studies have provided a comprehensive assessment of communicative-pragmatic ability in SCZ in order to evaluate the relationship between cognitive functions, ToM [8,20,37,39], and pragmatic behavior expressed through different expressive modalities [7,16,77]. The results of these studies are still inconclusive, potentially due to the large natural heterogeneity in SCZ, the complexity of the multivariate relationship among pragmatic ability, cognition and ToM, and the limitations of classical statistical methods in accounting for these conditions. ML methods offer the opportunity to overcome such limits and set the stage for an improvement in modeling the relationship between cognition, pragmatic ability and ToM. The present research is the first attempt to apply a Bayesian network analysis in order to identify the complex relationship among the above-mentioned variables in patients with SCZ. First of all, to evaluate the reliability of the model in classifying patients vs. controls, we measured sensitivity, accuracy and precision. The sensitivity of the model showed that all patients with schizophrenia were classified correctly. The overall accuracy was very high, showing that the model was able to correctly distinguish between patients with SCZ and HC, and precision was also good, showing a very small number of false positives, i.e. HC classified as SCZ. Given the generated model, the set of variables used to classify a new case consists of the parents, the children and the parents of the children of the class attribute (called Markov Blanket) [74]. Therefore, the following variables are sufficient to discriminate between patients and controls: for pragmatic ability, Linguistic, Extralinguistic and Paralinguistic, for cognitive abilities, Planning, Inhibition and Cognitive Flexibility and, lastly, ToM. However, the structure of the network revealed that only pragmatic Linguistic ability has a direct influence in classifying patients and controls. Indeed, the model showed that a score below the normative value [63] on the Linguistic scale of the ABaCo, indicates a high probability of classification as a patient. We noticed that, of the scales we used during the pragmatic assessment, the Linguistic one best represents pragmatic-communicative ability from an “ecological” perspective, and this might explain why it, and not the other scales, has a direct influence on the classification of patients and controls. This result highlights the importance of assessing linguistic pragmatic ability in SCZ, and is in line with [38], suggesting that an impairment in pragmatic ability could be considered a core dysfunction of the illness. In line with this theoretical perspective, a recent study proposed that communicative-pragmatic deficit could be considered a vulnerability marker of the schizophrenic pathology, and that its assessment, during the early stage of the illness, could improve the diagnostic process [78]. Furthermore, the results showed that the type of participants, i.e. SCZ vs. HC, influences performance on the following tasks: ToM, Extralinguistic, Paralinguistic, Selective Attention, Planning, Inhibition and Cognitive Flexibility. More in detail, belonging to the schizophrenia group was found to be associated with poor performance on ToM, Selective Attention, Extralinguistic, Planning, and Inhibition tasks. A less clear association was found for Paralinguistic and Cognitive Flexibility, where patients obtained a wider range of performance values. These results are in line with a consolidated area of research, starting from Frith’s pioneering proposal [3], showing that patients with schizophrenia suffer from an impairment in mindreading ability [79]. Difficulties in planning, inhibition and cognitive flexibility are also well documented in SCZ [80–82], and our results confirm the importance of including all these functions in the assessment and rehabilitation programs aimed at overcoming patients’ difficulties [83]. Focusing on the relationship among the variables, the model showed that, given the type of subject–SCZ vs. HC—ToM, pragmatic, and cognitive abilities represent three distinct domains independent of one another. This is in line with previous studies [8,38] that, analyzing the pattern of co-occurrence of ToM, cognitive and pragmatic impairments in schizophrenia, pointed out that there is no stable relationship between these functions. This finding supports the idea that pragmatic and ToM deficits cannot simply be reduced to cognitive or ToM deficits [27]. Finally, Education, Age and Conversation are not linked to any other variable in the generated model, which means they are totally independent. The psychometric properties of the Conversational scale of the ABaCo are lower than for the other scales [62], therefore we can assume that this scale did not have a role in discriminating between SCZ vs. HC. As for Education and Age, this result is not surprising, since we kept these variables under control in matching the control and patient groups. However, it is important to highlight that not all the relationships that could exist among the investigated variables necessarily appear in the generated model. The learning algorithm used a strategy to select from among the possible models (BN structures) based on specialized scores for classification, i.e., to distinguish between patients and controls. Besides, we limited each variable to have at most one parent, that is, only one direct relationship between two variables was allowed (e.g. A → B, but not A → B ← C), to enable the model to choose the most relevant relationship between two variables from among all the possible ones. By using these parameter settings, we were able to find the simplest effective network structure in discriminating between SZ and HC, and we improved model interpretability. When assessing the relationships among cognitive and pragmatic variables, the model pointed out that performance on the Paralinguistic scale influences performance on the Context one. This result is not surprising, since the Paralinguistic scale assesses the ability to handle cues, such as tone of the voice and prosody that usually accompany a conversation, and that could indicate the real communicative intention of the speaker. The Context scale assesses the ability to comprehend the appropriateness of a speech act in a given situation, and in the pragmatic tasks administered, i.e. short video-clips, the paralinguistic elements are the most important elements enabling participants to detect the social inappropriateness of a proffered utterance (e.g., in an office the boss asks to the secretary to type a letter, and she answers—in a very impolite tone of voice- that she is very busy at moment). Furthermore, the model showed that Selective Attention influences Working Memory and Speed Processing, and Working Memory influences General Intelligence. In line with recent literature, this result confirms the direct influence that Selective Attention exerts on Working Memory, and the strong associations between these functions [84]. Speed processing and selective attention were evaluated with similar tasks in our sample, both of which require participants to identify the correct stimuli from among several distractors, and the association between these tasks might thus be explained by the task structure. Finally, working memory capacity and intelligence have also been demonstrated to be strongly correlated, and our result provided additional evidence supporting this association [85]. The study is not exempt from limitations: the most important of these is the relatively small sample size, considering that BNs benefit from large sample sizes in order to perform an in-depth investigation of complex dependencies in uncertain data. Future research, with larger samples, will allow the generalizability of the present findings to be assessed. To conclude, despite its limits, the principal novelty of the present investigation consists in the use of a ML approach to show that, among the cognitive, pragmatic and ToM variables investigated, pragmatic Linguistic ability appears to be the most important factor in classifying patients with SCZ and HC. Communicative-pragmatic difficulty has been associated with a reduction in quality of life, early age of onset, and poor response to treatment [38,86,87], and can thus contribute to the poor social functioning exhibited by patients with schizophrenia. The identification of communicative difficulties as an important factor in predicting SCZ may contribute to improving and refining existing rehabilitative training [88,89,90], psychoeducational programs and clinical interventions aimed at reducing relational and social impairments typical of the illness. However, our results also confirm the role played by other factors, i.e. EF and ToM, in explaining the SCZ profile, and highlight the importance of considering all of these elements at the same time and, at least, with the same level of attention, in order to manage the condition more effectively. As a final point, we have demonstrated the potential for using BNs to identify the causal influence of pragmatic and cognitive variables in SCZ. This approach enabled us to intuitively represent the network of connections between cognition and pragmatic ability in SCZ, which helped us to identify the most important and clinically relevant variables. The advantages of using ML methods, may lead to important opportunities for advancements in this research field.
TI - Pragmatics, Theory of Mind and executive functions in schizophrenia: Disentangling the puzzle using machine learning
JF - PLoS ONE
DO - 10.1371/journal.pone.0229603
DA - 2020-03-03
UR - https://www.deepdyve.com/lp/public-library-of-science-plos-journal/pragmatics-theory-of-mind-and-executive-functions-in-schizophrenia-0bmHvl6zqw
SP - e0229603
VL - 15
IS - 3
DP - DeepDyve
ER -