ARTICLE DOI: 10.1038/s41467-018-04614-w OPEN Mapping higher-order relations between brain structure and function with embedded vector representations of connectomes 1,2 3 4,5 4 6,7 Gideon Rosenthal , František Váša , Alessandra Griffa , Patric Hagmann , Enrico Amico , 6,7,8 1,2,9 10 Joaquín Goñi , Galia Avidan & Olaf Sporns Connectomics generates comprehensive maps of brain networks, represented as nodes and their pairwise connections. The functional roles of nodes are deﬁned by their direct and indirect connectivity with the rest of the network. However, the network context is not directly accessible at the level of individual nodes. Similar problems in language processing have been addressed with algorithms such as word2vec that create embeddings of words and their relations in a meaningful low-dimensional vector space. Here we apply this approach to create embedded vector representations of brain networks or connectome embeddings (CE). CE can characterize correspondence relations among brain regions, and can be used to infer links that are lacking from the original structural diffusion imaging, e.g., inter-hemispheric homotopic connections. Moreover, we construct predictive deep models of functional and structural connectivity, and simulate network-wide lesion effects using the face processing system as our application domain. We suggest that CE offers a novel approach to revealing relations between connectome structure and function. 1 2 Department of Cognitive and Brain Sciences, Ben-Gurion University of the Negev, P.O.B. 653, 8410501 Beer-Sheva, Israel. The Zlotowski Center for Neuroscience, Ben-Gurion University of the Negev, P.O.B. 653, 8410501 Beer-Sheva, Israel. Brain Mapping Unit, Department of Psychiatry, University of Cambridge, Cambridge CB2 0SZ, UK. Department of Radiology, Centre Hospitalier Universitaire Vaudois (CHUV) and University of Lausanne (UNIL), 1011 5 6 Lausanne, Switzerland. Signal Processing Laboratory 5 (LTS5), École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland. School of Industrial Engineering, Purdue University, West-Lafayette 47907 IN, USA. Purdue Institute for Integrative Neuroscience, Purdue University, West-Lafayette 8 9 47907 IN, USA. Weldon School of Biomedical Engineering, Purdue University, West-Lafayette 47907 IN, USA. Department of Psychology, Ben-Gurion University of the Negev, P.O.B. 653, 8410501 Beer-Sheva, Israel. Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN 47405, USA. Correspondence and requests for materials should be addressed to O.S. (email: firstname.lastname@example.org) NATURE COMMUNICATIONS (2018) 9:2178 DOI: 10.1038/s41467-018-04614-w www.nature.com/naturecommunications 1 | | | 1234567890():,; ARTICLE NATURE COMMUNICATIONS | DOI: 10.1038/s41467-018-04614-w n organism’s nervous system is composed of specialized meaningful and can be manipulated using linear operations. brain regions, each associated with distinctive processing Then, we demonstrate that CE representations can predict Acapacities and responses. However, these regions do not functional connectivity from structural connectivity with high work in isolation, and in fact, a region’s functional role is tightly accuracy for both direct and indirect connections. Lastly, we use linked to its anatomical connectivity and physiological interac- CEs to predict network-level functional effects of localized lesions tions with other regions in the system. The totality of these in structural networks. connections and interactions can be summarized, and system- atically analyzed by using concepts of network science. Networks comprise a set of elements and their dyadic (pairwise) connec- Results tions which allow characterizing each element’s connection pat- Inter-hemispheric analogies test. To test whether CE vector tern. The connectome offers such a network description, representations are consonant with known attributes of brain summarizing an organism’s complete nervous system as a graph topology/topography and can be interpreted and manipulated which represents the complete set of connections between pairs of using linear operations, we formulated a brain speciﬁc bench- 1 4, neurons or brain regions . ). mark, namely, an inter-hemispheric analogies test. Despite recent advances in connectome mapping , the result- One of the basic organizational characteristics of the human ing collections of dyadic relations do not, by themselves, fully brain is functional homotopy, i.e., symmetric inter-hemispheric 11–14 represent and quantify higher-order relations among nodes correlations between bilaterally homologous brain regions . within the network. At the level of the network map, each node is Functional homotopy is supported by a high proportion of 14, 15 deﬁned by a vector corresponding to its connections with all callosal ﬁbers contributing to homotopic connectivity . other nodes, arranged in a high-dimensional topological space. Moreover, the structural and functional connectivity patterns of Such a dyadic description does not readily allow visualization, the two hemispheres exhibit high levels of cross-hemisphere classiﬁcation, prediction of missing edges and nodes, and similarity. Consequently, to design a benchmark for testing and 3, 4 understanding relations between different networks . While tuning connectome embeddings, we postulated that the relation there are many descriptive graph measures that can capture local between each pair of regions in one hemisphere should be and global network features , most of these measures are not analogous to the same pairwise relation in the other hemisphere. designed to capture the shape of the topological space within We tested all possible inter-hemispheric analogies between all which individual nodes of the network are embedded. Assessing nodes for both node2vec and spectral embedded vectors. For each 7–9 distances among connectivity proﬁles and subsequent dimension analogy, the cosine similarity which ranges between −1 and 1 reduction (e.g., through PCA or multidimensional scaling) can and captures the cosine of the angle between two vectors, was reveal pairwise similarities but this approach does not capture computed between a linearly combined vector [vector(“Right other relations such as homologies or higher-order regularities. Node A”)—vector (“Right Node B”)+vector (“Left Node B”)] and Outside of connectomics, another ﬁeld focused on mapping all of the nodes vector embeddings. This procedure produced a relationships between elements is natural language processing, vector of cosine similarity distances which was then ranked in an where words may be represented or embedded in a low- ascending order. The rank of the expected vector (“Left Node A”) 7, 8 dimensional distributed vector space . This representation can was logged for each analogy. We term this procedure the inter- facilitate higher-level natural language processing tasks by hemispheric analogies test. grouping similar words into a similar embedded representation. We benchmarked our results against a more standard spectral One recent set of models for learning vector representations of embedding algorithm which is an unsupervised method aimed at words is word2vec which encodes linguistic regularities and calculating low dimensional non-linear embeddings of the data 16, 17 patterns. These regularities may be manipulated using linear using a decomposition of the graph Laplacian . One of the operations. For example, the result of a vector calculation vec strong underlying assumptions of spectral embedding is that (“King”)—vec(“Man”) + vec(“Woman”) is closest to vec interconnected nodes should be embedded together in the vector 7–9 (“Queen”) than to any other word vector . Importantly, space (homophily) and that these embeddings are useful for word2vec algorithms have been recently generalized for repre- classiﬁcation. This might not be the case for some networks and senting networks instead of text. The analog for sentences in the tasks, such as the current analogies task, which requires a network domain are streams of randomly generated walks in the representation of the structural role of each node (structural 3, 4, 10 network (for example Deepwalk, Node2vec, and Grarep; ). equivalence) or a mixture between homophily and structural 3, 18 The resultant latent nodes representation captures neighborhood equivalence . similarity and community membership in a continuous vector For each hemispheric analogy (vectors’ linear combinations; space with a relatively small number of dimensions . These low- see methods for details), the median rank was calculated dimensional embeddings are useful for subsequent machine across 500 node2vec iterations. For example, vec(“Left Amyg- learning applications directed at uncovering structural relations dala”)—vec(“Left Fusiform Gyrus”) + vec(“Right Fusiform and similarities. Gyrus”) should yield a vector that has the smallest distance to Here we build upon these advances with an embedded repre- vec(“Right Amygdala”) compared with all other node vector sentation of the human connectome (connectome embedding; embeddings (see Fig. 1e). If the calculated vector is indeed CE). The aim is to capture the structural network-level relations closest to vec(“Right Amygdala”), then the rank difference of the between brain regions in a low-dimensional continuous vector analogy would be 0; Hence we termed this node (in this space to allow inferences about their functional roles and rela- example, the Right Amygdala) as the expected node. Note that tionships. We suggest that CE provides a general approach for higher ranking means that the expected node embedding modeling connectome data that has many potential applications, was less similar to the calculated vector and consequently including development, individual differences and clinical/trans- higher ranking reﬂects worse performance. Thus, if across all lational studies. possible analogies, a high proportion of the expected nodes To test the utility of CE, ﬁrst, a network embedding algorithm would have a low rank (a small distance from the is used to embed a diffusion structural MRI connectome into a linearly combined vector), then one could infer that the obtained continuous vector representation, or CE (Fig. 1a-e). Next, we vector representations encompass meaningful topological demonstrate that CE representations are neurobiologically information. 2 NATURE COMMUNICATIONS (2018) 9:2178 DOI: 10.1038/s41467-018-04614-w www.nature.com/naturecommunications | | | NATURE COMMUNICATIONS | DOI: 10.1038/s41467-018-04614-w ARTICLE R L g′ f d′ d f′ b b′ a′ e e′ c′ Random walk on structural b Region a Region a′ Region b′ connectivity Region b Region c Region d Region e′ Region f′ Region g′ Region e Region f Region g Region c′ Region d′ Region e′ Input R(t –1) Output Projection R(t ) R(t +1) Word2vec Vector Vector Vector Vector Vector Region label Element 1 Element 2 Element 3 Element 4 Element 5 d 0.34 0.73 0.35 0.12 0.99 .. e 0.54 0.21 0.45 0.32 0.43 .. d′ 0.52 0.22 0.45 0.96 0.68 .. e′ 0.27 0.34 0.72 0.19 0.34 .. e′ Node vector space d v (e) – v (d) = v (e′) – v (d′) d′ Fig. 1 Connectome embeddings workﬂow. a The input to the connectome embeddings algorithm is a structural connectome, describing pairwise connectivity between brain regions (nodes). Letters denote unique, corresponding nodes in the right and left hemispheres of the brain (for example a and a’ represent homotopic regions). Dashed lines represent possible direct edges between regions according to the structural connectivity matrix, while full lines represent random walks . b Random walks are performed on the network using the node2vec algorithm producing node sequences (note that we present sequences of 3 steps for demonstrational purposes, the sequence may be longer). c The sequence for each node is used as an input to a word2vec Continuous Bag of Words (CBOW) algorithm. Brieﬂy, for each sequence, each node in turn is considered a target, R(t), which is predicted from the other nodes in the same sequence [R(t − 1), R(t + 1)..]. The goal is to maximize the conditional probability p(R(t)| R(t − 1), R(t + 1)..; θ) by estimating the parameters θ using a 2 layers neural network (Goldberg & Levy, 2014). d The obtained parameters θ, or vectors capture regularities and may be the basis for various subsequent tasks [7, 8] which forms a vector distributed representation of each node. e The direction of the produced vector representation of a node has a topological meaning. For example, the differences between homological nodes in opposing hemispheres are analogs (see results for details) Dataset 1: Across all hemispheric analogies, 54% of the percentage of the top 5 ranked nodes signiﬁcantly differed across expected nodes, 444 out of 820 (number of possible inter these two embedding methods, χ (1, N = 1640) = 68.6, p < 2.2e- hemispheric analogies with 82 homologous nodes), were ranked 16 (Supplementary Fig. 1). as one of the top ﬁve nodes when using connectome embedding. In contrast, only 18.6%, 153 out of 820, of the expected nodes were ranked in the top ﬁve nodes using a conventional spectral Similarity of node representation. As implied by the inter- embedding algorithm. The percentage of the top 5 ranked nodes hemispheric analogies test, the relation between the learned CE signiﬁcantly differed across these two embedding methods, χ (1, vectors encompasses meaningful neurobiological information. To N = 1640) = 223, p < 2.2e-16 (Fig. 2). Dataset 2: As was the case further explore this issue and understand the nature of the pair- for Dataset 1, the difference between the connectome embedding wise relation between each pair of nodes in relation to functional and the spectral clustering analogies was signiﬁcant. Across all homotopy, we characterized the similarity between the repre- hemispheric analogies, 30% of the expected nodes, 246 out of 820 sentations of their respective CE vectors. Speciﬁcally, the cosine (number of possible inter hemispheric analogies with 82 similarity was calculated between each pair of connectome homologous nodes), were ranked as one of the top ﬁve nodes embedding vectors (Fig. 3). This procedure resulted in a recon- when using connectome embedding. In contrast, only 13%, 108 struction of the structural connectivity matrix (embedding out of 820, of the expected nodes were ranked in the top ﬁve reconstruction). We did not expect a perfect reconstruction of the nodes using a conventional spectral embedding algorithm. The original structural matrix. Rather, we assumed that if CE manages NATURE COMMUNICATIONS (2018) 9:2178 DOI: 10.1038/s41467-018-04614-w www.nature.com/naturecommunications 3 | | | ARTICLE NATURE COMMUNICATIONS | DOI: 10.1038/s41467-018-04614-w Embedding method Node2vec Spectral embedding 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 Cosine similarity ranking Fig. 2 Performance of two node embedding algorithms on the inter-hemispheric analogies test. The inter-hemispheric analogies test evaluated the capacity of two node embeddings to infer the relation between each pair of nodes in one hemisphere, given the same pairwise relation in the other hemisphere. Predictions across all pairwise analogies were ranked, such that a lower rank corresponds to better performance. Across 500 iterations of the node2vec algorithm, the ranking of the expected nodes were binned into bins of 5. The boxplots represent the binning of the ranking across 500 node2vec permutations. The band inside the box represents the median, the lower and upper hinges correspond to the ﬁrst and third quartiles (the 25th and 75th percentiles) and the whiskers represent 1.5 times the inter-quartile range (the distance between the ﬁrst and third quartiles). The red triangles represent the binning of the spectral ranking. Importantly, the node2vec algorithm produced a higher proportion of expected nodes in the lowest rank bin (0–5 ranking). Note the relatively high proportion of spectral embedding analogies with high ranking which suggest worst performance in this task. This result demonstrates that node2vec vector embeddings successfully encompass functional homotopy information to capture high-level topological attributes, it should be reﬂected applied to the Z-score. Dataset 1: The percentage of homotopic in the CE pair-wise relation. inter-hemispheric edges was 73% for the node2vec reconstructed We ﬁrst calculated the Spearman’s Rho (rank correlation matrix compared to 48% in the original connectome coefﬁcient) between each embedding reconstruction (node2vec matrix at 0 threshold. This difference was statistically signiﬁcant and spectral) and the original matrix of structural connectivity (χ (1, N = 82) = 9.76, p = 0.001). Similar patterns emerged edges that were obtained using diffusion imaging. when various thresholds up to 0.9 were applied, but disappeared Dataset 1: The connectome structural matrix was weakly at a threshold of 1, with only 44% and 34% of homotopic inter- correlated with the spectral embeddings reconstruction r = 0.2, p hemispheric edges apparent for the node2vec matrix and −6 < 10 , and strongly correlated with the node2vec embeddings original structural connectivity matrix, respectively. The differ- −6 reconstruction r = 0.62, p< 10 (Fig. 3). Dataset 2: Similarly to ence in homotopic inter-hemispheric edges between the the correlations measured in Dataset 1, the connectome structural spectral embedding reconstruction (53%) and the structural matrix was not correlated with the spectral embeddings connectivity matrix (48%) was not signiﬁcant for the 0 threshold reconstruction r = −0.02, p = 0.11 but was strongly correlated (χ (1, N = 82) = 0.39, p = .53). Similar results were apparent −6 with the node2vec embeddings reconstruction r = 0.63, p< 10 across thresholds up to 0.9. Dataset 2: Equivalently to the (Supplementary Fig. 2). results obtained from Dataset 1, the percentage of homotopic We hypothesized that the difference between the node2vec and inter-hemispheric edges was 56% for the node2vec reconstructed the spectral embedding reconstructions may, in part, be due to matrix compared to 21% in the original connectome matrix at 0 the tendency of the embedding algorithm to infer missing threshold. This difference was statistically signiﬁcant (χ (1, N = −6 connections on the basis of existing higher-order relationships. 82) = 27.9, p< 10 ). Similar patterns emerged when various For example, homotopic inter-hemispheric structural connections thresholds up to 0.9 were applied, but disappeared at a are not well captured using diffusion imaging . Recently, an threshold of 1, with only 19% and 12% of homotopic inter- analysis which used 32,350 connection reports, expertly collated hemispheric edges apparent for the node2vec matrix and original from published pathway tracing experiments in rats, suggested structural connectivity matrix, respectively. Note that at 0.6, 0.8 that about two-thirds of all cortical regions send a homotopic and 0.9 thresholds only non-signiﬁcant trends were apparent (χ commissural connection . Thus, homotopic inter-hemispheric (1, N = 82) = 3.12, 3.64 and 3.64, p = 0.07, 0.056 and 0.056 connections may be better recovered in the node2vec connectivity respectively). reconstruction, due to topological features which are captured by At lower Z-thresholds (0.0–0.1), there were still signiﬁcant the embedding. differences in homotopic inter-hemispheric edges between the To compare the number of homotopic inter-hemispheric spectral embedding reconstruction and the structural connectivity connections between the original structural connectome and the matrix (χ (1, N = 82) = 6.97 and 6.2, p = 0.008 and 0.01). Once reconstructed embedding connectivity matrices, we applied Z- increasing the threshold above 0.1 there were no signiﬁcant score normalization to each matrix, followed by a threshold differences at any of the thresholds. 4 NATURE COMMUNICATIONS (2018) 9:2178 DOI: 10.1038/s41467-018-04614-w www.nature.com/naturecommunications | | | Proportion of analogies NATURE COMMUNICATIONS | DOI: 10.1038/s41467-018-04614-w ARTICLE ab Original structural connectivity Original functional connectivity Lateralorbitofrontal Parsorbitalis Frontalpole Medialorbitofrontal RH Parstriangularis Parsopercularis Rostralmiddlefrontal Superiorfrontal Caudalmiddlefrontal Precentral Paracentral Rostralanteriorcingulate Caudalanteriorcingulate Posteriorcingulate Isthmuscingulate LH Postcentral Supramarginal Superiorparietal Inferiorparietal Precuneus Cuneus Pericalcarine Lateraloccipital 0.0 0.16 –0.8 0 0.8 Lingual Mean fiber density Mean correlation coefficient Fusiform Parahippocampal cd Spectral embeddings reconstruction Node2vec reconstruction Entorhinal Temporalpole Inferiortemporal Middletemporal Bankssts Superiortemporal Transversetemporal RH Insula Thalamusproper Caudate Putamen Pallidum Accumbensarea Hippocampus Amygdala Brainstem LH –0.8 0 0.8 –0.8 0 0.8 Cosine similarity Cosine similarity Fig. 3 Cosine similarity of the structural connectivity matrix and node embeddings. a The original structural connectivity matrix with 83 predeﬁned regions of interest (see Cammoun et al., 2012). Each cell represents a structural connection between a pair of regions. The same regions are used in all matrices. b The original mean functional connectivity. c Cosine similarity between spectral embeddings. See text for a statistical analysis. d Cosine similarity between node2vec embeddings 24, 27 Relation to resting state functional connectivity.As ﬁndings by direct pairwise structural connectivity . For example, in reported so far suggest that CE provides a meaningful repre- addition to resting-state functional connections between nodes sentation of the structural connectome, we move to examining its that are directly anatomically connected (direct connections), relation to functional networks estimated from resting state numerous functional connections also exist between nodes that connectivity. Speciﬁcally, statistical dependence among regional are not directly anatomically connected (indirect connections), time courses is generally called functional connectivity , and due to indirect interactions throughout the network and the numerous previous studies have shown that functional con- transitivity of cross-correlations . As demonstrated, CE recon- nectivity recorded during long sessions of resting state are structed matrices contain high-level topological connectivity 22–26 robustly related to the underlying structural connectivity . information. We hypothesized that such information may be While such resting state connectivity is dependent on a associated with resting state functional connectivity to a greater structural backbone, it also expresses higher level interactions extent than the original structural connectivity matrix, as it may between nodes of the network which are not necessarily captured capture a signiﬁcant proportion of indirect effects. NATURE COMMUNICATIONS (2018) 9:2178 DOI: 10.1038/s41467-018-04614-w www.nature.com/naturecommunications 5 | | | ARTICLE NATURE COMMUNICATIONS | DOI: 10.1038/s41467-018-04614-w Dataset 1: Indeed, a higher correlation coefﬁcient was obtained positive correlation between the node2vec reconstruction edges between the node2vec reconstructed embedding matrices and the and the functional connectivity matrix r = 0.27, p = 0.003 but −6 functional connectivity edges (r = 0.328, p< 10 ; Fig. 4c), there was no signiﬁcant correlation between the spectral compared to the correlation between the functional connections embedding reconstruction and the functional connectivity and spectral embedding reconstructions edges (r = 0.13, p< 10 connections r = 0.17, p = 0.069. s s −6 ; Fig. 4b), as well as between the functional connections and the These ﬁndings suggest that node2vec embeddings capture −6 original structural edges (r = 0.311, p< 10 ; Fig. 4a). Impor- signiﬁcant information about functional relations as measured in tantly, when considering node pairs that are not directly resting-state functional connectivity. connected in the original structural matrix, we obtained a positive correlation between the node2vec reconstruction and the −6 functional connectivity matrix (r = 0.127, p< 10 ) but no Deep learning for structural to functional mapping.To signiﬁcant correlation between the spectral embedding recon- examine whether the mapping between the reconstructed CE and struction and the functional connectivity connections (r = functional connectivity could be further improved, we adopted a −0.02, p = 0.52). supervised deep learning framework. To this end, a representa- Dataset 2: Similarly to the results obtained with Dataset 1, a tion of edges was required as opposed to the single node higher correlation coefﬁcient was measured between the node2- embeddings . The mapping between structural embeddings and vec reconstructed embedding matrices and the functional functional connectivity was learned utilizing a node-pairs repre- −6 connectivity (r = 0.31, p< 10 ; Supplementary Fig. 3c), com- s sentation, while adopting a supervised learning cross-validation pared to the correlation between the functional connections and scheme (see Methods for details). −6 spectral embedding reconstructions (r = 0.15, p< 10 ; Supple- s Dataset 1: When assessing the correspondence between mentary Fig. 3b), as well as between the functional connections predicted functional connections and the empirical functional −6 and the original structural edges (r = 0.21, p< 10 ; Supple- connections in the testing set, we obtained a strong positive −6 mentary Fig. 3a). When examining the nodes that are indirectly correlation (r = 0.6, p< 10 ) (Fig. 5) which was apparent for −6 connected in the original structural matrix, we measured a both direct connections (r = 0.6, p< 10 ) and indirect a b 2.0 2.0 1.5 1.5 1.0 1.0 0.5 0.5 0.0 0.0 –0.5 –0.5 –1.0 –1.0 –0.05 0.00 0.05 0.10 0.15 0.20 –0.5 0.0 0.5 1.0 Original DSI Edge Spectral embeddings edge reconstructions 2.0 1.5 1.0 0.5 0.0 –0.5 –1.0 –1.0 –0.5 0.0 0.5 1.0 1.5 Node2vec edge reconstructions Fig. 4 Correspondence between resting state functional connectivity and structural connectivity. Correlation between resting state functional connectivity −6 (after Fisher’s Z-transformation) and a original DSI connectivity matrix(r = 0.311, p< 10 ), b spectral embeddings matrix reconstructions (r = 0.13, p< s s −6 −6 10 ) and c node2vec matrix reconstructions (r = 0.328, p< 10 ) 6 NATURE COMMUNICATIONS (2018) 9:2178 DOI: 10.1038/s41467-018-04614-w www.nature.com/naturecommunications | | | Empirical functional connectivity Empirical functional connectivity Empirical functional connectivity NATURE COMMUNICATIONS | DOI: 10.1038/s41467-018-04614-w ARTICLE −6 connections (r = 0.52, p< 10 ) (respectively green and red lesion in this region may result in acquired prosopagnosia, a symbols in Fig. 5). severe deﬁcit in face perception . Nevertheless, network-wide Dataset 2: We obtained a strong positive correlation (r = 0.52, effects associated with such a lesion were not yet examined −6 p< 10 ) between predicted functional connections and the explicitly. However our previous studies revealed that critical empirical functional connections in the testing set which was regions such as the right FFA serve as a hub in the face network −6 apparent for both direct connections (r = 0.52, p< 10 ) and when participants view an intact face but its connectivity is indirect connections (r = 0.6, p = 0.001) (respectively green and disrupted by physically manipulating the face (e.g., 180 degree red symbols in Supplementary Fig. 4). rotation of the face ). Critically under such disrupted conditions, Thus, the CE encompasses considerable information regarding additional regions (the right LOC, the right IPS and the right indirect functional connections matching, or even exceeding prior inferior temporal cortices) become involved and take on the roles structure-function correspondences obtained by computer simu- of hubs in this modiﬁed network . Similar ﬁndings are also 28 22 lation , as well as analytic graph-based models . obtained when intact faces are perceived by individuals with As a further validation, we also conducted a simple linear impaired congenital face processing abilities (congenital proso- regression analysis. Dataset 1: a positive correlation (r = 0.45, p pagnosia—CP ). Hence, we predicted that a lesion to the right −6 < 10 ) was obtained between the predicted functional connec- FFA would simulate a disruption of the face network that mimics tion and the empirical functional connection in the testing set conditions of impaired face perception and would consequently which was apparent for both direct connections r = 0.41, p< 10 affect the connectivity of the related hubs. Here we attempt to −6 −6 and indirect connections (r = 0.32, p< 10 ). Dataset 2: A simulate network modiﬁcations to the face system that may elicit −6 positive correlation (r = 0.41, p< 10 ) was measured between similar effects to those described above using an artiﬁcial lesion. the predicted functional connection and the empirical functional Employing the CE framework, it is feasible to estimate how a connection in the testing set which was apparent for both direct lesion of the right FFA node, a major hub of the face network, −6 connections (r = 0.41, p< 10 ) and indirect connections (r = would causally affect the entirety of the brain network. A node s s −6 0.57, p< 10 ). The ﬁts obtained from the linear regression are lesion is performed by setting all of its connections to zero. Using below those obtained with our deep learning pipeline. a permutation test with 10,000 iterations (see methods for details), we calculated differences between the pre-lesion and post-lesion simulated functional connectivity. Following the Prediction of functional connectivity following FFA lesion. The lesion, the functional connectivity of each edge could either high predictive power of connectome embeddings provides an decrease (pre-lesion > post-lesion) or increase (post-lesion > pre- opportunity for a new type of predictive model to bridge brain lesion). Only simulated edge differences which were greater than structure and function. One potential application is to predict all 10,000 permutation differences were considered statistically changes in functional connectivity that result from changes in signiﬁcant. structure-based connectome embeddings. Speciﬁcally, one can Dataset 1: The differences between the pre-lesioned and post- create an embedding of the structural connectome after a lesioned predicted functional connectivity brain network were manipulation such as an artiﬁcial lesion or a selective enhance- quantiﬁed using a measure of node degree difference, which ment of speciﬁc nodes or edges (e.g., ). This embedding can captures the difference in the number of signiﬁcant edges then be used to predict the functional connectivity following the attached to a node . Following the lesion, the right lateral manipulation. occipital cortex (LOC) and the right inferior parietal sulcus (IPS) We utilized the face network as a testbed for exemplifying this nodes showed the highest increase in nodal degree (an increase of framework. Face perception is accomplished via the coordinated 27 and 9 edges, respectively). Conversely, the right LOC and right activity of a face processing network . One of the major hubs of inferior temporal cortex showed the highest decrease in nodal 31, 32 the face network is the right fusiform face area (FFA) .A degree as a result of the lesion (an increase of 20 and 17 edges, respectively). Dataset 2: Following the lesion, the right lateral occipital cortex (LOC) and the right inferior temporal cortex nodes showed the highest increase in nodal degree (an increase of 1.5 20 and 14 edges, respectively). The inferior parietal cortex was th ranked 6 in terms of nodal degree. Conversely, the right LOC 1.0 and right parahippocampal area showed the highest decrease in nodal degree as a result of the lesion (an increase of 36 and 15 edges, respectively), and the right inferior temporal cortex was 0.5 only ranked third (10 edges). These simulated results are consistent with the hubs associated with CP but they are also 0.0 evident when manipulating the network using a behavioral face 32, 34 inversion paradigm (Fig. 6). Note, that the two datasets produced some minor differences in –0.5 the ranking of the nodes, which showed the highest increase in Direct edges Indirect edges nodal degree in both contrasts and in the speciﬁc affected edges. –1.0 Given that the two datasets are completely independent and have –1.0 –0.5 0.0 0.5 1.0 1.5 2.0 distinct preprocessing pipelines, as well as different extracted Empirical functional connectivity measures for structural connectivity (See Methods for details), Fig. 5 Prediction of resting state functional connectivity from structural such differences in the observed ﬁndings are conceivable. embbedings using deep learning. Green and red dots mark direct and indirect edges respectively. A signiﬁcant correlation between the empirical functional connections and the predicted functional connections is apparent Relationship of embeddings and standard topological mea- −6 when all connections are taken into account (r = 0.6, p< 10 ), as well sures. As is evident, CE captures important topological infor- −6 within the direct (r = 0.6, p< 10 ) and indirect connections separately (r mation. Nevertheless, to investigate whether there is a potential s s −6 = 0.52, p< 10 ) relationship between CE and more standard lower-order NATURE COMMUNICATIONS (2018) 9:2178 DOI: 10.1038/s41467-018-04614-w www.nature.com/naturecommunications 7 | | | Predicted functional connectivity Ih_frontalpole Ih_frontalpole brainstem Ih_amygdala Ih_amygdala brainstem rh_insula rh_insula Ih_accumbensarea Ih_accumbensarea Ih_parsorbitalis Ih_parsorbitalis Ih_hyppocampus rh_transversetemporal Ih_hyppocampus rh_transversetemporal Ih_putamen Ih_putamen Ih_lateralorbitofrontal Ih_lateralorbitofrontal Ih_pallidum Ih_pallidum rh_superiortemporal rh_superiortemporal rh_hyppocampus rh_hyppocampus Ih_caudate Ih_caudate Ih_putamen Ih_putamen rh_bankssts rh_bankssts Ih_thalamusproper Ih_thalamusproper rh_accumbensarea rh_accumbensarea Ih_thalamusproper Ih_thalamusproper rh_middletemporal rh_middletemporal rh_putamen rh_putamen Ih_transversetemporal Ih_transversetemporal Ih_middletemporal Ih_middletemporal rh_caudate rh_caudate rh_entorhinal Ih_superiortemporal Ih_superiortemporal rh_entorhinal Ih_parahippocampal Ih_parahippocampal rh_parahippocampal rh_insula rh_insula rh_parahippocampal Ih_bankssts Ih_bankssts Ih_fusiform Ih_fusiform rh_lingual rh_lingual Ih_middletemporal Ih_middletemporal rh_middletemporal rh_middletemporal Ih_lingual Ih_lingual rh_lateraloccipital Ih_inferiortemporal Ih_inferiortemporal rh_lateraloccipital rh_inferiortemporal rh_inferiortemporal Ih_lateraloccipital Ih_lateraloccipital rh_pericalcarine Ih_temporalpole Ih_temporalpole rh_pericalcarine rh_entorhinal rh_entorhinal Ih_pericalcarine Ih_pericalcarine rh_parahippocampal rh_parahippocampal rh_cuneus Ih_entorhinal Ih_entorhinal rh_cuneus Ih_cuneus Ih_cuneus rh_lingual rh_lingual Ih_parahippocampal Ih_parahippocampal rh_precuneus rh_precuneus Ih_precunesus Ih_precuneus Ih_fusiform Ih_fusiform rh_lateraloccipital rh_lateraloccipital rh_inferiorparietal rh_inferiorparietal Ih_inferiorparietal Ih_inferiorparietal rh_pericalcarine rh_pericalcarine Ih_lingual Ih_lingual ARTICLE NATURE COMMUNICATIONS | DOI: 10.1038/s41467-018-04614-w Post-lesion > pre-lesion Pre-lesion > Post-lesion Fig. 6 Simulating the effect of an artiﬁcial lesion to the right FFA on functional connectivity. Green and purple denote right and left hemispheric nodes respectively and the simulated edge differences which were signiﬁcantly affected by the lesion are depicted by blue lines (edges) connecting the nodes. Red denotes a selected node and its statistically signiﬁcant edges. a The right LOC and the right IPS nodes had the highest nodal degree in the post-lesion > pre-lesion contrast. b The right inferior temporal cortex and the right LOC nodes had the highest nodal degree in the pre-lesional > lesion contrast topological measures, the correlation between each dimension of theoretical measures were low and followed no obvious pattern the CE and several topological measures was calculated across (Fig. 7). The signiﬁcant correlations seem sporadic and they do nodes. Speciﬁcally, the Spearman’s correlation coefﬁcient between not account for most of the variance associated with CE. This CE and measures of node centrality (degree, strength, eigenvector suggests that CE captures network attributes beyond those centrality), a nodal measure of integration (betweenness cen- captured by more standard topological measures. trality) and a nodal measure of segregation (clustering coefﬁcient) was examined. Discussion Of the measures tested, the eigenvector centrality showed the The utilization of word embeddings techniques such as word2vec highest magnitude of correlation to only 2 of the CE dimensions for network science in general, and speciﬁcally in the context of −6 3, 35 (maximum Spearman’s correlation of ρ = 0.51, p< 10 and connectomics, holds great promise . In the current study we −6 minimum correlation of ρ = −0.57, p< 10 ). Nevertheless, the demonstrated that CE representation encompasses high-level correlation values between the CE elements and the graph topological information such as inter-hemispheric similarities. 8 NATURE COMMUNICATIONS (2018) 9:2178 DOI: 10.1038/s41467-018-04614-w www.nature.com/naturecommunications | | | rh_superiorparietal rh_superiorparietal Ih_superiorparietal Ih_superiorparietal rh_inferiorparietal rh_supramarginal rh_inferiorparietal Ih_lateraloccipital Ih_lateraloccipital rh_supramarginal Ih_supramarginal Ih_supramarginal Ih_pericalcarine Ih_pericalcarine rh_superiorparietal rh_postcentral rh_superiorparietal rh_postcentral Ih_caudalanteriorcingulate rh_supramarginal rh_supramarginal Ih_caudalanteriorcingulate Ih_precuneus Ih_precuneus rh_isthmuscingulate rh_isthmuscingulate Ih_rostralanteriorcingulate rh_isthmuscingulate rh_isthmuscingulate Ih_rostralanteriorcingulate Ih_inferiorparietal Ih_inferiorparietal rh_caudalanteriorcingulate rh_caudalanteriorcingulate Ih_superiorparietal Ih_superiorparietal rh_posteriorcingulate Ih_caudalmiddlefrontal rh_posteriorcingulate Ih_caudalmiddlefrontal rh_rostralanteriorcingulate rh_rostralanteriorcingulate rh_paracentral rh_paracentral Ih_parsopercular is Ih_supramarginal Ih_supramarginal Ih_parsopercularis rh_precentral rh_precentral rh_precentral rh_precentral Ih_frontalpole Ih_postcentral Ih_postcentral Ih_frontalpole rh_caudalmiddlefrontal rh_caudalmiddlefrontal Ih_isthmuscingulate Ih_isthmuscingulate rh_caudalmiddlefrontal rh_caudalmiddlefrontal Ih_lateralorbitofrontal Ih_lateralorbitofrontal rh_rostralmiddlefrontal rh_rostralmiddlefrontal Ih_caudalanteriorcingulate rh_amygdala Ih_caudalanteriorcingulate rh_superiorfrontal rh_superiorfrontal rh_amygdala rh_parsopercularis rh_parsopercularis Ih_paracentral Ih_paracentral rh_rostralmiddlefrontal rh_rostralmiddlefrontal rh_accumbensarea rh_accumbensarea rh_parstriangularis rh_parstriangularis rh_parsopercularis Ih_precentral Ih_precentral rh_parsopercularis rh_pallidum rh_pallidum rh_parstriangularis rh_parstriangularis rh_medialorbitofrontal rh_medialorbitofrontal Ih_caudalmiddlefrontal Ih_caudalmiddlefrontal rh_putamen rh_putamen rh_parsorbitalis rh_parsorbitalis rh_parsorbitalis rh_parsorbitalis Ih_superiorfrontal Ih_superiorfrontal rh_caudate rh_caudate rh_lateralorbitofrontal Ih_rostralmiddlefrontal Ih_rostralmiddlefrontal rh_lateralorbitofrontal rh_lateralorbitofrontal rh_lateralorbitofrontal rh_thalamusproper rh_thalamusproper Spearman NATURE COMMUNICATIONS | DOI: 10.1038/s41467-018-04614-w ARTICLE the Human Connectome Project (100 subjects) and replicated the main results. Note that due to technical differences in data 0.5 acquisition, the preprocessing pipeline, as well as the extracted measure for structural connectivity were different between the datasets which further strengthens the generalizability of our approach and methods. The initial inter-hemispheric analogies test demonstrated that CE vector representations capture the 0.25 11–14 known functional homotopy organizational principle .As predicted, the relation between most pairs of regions in one hemisphere was analogous to the same pairwise relation in the other hemisphere, with the CE approach exhibiting superior performance over previous embedding techniques. Next, we examined whether the correlation between CE and the structural connectivity matrix reﬂects higher-order attributes over and above the dyadic pattern of structural connections. This indeed turned out to be the case as homotopic inter-hemispheric connections were more prominent in the CE connectivity reconstruction compared with the original structural connectivity –0.25 matrix, due to topological features which are captured by the embedding such as homotopic inter-hemispheric connectivity. Moreover, the CE matrices were more strongly correlated with resting state functional connectivity than the original connectivity matrix. Furthermore, a deep learning algorithm was utilized to –0.5 improve the mapping between CE and functional connectivity utilizing CE representations. This mapping produced high cor- relation coefﬁcients between the predicted and empirical func- tional connectivity values, both for direct, as well as in-direct connections, which are more difﬁcult to estimate . The CE approach outperformed previous models of structure-function Topological measure 22, 28 correspondences . Future studies may utilize the same pre- Fig. 7 Correlations between CE dimensions and standard topological dictive algorithm to predict missing structural connectivity in measures. The eigenvector centrality showed the highest magnitude of species and modalities where only partial structural connectivity correlation to only two of the CE dimensions (maximum Spearman 42 data is available . −10 correlation of ρ = 0.51, p <10 and minimum correlation of ρ = −0.57, p To capitalize on the high predictive power of CE-functional −10 <10 ). Nevertheless, generally the correlation values between the CE mapping, we tested whether it is possible to predict changes in elements and the graph theoretical measures were low and followed no functional connectivity that result from changes in structure- obvious, meaningful pattern based connectome embeddings. Speciﬁcally, we used the face network as a test-bed and simulated a structural lesion to the 31, 32 Moreover, CE was able to reveal, at levels superior to previous right FFA, a well-documented hub of the face network . The methodology, the relationship and mutual prediction of func- results of the simulation aligned well with empirical ﬁndings tional and structural connectivity, and was able to simulate the exploring the face network. Hyper-connectivity in the right LOC, effects of localized network lesions on the global pattern of inferior temporal cortex and IPS which was reported in previous 32, 34, 43 functional connectivity. empirical studies , was also predicted by our CE-based 44, 45 The integration of machine learning techniques with models of model. In line with previous work , our ﬁndings suggest that brain networks is a relatively new domain and examples of suc- network-wide functional changes can result from a localized cessful applications are still limited . Previous studies have manipulation such as the suppression of a single node. mostly utilized embeddings as a dimensionality reduction step of Our results, along with the modeling framework, make a fur- fMRI data for subsequent machine learning tasks such as classi- ther step towards the possibility of examining causality in the 36 37 ﬁcation of patients with schizophrenia , depression , Alzhei- context of structural and functional network alterations. The 38 39 mer’s disease and multiple sclerosis . Differences between same framework can be used to induce simulated lesions, over- structural connectomes and deterioration of connectomes as a expression of nodes, edges, as well as entire sub-networks. Such result of edge deletion have previously been investigated using the simulations might help to elucidate the structural basis for net- average similarity of heat diffusion . However, the word2vec work alterations which occur in neuro-developmental disorders family of models together with deep learning algorithms have not such as Autism Spectrum Disorder (ASD) in which hyper- yet been applied in the context of brain networks. Furthermore, connectivity is apparent , and developmental dyslexia and this study is the ﬁrst to create a comprehensive machine-learning acquired prosopagnosia where the left and right fusiform gyri are 31, 46 framework which translates meaningful structural embeddings to focally implicated, respectively . Moreover, one may simulate functional connectivity, giving rise to a novel predictive model of the changes in network topology observed in normal participants how functional connectivity is affected by alterations of structural under different cognitive and perceptual demands. For example, a elements which may become useful in the investigation of number of studies have demonstrated the complementary abnormal brain networks. extrinsic and intrinsic networks associated with external inputs To test whether CE vector representations reﬂect known and intrinsically driven processing respectively. . attributes of brain topology/topography and can be interpreted The embedding algorithm (node2vec ) and the parameters and manipulated using linear operations, several benchmarks employed in the current study are not necessarily optimal, and were tested on two independent datasets. Critically, we performed are subject to further improvement in future extensions of this an extensive validation with a rigorously preprocessed subset of work. Our work suggests that CE provides a powerful approach NATURE COMMUNICATIONS (2018) 9:2178 DOI: 10.1038/s41467-018-04614-w www.nature.com/naturecommunications 9 | | | Degree Strength Eigen. cent. Clust. coeff. Betw. cent Connectome embedding dimension ARTICLE NATURE COMMUNICATIONS | DOI: 10.1038/s41467-018-04614-w 56, 57 The data was processed using the HCP functional pipeline . This pipeline for exploring the higher-order network structure of connectome included artifact removal, motion correction and registration to standard space. data sets, with potential applications in modeling and comparing 56, 57 Full details on the pipeline can be found in . The main steps were: spatial individual differences in human connectomes across development (minimal) pre-processing, in both volumetric and grayordinate forms (i.e., where and in clinical conditions. Another future application is to use CE brain locations are stored as surface vertices ; weak highpass temporal ﬁltering (>2000s full width at half maximum) applied to both forms, achieving slow drift to uncover relationships and homologies among brain archi- removal. MELODIC ICA applied to volumetric data; artifact components tectures across species. identiﬁed using FIX . Artifacts and motion-related time courses were regressed out (i.e., the 6 rigid-body parameter time-series, their backwards-looking temporal derivatives, plus all 12 resulting regressors squared) of both volumetric and grayordinate data . Furthermore, global gray matter signal was regressed out of Methods the voxel time courses ; a bandpass ﬁrst-order Butterworth ﬁlter in forward and MRI Data. Datasets: The analyses were conducted on a dataset of 40 subjects 70, 71 reverse directions [0.001 Hz, 0.08 Hz] was applied (Matlab functions butter (Dataset 1) and validated on an independent dataset of 100 subjects (Dataset 2). and ﬁltﬁlt); the voxel time courses were Z-scored and then averaged per brain Dataset 1 (Lausanne). Prior to collection of MRI data, the project was submitted 51, 52 region of the 82 regions of the Desikan-Kiliany atlas , excluding outlier time for approval to the University of Lausanne Ethics Committee [Institutional Review points outside of 3 standard deviation from the mean, using the workbench Board (IRB)]. The study protocol was approved by the local IRB and informed software (workbench command-cifti-parcellate). written consent from each subject was obtained prior to study inclusion. Pearson correlation coefﬁcients between pairs of nodal time courses were Forty healthy subjects (16 females; 25.3 ± 4.9 y old), with no relevant medical or calculated (MATLAB command corr), resulting in a symmetric connectivity matrix psychiatric history, underwent an MRI session on a 3 T Siemens Trio scanner with 22, 48 for each fMRI session of each subject. Finally, the mean connectivity matrix was a 32-channel head coil (previously reported on ). T1 weighted magnetization- calculated for each subject across all 4 acquisitions. prepared rapid acquisition with gradient echo (MPRAGE) sequence was 1-mm in- 49 For both of the datasets, the subsequent structural connectivity analyses and plane resolution and 1.2-mm slice thickness. Diffusion Spectrum Imaging (DSI) modeling were carried out on a group consensus matrix, built by averaging over all included 128 diffusion weighted volumes + 1 reference b0 volume, maximum b- existing connections (expressed as ﬁber densities) that were present in at least 25% value 8000 s/mm2, 2.2 × 2.2 × 3.0 mm voxel size and with TR 6800 ms and TE 144 of participants . For the functional data, the consensus matrix was built from ms. BOLD contrast was recorded with a gradient echo EPI sequence of 3.3-mm in- averaging over all participants. plane resolution and 3.3-mm slice thickness and with TR 1920 ms and TE 30 ms.. DSI, resting-state fMRI, and MPRAGE data were processed using the Connectome Mapping Toolkit Word and network embedding. Word2vec mainly consists of two models; skip- Segmentation of gray and white matter was based on MPRAGE volumes. gram and the continuous bag of words (CBOW). Brieﬂy, given a corpus of words w Cerebral cortex was parcellated into a set of 83 regions of the Desikan-Kiliany and their context c, the goal is to maximize the conditional probability p(c|w; θ)in 51, 52 atlas . Whole-brain streamline tractography was performed on reconstructed the skip-gram model or p(w|c; θ) in the CBOW model by estimating the parameters DSI data . During the resting-state fMRI acquisition, subjects were lying in the θ . The produced parameters or vectors capture linguistic regularities and may be 7, 8 scanner with eyes open for 9 min. Functional data preprocessing included motion the basis for various subsequent tasks . correction, white matter, cerebrospinal ﬂuid, global and movement signals The analog for context in the network domain are streams of randomly 54, 55 regression, linear detrending, motion scrubbing, and low-pass ﬁltering . generated walks in the network. The way the network sentences are generated Average time series were computed for each cortical region and functional distinguishes between different approaches of network embeddings. One of the connectivity was estimated as Pearson cross-correlation . recent implementation of network node embeddings is node2vec which controls Dataset 2 (Human Connectome Project; HCP). Prior to collection of MRI data, for the depth of the random walk using 2 parameters allowing for local or global the HCP scanning protocol was approved by the local Institutional Review Board at random walks which lead to different representation of the nodes (Fig. 1a-e) . Washington University in St. Louis and informed written consent from each Compared with unsupervised feature learning approaches, which utilize the subject was obtained prior to study inclusion. Full details on the HCP dataset have spectral properties of graphs, the node2vec model has been shown to have higher 56–58 been published previously . Out of the HCP 900 subjects data release, 100 predictive power across a range of subsequent supervised learning node hundred unrelated subjects were used for the analysis . Individual structural and classiﬁcation tasks and link prediction of edges . Moreover, it has been shown that functional connectomes were estimated following the same processing procedures similar network embeddings algorithms capture the k-step (k = 1, 2, 3,..) relation as detailed in Amico & Goni . between each vertex and its k-step neighbors in the graph while projecting all such Structural data: Very high-resolution acquisitions (1.25 mm isotropic) were k-step relational information into a common subspace . 3, 74 obtained by using a Stejskal–Tanner (monopolar) diffusion-encoding scheme. We used Node2vec and underlying Gensim python package to run the Sampling in q-space was performed by including 3 shells at b = 1000, 2000, and CBOW node2vec algorithm 500 times on the structural connectivity matrix, as it 3000 s/mm2. For each shell corresponding to 90 diffusion gradient directions and 5 can produce different outcomes in each iteration. Each iteration consisted of 800 b = 0’s acquired twice were obtained, with the phase encoding direction reversed random walks with a length of 20 steps. The dimension of the embedded vectors for each pair (i.e., LR and RL pairs). Directions were optimized within and across was set to 30 (the length of each vector which represents a node) and the window shells (i.e., staggered) to maximize angular coverage using the approach of Caruyer, size (the number of steps from each node) determining the context of each node, et al. (2011) (http://www-sop.inria.fr/members/Emmanuel.Caruyer/q-space- was set to 3. Parameters of the algorithm were set to correspond to a localized sampling.php), and form a total of 270 non-collinear directions for each PE random walk (p = 0.1, q = 1.6). The walk probabilities were weighted according to direction. Correction for EPI and eddy-current-induced distortions in the diffusion the weight of the connectome edges. data was based on manipulation of the acquisitions so that a given distortion manifests itself differently in different images . To ensure better correspondence Deep learning better predicts functional connectivity. Previously, Grover and between the phase-encoding reversed pairs, the whole set of diffusion-weighted Leskovec (2016) have shown that the Hadamard operation, the element-wise (DW) volumes is acquired in six separate series. These series were grouped into multiplication between pair of vectors, was efﬁcient for learning edge features three pairs, and within each pair the two series contained the same DW directions across various domains. The mapping between structural embedding and func- but with reversed phase-encoding (i.e., a series of Mi DW volumes with RL phase- tional connectivity can be learned in such a setting utilizing node-pairs repre- encoding is followed by a series of Mi volumes with LR phase-encoding, i = [1–3]). sentations, while adopting a supervised learning cross validation scheme. The HCP DWI data were processed following the MRtrix3 guidelines (http:// Speciﬁcally, the edges of the mean functional connectivity matrix were randomly mrtrix.readthedocs.io/en/latest/tutorials/hcp_connectome.html). In summary, a divided into training (75%) and testing sets (25%). A deep learning multi-layer tissue-segmented image appropriate for anatomically constrained tractography was 63 perceptron model, as implemented in Keras with a Tensorﬂow , was used in generated (ACT , MRtrix command 5ttgen); the multi-shell multi-tissue response which the independent variables were deﬁned as the Hadamard embbedings and function was estimated (ref. , MRtrix command dwi2response msmt_5tt) and a the dependent variable was deﬁned as the functional connectivity. We ﬁrst opti- multi-shell, multi-tissue constrained spherical deconvolution was performed (ref. 65 mized the architecture of the network using a cross validation grid-search over the , MRtrix dwi2fod msmt_csd); afterwards, an initial tractogram was generated parameter space using only the training set. This yielded a 4 layer fully connected (MRtrix command tckgen, 10 million streamlines, maximum tract length = 250 network with 350 neurons in each layer, dropout rate of 0.1, rectiﬁed linear units millimeters, FA cutoff = 0.06) and the successor of spherical-deconvolution 66 (RELU) activation function, batch size of 140, 170 epochs and Adam optimizer. Informed Filtering of Tractograms (SIFT2, ) methodology (MRtrix command 67 66 tcksift2) was applied. Both SIFT and SIFT2 methods provides more biologically meaningful estimates of structural connection density . Finally, the SIFT2 Prediction of functional outcome due to lesion. To test the effect of an artiﬁcial outputed streamlines were parcellated into a set of 82 regions of the Desikan- lesion on the functional connectivity, we ﬁrst constructed new 500 connectome 51, 52 Kiliany atlas (MRtrix command tck2connectome). embbedings following a right FFA removal (post-lesion embeddings compared Functional data: The fMRI resting-state runs (HCP ﬁlenames: rfMRI_REST1 with the original pre-lesion embeddings) as this is a major hub of the face 31, 32 and rfMRI_REST2) were acquired in separate sessions on two different days, with network . This was done by running the node2vec algorithm 500 times with two different acquisitions (left to right or LR and right to left or RL) per day. For all random initializations on the structural connectivity matrix after setting to 0 all of sessions, data from both the left-right (LR) and right-left (RL) phase-encoding runs the right FFA edges. As the embedded vector elements may change due to the 56, 57 were used to calculate connectivity matrices . random weights initialization conditions (but the cosine similarity between vectors 10 NATURE COMMUNICATIONS (2018) 9:2178 DOI: 10.1038/s41467-018-04614-w www.nature.com/naturecommunications | | | NATURE COMMUNICATIONS | DOI: 10.1038/s41467-018-04614-w ARTICLE is stable), the learned mapping between the pre-lesioned embedding and the 12. Lowe, M. J., Mock, B. J. & Sorenson, J. A. Functional connectivity in single and functional connectivity mapping is not generalizable to embeddings of another multislice echoplanar imaging using resting-state ﬂuctuations. Neuroimage 7, connectome. Therefore, we devised a training procedure whose goal is to learn a 119–132 (1998). mapping between the connectome embeddings to the functional connectivity that 13. Salvador, R. et al. Neurophysiological architecture of functional magnetic is invariant to the weight initialization conditions. To this end, we implemented a resonance images of human brain. Cereb. Cortex 15, 1332–1342 (2005). nested cross validation scheme in which the mapping between pre-lesion structural 14. Zuo, X.-N. et al. Growing together and growing apart: regional and sex embeddings and functional connectivity is trained across many random weights differences in the lifespan developmental trajectories of functional homotopy. initializations of the connectome embeddings using only a subset of the edges and J. Neurosci. Off. J. Soc. Neurosci. 30, 15034–15043 (2010). connectome embeddings to avoid overﬁtting the data. Speciﬁcally, the edges are 15. Innocenti, G. M. General organization of callosal connections in the cerebral divided into 3 folds (sub groups) such that in each iteration, 2 folds are used for cortex. in Sensory-Motor Areas and Aspects of Cortical Connectivity (eds. training (2268 edges) and 1 fold is used for testing (1134 edges). Moreover, the 500 Jones, E. G. & Peters, A.) 291–353 (Springer US, 1986). connectome embeddings are also split randomly 3 times with a training set of 90% 16. Belkin, M. & Niyogi, P. Laplacian eigenmaps for dimensionality reduction and of the connectome embeddings (450 out of 500), and a testing set of 10% (50 out of data representation. Neural Comput. 15, 1373–1396 (2003). 500). The predictions were always made on embedding vectors and edges which 17. Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. were not in the training set. Furthermore, the training is conducted on the pre- Res. 12, 2825–2830 (2011). lesion connectome embeddings and the prediction is applied to the pre-lesion, as 18. Tang, L. & Liu, H. Leveraging social media networks for classiﬁcation. Data well as to the post-lesion connectome embeddings. Min. Knowl. Discov. 23, 447–478 (2011). Finally, a permutation resampling test was performed with 10,000 iterations to 19. Jbabdi, S. & Johansen-Berg, H. Tractography: where do we go from here? compare each edge between the pre-lesion and the post-lesion connectome Brain Connect. 1, 169–183 (2011). predictions. Speciﬁcally, in each iteration the groups are randomly permuted and 20. Swanson, L. W., Hahn, J. D. & Sporns, O. Organizing principles for the the difference of each edge is calculated between the two groups, effectively forming cerebral cortex network of commissural and association connections. Proc. the null hypothesis that the groups are invariant under label permutation. Only Natl Acad. Sci. 114, E9692–E9701 (2017). edges which had a mean difference greater than the permuted mean difference 21. Friston, K. J. Functional and effective connectivity in neuroimaging: a (either for pre-lesion > post-lesion or post-lesion > pre-lesion) across all 10,000 synthesis. Hum. Brain. Mapp. 2,56–78 (1994). iterations were considered signiﬁcantly different. 22. Goñi, J. et al. Resting-brain functional connectivity predicted by analytic measures of network communication. Proc. Natl Acad. Sci USA 111, 833–838 Code availability. Code was written using standard python functions and freely (2014). available packages (see connectome embedding implementation at https://github. 23. Hermundstad, A. M. et al. Structural foundations of resting-state and task- com/gidonro/Connectome-embeddings). The original node2vec implementation based functional connectivity in the human brain. Proc. Natl Acad. Sci. 110, can be found at https://github.com/aditya-grover/node2vec. 6169–6174 (2013). 24. Honey, C. J. et al. Predicting human resting-state functional connectivity from Data availability. Human Connectome dataset: Data were provided by the Human structural connectivity. Proc. Natl Acad. Sci. 106, 2035–2040 (2009). Connectome Project, WU-Minn Consortium (Principal Investigators: David Van 25. Passingham, R. E., Stephan, K. E. & Kötter, R. The anatomical basis of Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and functional localization in the cortex. Nat. Rev. Neurosci. 3, 606–616 (2002). Centers that support the NIH Blueprint for Neuroscience Research; and by the 26. Vincent, J. L. et al. Intrinsic functional architecture in the anaesthetized McDonnell Center for Systems Neuroscience at Washington University (https:// monkey brain. Nature 447,83–86 (2007). doi.org/10.1038/nn.4361). 27. Smith, S. M. et al. Network modelling methods for FMRI. Neuroimage 54, Lausanne dataset: The relevant data are available from the authors upon 875–891 (2011). reasonable request. 28. Deco, G. et al. Identiﬁcation of Optimal Structural Connectivity Using Functional Connectivity and Neural Modeling. J. Neurosci. 34, 7910–7916 (2014). Received: 22 July 2017 Accepted: 18 April 2018 29. Supekar, K. et al. Brain hyper-connectivity in children with autism and its links to social deﬁcits. Cell Rep. 5, 738–747 (2013). 30. Grill-Spector, K., Weiner, K. S., Kay, K. & Gomez, J. The functional neuroanatomy of human face perception. Annu. Rev. Vis. Sci. 3, 167–196 (2017). 31. Parvizi, J. et al. Electrical stimulation of human fusiform face-selective regions References distorts face perception. J. Neurosci. 32, 14915–14920 (2012). 1. Sporns, O., Tononi, G. & Kötter, R. The human connectome: a structural 32. Rosenthal, G., Sporns, O. & Avidan, G. Stimulus dependent dynamic description of the human brain. PLoS Comput. Biol. 1, e42 (2005). reorganization of the human face processing network. Cereb. Cortex 27, 2. Craddock, R. C. et al. Imaging human connectomes at the macroscale. Nat. 4823–4834 (2016). Methods 10, 524–539 (2013). 33. Barton, J. J. S. Structure and function in acquired prosopagnosia: lessons from 3. Grover, A. & Leskovec, J. Node2Vec: Scalable feature learning for networks. in a series of 10 patients with brain damage. J. Neuropsychol. 2, 197–225 (2008). Proceedings of the 22Nd ACM SIGKDD International Conference on 34. Rosenthal, G. et al. Altered topology of neural circuits in congenital Knowledge Discovery and Data Mining 855–864 (ACM, 2016). prosopagnosia. eLife 6, e25069 (2017). 4. Perozzi, B., Al-Rfou, R. & Skiena, S. DeepWalk: Online Learning of Social 35. Richiardi, J., Achard, S., Bunke, H. & Ville, D. V. D. Machine Learning with Representations. in Proceedings of the 20th ACM SIGKDD International Brain Graphs: Predictive Modeling Approaches for Functional Imaging in Conference on Knowledge Discovery and Data Mining 701–710 (ACM, 2014). Systems Neuroscience. IEEE Signal Process. Mag. 30,58–70 (2013). 5. Rubinov, M. & Sporns, O. Complex network measures of brain connectivity: 36. Shen, H., Wang, L., Liu, Y. & Hu, D. Discriminative analysis of resting-state uses and interpretations. Neuroimage 52, 1059–1069 (2010). functional connectivity patterns of schizophrenia using low dimensional 6. Zhan, L. et al. Boosting brain connectome classiﬁcation accuracy in embedding of fMRI. Neuroimage 49, 3110–3121 (2010). Alzheimer’s disease using higher-order singular value decomposition. Front. 37. Craddock, R. C., Holtzheimer, P. E., Hu, X. P. & Mayberg, H. S. Disease state Neurosci. 9, 257 (2015). prediction from resting state functional connectivity. Magn. Reson. Med. 62, 7. Mikolov, T., Chen, K., Corrado, G. S. & Dean, J. Efﬁcient estimation of word 1619–1628 (2009). representations in vector space. ICLR workshop (2013). 38. Wang, K.et al. Discriminative analysis of early Alzheimer’s disease based on 8. Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S. & Dean, J. Distributed two intrinsically anti-correlated networks with resting-state fMRI. Medical representations of words and phrases and their compositionality. in Advances Image Computing andComputer Assisted Intervention. in MICCAI in Neural Information Processing Systems 26 (eds. Burges, C. J. C., Bottou, L., International Conference on Medical Image Computing and Computer Assisted Welling, M., Ghahramani, Z. & Weinberger, K. Q.) 3111–3119 (Curran Intervention. (9), 340–347 (Springer, Berlin, 2006). Associates, Inc., 2013). 39. Richiardi, J. et al. Classifying minimally disabled multiple sclerosis patients 9. Mikolov, T., Yih, W. & Zweig, G. Linguistic regularities in continuous space from resting state functional connectivity. Neuroimage 62, 2021–2033 (2012). word representations. Proc. NAACL Hlt. 13, 746–751 (2013). 40. Hammond, D. K., Gur, Y. & Johnson, C. R. Graph diffusion distance: A 10. Cao, S., Lu, W. & Xu, Q. GraRep: learning graph representations with global difference measure for weighted graphs based on the graph Laplacian structural information. in Proceedings of the 24th ACM International on exponential kernel. in 2013 IEEE Global Conference on Signal and Information Conference on Information and Knowledge Management 891–900 (ACM, Processing 419–422 (2013). 2015). 41. Adachi, Y. et al. Functional connectivity between anatomically unconnected 11. Cordes, D., Haughton, V., Carew, J. D., Arfanakis, K. & Maravilla, K. areas is shaped by collective network-level effects in the Macaque cortex. Hierarchical clustering to measure connectivity in fMRI resting-state data. Cereb. Cortex 22, 1586–1592 (2012). Magn. Reson. Imaging 20, 305–317 (2002). NATURE COMMUNICATIONS (2018) 9:2178 DOI: 10.1038/s41467-018-04614-w www.nature.com/naturecommunications 11 | | | ARTICLE NATURE COMMUNICATIONS | DOI: 10.1038/s41467-018-04614-w 42. Hinne, M. et al. The missing link: predicting connectomes from noisy and 69. Salimi-Khorshidi, G. et al. Automatic denoising of functional MRI data: partially observed tract tracing data. PLoS. Comput. Biol. 13, e1005374 (2017). combining independent component analysis and hierarchical fusion of 43. Matsuyoshi, D. et al. Dissociable cortical pathways for qualitative and classiﬁers. Neuroimage 90, 449–468 (2014). quantitative mechanisms in the face inversion effect. J. Neurosci. 35, 70. Power, J. D. et al. Methods to detect, characterize, and remove motion artifact 4268–4279 (2015). in resting state fMRI. Neuroimage 84, 320–341 (2014). 44. Aerts, H., Fias, W., Caeyenberghs, K. & Marinazzo, D. Brain networks under 71. Amico, E. et al. Mapping the functional connectome traits of levels of attack: robustness properties and the impact of lesions. Brain 139, 3063–3083 consciousness. Neuroimage 148, 201–211 (2017). (2016). 72. Marcus, D. S. et al. Informatics and data mining tools and strategies for the 45. Váša, F. et al. Effects of lesions on synchrony and metastability in cortical human connectome project. Front. Neuroinformatics 5, 4 (2011). networks. Neuroimage 118, 456–467 (2015). 73. Goldberg, Y. & Levy, O. word2vec Explained: deriving Mikolov et al.’s negative- 46. Ma, Y. et al. Cortical thickness abnormalities associated with dyslexia, sampling word-embedding method. Preprint at arXiv:1402.3722 (2014). independent of remediation status. Neuroimage Clin. 7, 177–186 (2015). 74. Řehůřek, R. & Sojka, P. Software framework for topic modelling with large 47. Golland, Y. et al. Extrinsic and intrinsic systems in the posterior cortex of the corpora. in Proceedings of the LREC 2010 Workshop on New Challenges for human brain revealed during natural sensory stimulation. Cereb. Cortex 17, NLP Frameworks 45–50 (ELRA, 2010). 766–777 (2007). 75. Abadi, M., et al. TensorFlow: a system for large-scale machine learning. in 48. Avena-Koenigsberger, A. et al. Using Pareto optimality to explore the Proceedings of the 12th USENIX Conference on Operating Systems Design and topology and dynamics of the human connectome. Philos. Trans. R. Soc. B Implementation 265–283 (2016). 369, 20130530 (2014). 49. Wedeen, V. J., Hagmann, P., Tseng, W. Y. I., Reese, T. G. & Weisskoff, R. M. Acknowledgements Mapping complex tissue architecture with diffusion spectrum magnetic We thank Dr. Carmel Sofer and Mr. Gidon Levakov for fruitful discussions regarding the resonance imaging. Magn. Reson. Med. 54, 1377–1386 (2005). theoretical framework of this work. This work was support by Israel Science Foundation 50. Daducci, A. et al. The connectome mapper: an open-source processing (ISF) grant 296/15 to G.A.; O.S. acknowledges support from the Indiana Clinical pipeline to map connectomes with MRI. PLoS One 7, e48121 (2012). Translational Sciences Institute (NIH UL1TR0011808), the National Science Foundation 51. Cammoun, L. et al. Mapping the human connectome at multiple scales with (1636892) and the US National Institutes of Health (R01-AT009036). F.V. acknowledges diffusion spectrum MRI. J. Neurosci. Methods 203, 386–397 (2012). support from the Gates Cambridge Trust. A.G. acknowledges support from the Swiss 52. Desikan, R. S. et al. An automated labeling system for subdividing the human National Science Foundation (grant #310030-156874). P.H. was supported by SNSF grant cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage #310030_156874 and #320030_130090. Imaging was performed at Centre d’Imagerie 31, 968–980 (2006). BioMedicale of the EPFL, UNIL, UNIGE, HUG, CHUV and the support of Leenards and 53. Hagmann, P. et al. Mapping the structural core of human cerebral cortex. Jeantet Foundation. J.G. acknowledges NIH R01EB022574 and NIH R01MH108467 and PLoS Biol. 6, e159 (2008). funding by the Indiana Clinical and Translational Sciences Institute (Grant Number 54. Fox, M. D., Zhang, D., Snyder, A. Z. & Raichle, M. E. The global signal and UL1TR0011808) from the NIH, National Center for Advancing Translational Sciences, observed anticorrelated resting state brain networks. J. Neurophysiol. 101, Clinical and Translational Sciences Award. 3270–3283 (2009). 55. Power, J. D., Barnes, K. A., Snyder, A. Z., Schlaggar, B. L. & Petersen, S. E. Spurious but systematic correlations in functional connectivity MRI networks Author contributions: arise from subject motion. Neuroimage 59, 2142–2154 (2012). G.R. conceptualized the research. G.R., O.S., G.A., and F.V. designed the methodology. G.R. 56. Glasser, M. F. et al. The minimal preprocessing pipelines for the human performed the formal analysis. G.R., O.S., and G.A., administrated the project, wrote, connectome project. Neuroimage 80, 105–124 (2013). reviewed, and edited the manuscript. E.A., J.G., A.G., and P.H. curated and pre-processed 57. Smith, S. M. et al. Resting-state fMRI in the human connectome project. the human imaging data. A.G. and P.H. acquired the Lausanne dataset. Neuroimage 80, 144–168 (2013). 58. Van Essen, D. C. et al. The human connectome project: a data acquisition perspective. Neuroimage 62, 2222–2231 (2012). Additional information Supplementary Information accompanies this paper at https://doi.org/10.1038/s41467- 59. Amico, E. & Goñi, J. Mapping hybrid functional‐structural connectivity traits in the human connectome. Netw. Neurosci.1–34 (2018). https://doi.org/ 018-04614-w. 10.1162/NETN_a_00049 60. Caruyer, E. et al. Optimal Design of Multiple Q-shells experiments for Diffusion Competing interests: The authors declare no competing interests. MRI. (2011). 61. Andersson, J. L. R., Skare, S. & Ashburner, J. How to correct susceptibility Reprints and permission information is available online at http://npg.nature.com/ distortions in spin-echo echo-planar images: application to diffusion tensor reprintsandpermissions/ imaging. Neuroimage 20, 870–888 (2003). 62. Tournier, J. D., Calamante, F. & Connelly, A. MRtrix: diffusion tractography in crossing ﬁber regions. Int. J. Imaging Syst. Technol. 22,53–66 (2012). 63. Smith, R. E., Tournier, J. D., Calamante, F. & Connelly, A. Anatomically- constrained tractography: improved diffusion MRI streamlines tractography through effective use of anatomical information. Neuroimage 62, 1924–1938 Open Access This article is licensed under a Creative Commons (2012). Attribution 4.0 International License, which permits use, sharing, 64. Christiaens, D. et al. Global tractography of multi-shell diffusion-weighted adaptation, distribution and reproduction in any medium or format, as long as you give imaging data using a multi-tissue model. Neuroimage 123,89–101 (2015). appropriate credit to the original author(s) and the source, provide a link to the Creative 65. Jeurissen, B., Tournier, J.-D., Dhollander, T., Connelly, A. & Sijbers, J. Multi- Commons license, and indicate if changes were made. The images or other third party tissue constrained spherical deconvolution for improved analysis of multi- material in this article are included in the article’s Creative Commons license, unless shell diffusion MRI data. Neuroimage 103, 411–426 (2014). indicated otherwise in a credit line to the material. If material is not included in the 66. Smith, R. E., Tournier, J.-D., Calamante, F. & Connelly, A. SIFT2: Enabling article’s Creative Commons license and your intended use is not permitted by statutory dense quantitative assessment of brain white matter connectivity using regulation or exceeds the permitted use, you will need to obtain permission directly from streamlines tractography. Neuroimage 119, 338–351 (2015). the copyright holder. To view a copy of this license, visit http://creativecommons.org/ 67. Smith, R. E., Tournier, J.-D., Calamante, F. & Connelly, A. SIFT: Spherical- licenses/by/4.0/. deconvolution informed ﬁltering of tractograms. Neuroimage 67, 298–312 (2013). 68. Jenkinson, M., Beckmann, C. F., Behrens, T. E. J., Woolrich, M. W. & Smith, S. © The Author(s) 2018 M. FSL. Neuroimage 62, 782–790 (2012). 12 NATURE COMMUNICATIONS (2018) 9:2178 DOI: 10.1038/s41467-018-04614-w www.nature.com/naturecommunications | | |
Nature Communications – Springer Journals
Published: Jun 5, 2018
It’s your single place to instantly
discover and read the research
that matters to you.
Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.
All for just $49/month
Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly
Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.
Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.
Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.
All the latest content is available, no embargo periods.
“Hi guys, I cannot tell you how much I love this resource. Incredible. I really believe you've hit the nail on the head with this site in regards to solving the research-purchase issue.”Daniel C.
“Whoa! It’s like Spotify but for academic articles.”@Phil_Robichaud
“I must say, @deepdyve is a fabulous solution to the independent researcher's problem of #access to #information.”@deepthiw
“My last article couldn't be possible without the platform @deepdyve that makes journal papers cheaper.”@JoseServera