A tale of two connectivities: intra- and inter-subject functional connectivity jointly enable better prediction of social abilities

Naturalistic functional magnetic resonance imaging (fMRI) paradigms, such as movie viewing, are attracting increased attention, given their ability to mimic the real-world cognitive demands on attention and multimodal sensory integration. Moreover, naturalistic paradigms allow for characterizing brain network responses associated with dynamic social cognition in a model-free manner using inter-subject functional connectivity (ISFC). While intra-subject functional connectivity (FC) characterizes the individual’s brain functional architecture, ISFC characterizes the neural coupling driven by time-locked extrinsic dynamic stimuli across individuals. Here, we hypothesized that ISFC and FC provide distinct and complementary information about individual differences in social cognition. To test this hypothesis, we examined a public movie-viewing fMRI dataset with 32 healthy adults and 90 typically developing children. Building three partial least squares regression (PLS) models to predict social abilities using FC and/or ISFC, we compared predictive performance to determine whether combining two connectivity measures could improve the prediction accuracy of individuals’ social-cognitive abilities measured by a Theory of Mind (ToM) assessment. Our results indicated that the joint model (ISFC + FC) yielded the highest predictive accuracy and significantly predicted individuals’ social cognitive abilities (rho = 0.34, p < 0.001). We also confirmed that the improved accuracy was not due to the increased feature dimensionality. In conclusion, we demonstrated that intra-/inter-subject connectivity encodes unique information about social abilities, and a joint investigation could help us gain a more complete understanding of the complex processes supporting social cognition.

Introduction

Humans are inherently social beings capable of drawing inferences about other people’s unobservable beliefs and intentions to navigate their social worlds. Using a suite of carefully designed cognitive tasks or resting-state paradigms, traditional functional magnetic resonance imaging (fMRI) studies have consistently identified a set of functional networks, termed the social brain, that supports this ability to perceive, understand, and react to the social world around us (Gallagher and Frith, 2003; Gobbini et al., 2007; Mitchell, 2008; Alcalá-López et al., 2018). While much of this work relies on artificial contexts, more recent novel paradigm-free functional neuroimaging such as naturalistic fMRI provides a mechanism to examine how the social brain may respond in complex, dynamic, real-world situations. With dynamic and rich stimuli such as movies, stories, games, and virtual reality, naturalistic paradigms more closely mimic real-world demands on attention and multimodal sensory integration than the abstract and restricted stimuli employed in conventional fMRI paradigms, allowing for researchers to study humans’ highly interactive socio-cognitive processes “in the wild” (Finn et al., 2018, 2020; Razi et al., 2018; Richardson et al., 2018; Redcay and Moraczewski, 2020). In addition to its higher ecological validity, naturalistic fMRI also enables examining multiple neural measures (e.g., intra- and inter-subject neural coupling) to probe the mechanisms underlying the fleeting socio-cognitive information processing (Sonkusare et al., 2019). Despite the recent progress made, we argue that a gap of knowledge still exists since most of the previous naturalistic studies have looked at neural coupling measures in isolation (c.f., Simony et al., 2016; Kim et al., 2018; Lynch et al., 2018; Demirtaş et al., 2019), which begs the question of whether we can benefit from jointly studying the two neural coupling measures.

One such neural measure is intra-subject neural coupling, or functional connectivity (FC), computed as the temporal correlation of time-series across different brain regions within an individual. Intra-subject neural coupling is commonly examined through resting-state FC (RSFC), which measures the intrinsic functional network architecture driven by spontaneous brain activities (Fox et al., 2005). Task-evoked FC, however, reflects brain network architecture patterns associated with a cognitive state, driven by a mixture of spontaneous and task-evoked brain activities (Cole et al., 2014). This task-evoked FC during movie viewing has shown great promises in studying individual differences as naturalistic paradigms may serve as an “amplifier” tapping into specific socio-cognitive domains, consistently outperforming RSFC (Vanderwal et al., 2017; Finn et al., 2020; Finn and Bandettini, 2021). For example, movie-viewing FC has led to more accurate predictions of phenotypes in the cognition and emotion domain (Finn and Bandettini, 2021).

Another neural measure is inter-subject neural coupling. Unique to naturalistic paradigms, inter-subject neural coupling characterizes brain responses associated with dynamic social cognition in a model-free manner (Nummenmaa et al., 2018), namely inter-subject correlation (ISC) and inter-subject functional connectivity (ISFC). Unlike the intra-subject FC, ISC and ISFC measure the neural coupling across individuals. ISC identifies the shared activation patterns of a given brain region across subjects (Hasson et al., 2004), and ISFC further delineates common functional connectivity patterns driven by the extrinsic time-locked dynamic stimuli (Simony et al., 2016). The shared activity and connectivity patterns across individuals reflect the shared understanding of the narratives (Nguyen et al., 2019), differ based on clinical diagnosis, e.g., autism (Salmi et al., 2013; Bolton et al., 2018) and attention-deficit/hyperactivity disorder (Salmi et al., 2020), are associated with collaboration outcomes (Xie et al., 2020), personality traits (Finn et al., 2018) and brain functional specialization in childhood (Moraczewski et al., 2018; Richardson, 2019).

Despite both intra- and inter-subject neural coupling showing great promise in furthering our understanding of individual differences in the neural underpinnings of social cognition, only a handful of studies have systematically evaluated two types of neural coupling measures together (Simony et al., 2016; Kim et al., 2018; Lynch et al., 2018; Demirtaş et al., 2019). We argue that a joint investigation of both neural coupling measures is needed because each measure may provide unique information about brain functions. Those studies that did examine both measures fell short of addressing the issue as they mainly focused on comparing ISFC and FC patterns. For instance, Lynch and colleagues found that ISFC patterns could not fully explain the FC changes during movie viewing (Lynch et al., 2018). Similarly, Demirtaş and colleagues confirmed that while there was overlap between the two neural coupling patterns, increased coupling within frontal brain regions and reduced coupling between frontal-parietal brain regions were observed in intra-subject FC during movie viewing (Demirtaş et al., 2019). Moreover, recent evidence has suggested that task-evoked FC modulation only accounts for a relatively small portion of individuals’ connectivity patterns during tasks, indicating that FC measured during tasks may still primarily reflect brains’ baseline functional architecture, i.e., the FC fingerprint (Gratton et al., 2018; Xie et al., 2018a). In another study comparing two neural patterns, Simony and colleagues demonstrated the greater sensitivity of ISFC than standard intra-subject FC directly measured during movie viewing in detecting stimulus-induced connectivity patterns (Simony et al., 2016).

Taken together, while the two neural coupling measures are clearly distinct from one another in some ways, there is a gap in our knowledge concerning whether the information encoded in these two connectivity patterns is complementary or redundant. To answer our question, we used an open-access movie-viewing fMRI dataset containing typically developing children and healthy adults and built a partial least square (PLS; Krishnan et al., 2011) regression model to predict children’s socio-cognitive abilities using intra- and/or inter-subject connectivity (the latter was computed using adults’ responses as the reference). Here, we focused on the social brain network (Alcalá-López et al., 2018), which undergoes rapid functional specialization throughout childhood while children advance in their social-cognitive abilities (Gweon et al., 2012; Gweon and Saxe, 2013; Richardson et al., 2018). We postulated that if both connectivity measures provided complementary information about individuals’ social-cognitive abilities, the joint model with both connectivity measures would offer the highest predictive performance. We also conducted a confirmatory analysis to verify previous studies’ observations on the similarity between ISFC and FC patterns (Simony et al., 2016; Kim et al., 2018; Lynch et al., 2018; Demirtaş et al., 2019) and carried out an additional exploratory analysis to assess the neurobiological significance of such similarity.

Materials and methods

Participants

An open-access dataset was used in this study¹, which contains a large sample of children (n = 122, 3.5–12 years old, 64 females, nine left-handed), and adults (n = 33, 18–39 years old, 20 females). No participants had any known cognitive or neural disorders. All adult participants and the parents/guardians of the child participants gave written consent. All protocols were approved by the Committee on the Use of Humans as Experimental Subjects at the Massachusetts Institute of Technology.

Movie task and behavioral battery

Participants underwent fMRI scans while watching a short silent version of “Partly Cloudy,” a 5.6-min animated movie with plots eliciting frequent inferences of characters’ mental states (beliefs, desires, emotions) and bodily sensations (particularly pain). The movie began after 10 s of a black screen and 10 s of opening credits. After fMRI scans, all children completed a socio-cognitive behavioral battery measuring theory of mind (ToM) abilities (Gweon et al., 2012). The custom-made 24-item ToM battery (available at https://osf.io/G5ZPV/) involved listening to an experimenter tell a story and answering prediction and explanation questions that required reasoning about characters’ mental states. The questions included varied aspects of ToM abilities including diverse desires and beliefs, false belief, emotion, moral judgment, and interpretation (Gweon et al., 2012). The ToM score used in the analysis corresponded to the proportion of questions answered correctly.

Image acquisition and preprocessing

Whole-brain structural and fMRI data were acquired on a 3-Tesla Siemens Tim Trio scanner at the Massachusetts Institute of Technology. All participants were scanned using the standard Siemens 32-channel head coil except for those under age five, who used custom 32-channel phased-array head coils made for younger children. T1-weighted structural images were collected in 176 interleaved sagittal slices with 1 mm isotropic voxels (Adults: FOV = 256 mm; children: FOV = 192 mm). Functional data were collected with a gradient-echo EPI sequence (#slices = 32; TR = 2 s, TE = 30 ms, flip angle = 90°). The details of scanning protocols can be found in Richardson et al. (2018).

We used the data preprocessed by the original authors (Richardson et al., 2018). Specifically, all functional images were first registered to the Montreal Neurological Institute (MNI) template, and registration of each individual’s brain to the MNI template was visually inspected. The registered data were then smoothed using a 5 mm Gaussian kernel. The Artifact Detection Tools² were used to detect timepoints with more than 2 mm framewise displacement (FD) to the previous time point or with a fluctuation in the global signal that exceeded a threshold of three standard deviations from the mean global signal. Additionally, we applied temporal interpolation on artifactual timepoints and regressed out the first five principal components using CompCor (Behzadi et al., 2007). The residual time courses were then simultaneously band-pass filtered between 0.008–0.15 Hz and detrended with the first- and second-order polynomials using 3dTproject. We also excluded the volumes corresponding to the opening credits (first 10 TRs). We adopted a stringent exclusion criterion and removed participants with a mean FD greater than 0.5 mm, leaving 90 children and 32 adults for further analysis. We performed scrubbing by removing TRs with FD greater than 1 mm.

Intra- and inter-subject functional connectivity

Following preprocessing, we extracted the ROI timeseries from the denoised data using the social brain atlas (Alcalá-López et al., 2018) containing 36 regions of interest (ROIs). The social brain atlas, developed using meta-analyses of social and affective abilities, consists of key regions responsible for social cognition, such as the amygdala, precuneus, medial prefrontal cortex (MPFC), and temporoparietal junction (TPJ), among many other regions responsible for social and affective information processing. The ROI masks were dilated by one voxel along each direction, and then masked by a binary group mask, resulting in 4 ROIs (bilateral temporal pole and cerebellum) being dropped due to little spatial coverage (fewer than five voxels). Next, we extracted the mean timeseries from each ROI mask. As shown in Figure 1, we computed the intra-subject FC by correlating the timeseries using Pearson correlation. For the inter-subject connectivity, Pearson correlation was computed across different individuals (i.e., child-to-adult) within a given ROI (ISC) and across two ROIs (ISFC). More specifically, we used the adults’ timeseries as a reference, and we correlated the children’s timeseries with those of the adults. We chose adults as the reference group, given the previous evidence showing stronger and more coherent neural coupling and activation patterns in adults than children (Cantlon and Li, 2013; Moraczewski et al., 2018). We averaged upper- and lower-diagonal ISFC values to obtain symmetric ISFC matrices. Both connectivity measures were then Fisher-transformed and normalized to have zero mean and unit variance. Since ISC corresponds to the diagonal terms of the ISFC matrix, hereinafter, we will use ISFC to refer to both ISC and ISFC.

FIGURE 1

Figure 1. A graphical depiction of intra-subject functional connectivity (FC) and inter-subject ISFC between two ROIs. S, task-evoked brain activity; I, intrinsic brain activity; N, noise. Left, FC is characterized as the neural coupling within a subject, and the RSFC is measured in the absence of the task-evoked signal (S). Right: ISFC measures brain synchronization across subjects. Only the neural coupling driven by the task-evoked brain activities is preserved, as the intrinsic brain activity and noise are uncorrelated across individuals. The ISC corresponds to the diagonal terms of ISFC. Figure adapted from Simony et al. (2016).

Building prediction model using partial least square regression

PLS regression is a statistical method that identifies a linear relationship between explanatory variables (X, connectivity) and response (Y, ToM scores). Briefly, PLS regression (Wold et al., 1983) seeks to optimize the covariance between the response (Y ∈ R^n×q) and explanatory variables (X ∈ R^n×p) by finding latent components T ∈ R^n×k for ill-posed problems (n < p, fewer samples than explanatory variables). PLS regression aims to find some lower-dimensional latent components T ∈ R^n×k, to be regressed on Y, which can be formulated as follows:

$X=T{P}^T+E,$

$Y=T{Q}^T+F,(1)$

where P ∈ R^p×k and Q ∈ R^q×k are loading matrices (in PLS regression, the loadings are not orthogonal), E and F are random errors following independent and identical normal distribution.

PLS regression is closely related to principal components regression, for which the latent components are simply derived by performing principal component analysis on X itself. By contrast, PLS regression uses both X and Y to construct latent components T as a linear transformation of X:

$T=XW,(2)$

where W ∈ R^p×q is a matrix of weights.

In our case of a univariate response (q=1), PLS regression iteratively maximizes the squared sample covariance between Y and the latent components under the condition that the latent components are mutually empirically uncorrelated and the columns of W are of unit length. Formally, to find the i^thcolumn of W, PLS regression solves the following optimization question:

$w_i=\underset{w}{\text{argmax}}{w}^T{X}^{T}Y{Y}^{T}Xw,$

$\mathrm{subject}\mathrm{to}{w}_i^T{w}_i=1\text{and}{t}_i^T{t}_j=w_i^T{X}^{T}X{w}_j=0,(3)$

where i = 1, …, k and j = 1, …, i − 1. Here, X and Y have mean-centered columns, and the maximum number of latent components is min(n − 1, p).

Once obtaining W, we can then plug it into the Equation 1 and solve Q using ordinary least squares:

$Q^T={(T^{T}T)}^{-1}{T}^{T}Y.(4)$

Finally, we can find the solution to the regression model $\widehat{Y}=T{Q}^T=XW{Q}^T=X{\beta }_{PLS}$ by combining Equations 1, 4:

${\widehat{\beta }}_{PLS}=W{Q}^T=W{(T^{T}T)}^{-1}{T}^{T}Y.(5)$

The beta weights ${\widehat{\beta }}_{PLS}\in R^{n\times K}$ are equivalent to the regression weights of multiple regression (although the problem at hand cannot be readily solved using OLS given n < p).

PLS regression is well-suited for questions where the dimensionality of independent variables (#connectivity) is much higher than the number of observations (#participants) and has been previously used in many associating brain patterns with behavior (Yoo et al., 2017; Ching Fong et al., 2018).

Here, we built three models, namely FC, ISFC, and a joint model (ISFC + FC), to predict the ToM scores after controlling for age (linear and quadratic effects), gender, and handedness. Ten-fold cross-validation was used to determine the optimal dimensionality of the latent components, and the cross-validation was repeated 100 times to minimize the influence of sampling variability. The model performance was evaluated by the mean absolute error (MAE) and Spearman correlation between the true and predicted ToM scores. To evaluate prediction performance for the chosen analyses, we ran 50,000 permutation tests to derive empirical null distributions of ToM prediction. We also conducted a partial correlation analysis to ensure that age and head motion did not drive our prediction.

Results

The joint model best predicted theory of mind scores

We compared the predictive performance of three models across a wide range of hyperparameters (#components = 1–10). As shown in Figures 2A,B, the optimal predictive performance, as reflected by the lowest mean absolute error (MAE) and highest correlation between the true and predicted ToM scores, was achieved with the joint model (ISFC + FC) with the component number equal to 2. To rule out the possibility that the increased feature dimensionality drove the improved predictive performance, we randomly sampled half of the FC and ISFC edges and re-evaluated the model performance 1,000 times. Despite the slight drop in predictive performance, the joint model using only half of the FC and ISFC features outperformed the models using ISFC and FC separately (paired t-test ps < 0.001). Furthermore, permutation results revealed that the optimal joint model significantly predicted ToM scores (mean rho = 0.34, p < 0.001), as shown in Figure 2C.

FIGURE 2

Figure 2. (A) Average mean absolute error (MAE) as a function of the number of latent components (#Comp) over 1,000 10-fold cross-validations. The shaded area indicates one standard deviation. (B) Spearman correlation (rho) between true and predicted ToM scores. (C) Null distribution of rho between true and predicted ToM scores with the vertical line indicating actual model performance.

Leave-one-region out analysis identifying key regions in the predictive model

We averaged 100 joint models with two components and visualized associated beta coefficients as shown in Figure 3A and showed identified latent components in Supplementary Figure 1. As is typical of a data-driven approach, regions with high beta values were diffusive across the social brain, with no single dominant anatomical pattern. To better delineate the key ROIs that contributed to the prediction accuracy, we conducted a leave-one-out (LOO) analysis by removing each ROI at a time and compared the predictive performance of the reduced model against the full joint model. As shown in Figure 3B, we found that excluding the anterior midcingulate cortex (aMCC), right middle temporal gyrus (rMTG), right TPJ (rTPJ), right anterior insula (rAI) and right supplementary motor area (rSMA), and left inferior frontal gyrus (lIFG) significantly lowered the predictive performance (Bonferroni corrected p < 0.05), suggesting the significance of these regions in our predictive model. We also conducted a follow-up LOO analysis excluding ISFC and FC edges separately. We found that excluding FC profiles of rMTG and lIFG as well as the ISFC profiles of rMTG, rSMA, and ventromedial prefrontal cortex (vMPFC) significantly lowered the model’s prediction performance (Bonferroni corrected p < 0.05).

FIGURE 3

Figure 3. (A) Visualization of beta coefficients $({\widehat{\mathrm{\beta }}}_{PLS})$ of the final joint model and the degree (row sum/column sum) of ISFC and FC patterns. The network assignment was color-coded based on the definition of Alcalá-López et al. (2018). Gray: high-level processing; orange: intermediate-level processing; purple: limbic; red: visual-sensory. (B) Key regions were identified in the final model by jointly removing FC and ISFC associated with a region (Bonferroni corrected p < 0.05), and the nodal size is proportional to the drop in predictive accuracy. For ROI abbreviations, see Supplementary Table 1.

On FC-ISFC similarity

Given the previous evidence showing high similarity between ISFC (excluding diagonal ISC) and intra-subject movie-viewing FC patterns (Kim et al., 2018; Demirtaş et al., 2019), we conducted a similar confirmation analysis by comparing the two connectivity patterns, which revealed a moderate positive correlation (mean rho = 0.34). Moreover, it should be noted that the FC-ISFC similarity is modulated by the diagonal ISC terms (for a detailed explanation, see Supplementary material), and higher ISC translates to more adult-like brain responses in children. We were interested in examining whether ISFC-FC similarity had any neurobiological significance, given the previous studies showing children with better task performance had more adult-like brain responses (Cantlon and Li, 2013; Cai et al., 2019). Thus, we carried out a follow-up exploratory analysis by correlating individuals’ ISFC-FC similarity with age and ToM abilities. We observed the similarity between ISFC and FC to be positively correlated with age (rho = 0.550, p < 0.001), while not correlated with ToM scores (rho = 0.12, p > 0.05).

Discussion

Previous research on the neural underpinnings of socio-cognitive information processing has typically relied on carefully constructed cognitive tasks and task-free paradigms (resting-state). However, these approaches are limited in that they have poor ecological validity. The emergence of naturalistic fMRI paradigms offers many theoretical and practical advantages, one of which is simultaneously examining intra- and inter-subject connectivity, namely FC and ISFC. However, no studies so far have examined the potential synergy of combining these two connectivity measures and whether they encode complementary information. Here, using the predictive performance as a proxy of features’ informativeness, we filled the knowledge gap by comparing the predictive efficacy of models using features of FC and/or ISFC to predict children’s social-cognitive abilities measured by ToM scores. Moreover, we also delineated key ROIs in our predictive model to better understand regions’ contribution to individual differences in social-cognitive abilities.

A synergy of intra- and inter-subject functional connectivity

By comparing the predictive efficacy of FC and ISFC, our results suggested that there may indeed be synergy between ISFC and FC to predict individual differences. As shown in Figure 2, the joint model using ISFC and FC not only significantly predicted participants’ ToM scores, but far exceeded individual models’ predictive accuracy. More importantly, the reduced joint model, which roughly matched the dimensionality of ISFC and FC, still outperformed individual models by a wider margin (ps < 0.001). We speculate that the superior performance of the model combining the connectivity measures may indicate that distinct cognitive factors may drive these two neural couplings. ISFC theoretically more precisely captures the shared connectivity patterns driven by the dynamic movie stimuli than standard task-FC (Simony et al., 2016), since the physiological and motion artifacts were uncorrelated across subjects. Moreover, empirical evidence has shown that the time-varying ISFC better tracked the movie cues than the time-varying FC (Bolton et al., 2020), and ISFC within the DMN exhibited reliable and distinct patterns during narrative processing (Simony et al., 2016), suggesting that ISFC may be a more cognitively relevant representation of movie-evoked connectivity patterns than the direct measurement of movie-FC. On the other hand, the intra-subject FC measured during movie viewing is a combination of task-evoked FC and individual FC fingerprint, with the FC fingerprint being the dominant one (Gratton et al., 2018). Thus, including FC measures in our model provides additional information about individuals’ FC fingerprints to facilitate the prediction. As a previous study has shown the improved performance of combining intra-subject FCs from multiple conditions for predictive modeling of phenotypic measures (Gao et al., 2019), we further demonstrated the synergy of intra- and inter-subject FC for predicting socio-cognitive abilities using a movie-viewing naturalistic paradigm.

To further elucidate the relationship between the two neural coupling measures, we examined the similarity of individuals’ ISFC (i.e., excluding the diagonal ISC terms) and FC patterns, which previous studies have focused on (Kim et al., 2018; Lynch et al., 2018; Demirtaş et al., 2019). Consistent with the prior work, we found that the two connectivity patterns were positively correlated (Kim et al., 2018; Demirtaş et al., 2019; c.f., Lynch et al., 2018). Both ISFC and FC patterns contained some degree of movie-evoked connectivity, which could have given rise to the overall positive ISFC-FC similarity. We also found that FC-ISFC similarity was significantly positively correlated with children’s age, suggesting more adult-like social brain responses in older children, consistent with the earlier observations (Richardson et al., 2018). Moreover, younger children could have less stable movie-evoked FC patterns as the underlying cognitive processes may be more variable and individualized, while stronger and more coherent movie-evoked FC patterns emerged in the older children, leading to a higher FC-ISFC similarity (Cantlon and Li, 2013; Moraczewski et al., 2018). Nevertheless, we noted that although FC-ISFC similarity was significantly correlated with age, there was no significant correlation between FC-ISFC and ToM scores, justifying the necessity of a more complex predictive model.

Highly predictive nodes concentrated in intermediate and high-level processing subnetworks

Our LOO analysis identified a few critical ROIs that significantly lowered the predictive accuracy in our final joint model. The majority of these key ROIs were found to be part of intermediate (i.e., rAI, aMCC, rSMA, and lIFG) and high-level processing subnetworks (i.e., rTPJ and rMTG), as defined by Alcalá-López et al. (2018). The high-level processing subnetwork consists of key regions closely associated with ToM, such as the TPJ, MTG, posterior cingulate cortex (PCC), and precuneus (Schurz et al., 2014). Per Alcalá-López and colleagues, this high-level subnetwork is more strongly connected within itself than any other subnetworks, and is most likely to be associated with the cognitive categories of social cognition in all subnetworks (Alcalá-López et al., 2018). The significance of the high-level processing network was confirmed when examining the latent components in the final model, which associates the connectivity profiles to ToM scores, as the second component was weighted heavily toward ROIs within the high-level processing subnetwork (Supplementary Figure 1). Indeed, the behavioral task included more higher-cognitive elements of ToM processing, such as false belief. Our LOO analysis also identified a few nodes from an intermediate-level subnetwork. The intermediate-level subnetwork, including the AI, aMCC, and IFG, among others, intertwines with the lower-level visual-sensory subnetwork that handles the preprocessed social-affective environmental inputs (Alcalá-López et al., 2018). AI is known for its role as a bridge between large-scale brain networks. Together with the aMCC, these two ROIs within the intermediate networks were involved in empathy and pain-related processing (Kurth et al., 2010; Lamm et al., 2011) and were also found among the ROIs that were highly predictive of ToM scores. Finally, the absence of lower-level visual-sensory ROIs among the most predictive regions suggests that although movie stimuli induce wide-spread connectivity changes in visual areas within and across individuals (Hasson et al., 2004; Lynch et al., 2018), such connectivity changes may be less socio-behaviorally relevant as compared to those in the high association cortices.

Limitations and future directions

Several limitations require further consideration. Firstly, we used adults as the reference group when computing ISFC. Our choice was justified by previous studies showing children with more variable inter-subject neural coupling and weaker activation patterns than adults across many cortical regions (Cantlon and Li, 2013; Moraczewski et al., 2018). Therefore, we believe the adults may be well-suited as a reference group given their higher homogeneity. Future investigations could focus on the impact of choosing different reference groups and better delineating developmental effects. A second limitation is that we chose a set of a priori ROIs using the social brain atlas since we are interested in predicting individuals’ social abilities. Future studies could examine a different parcellation, such as whole-brain parcellations (Power et al., 2011; Craddock et al., 2012; Shen et al., 2013), given the success of whole-brain FC predictive models (Shen et al., 2017; Beaty et al., 2018; Lake et al., 2019; Xiao et al., 2021). Alternatively, future studies could use atlases developed specifically for children since the brain parcelation used in the current study was derived from adult studies or conduct hyperalignment for better functional alignment (Haxby et al., 2020). Thirdly, given the limited sample size for studying individual differences in behavior (Marek et al., 2022), our results are subject to further validation. Future work should use consortium-level datasets and include different movie clips. Another potential remedy is using more advanced learning models to improve predictive performance, such as sparse group LASSO (Bai et al., 2020) and sparse tensor decomposition (Zhang et al., 2021). Fourth, we only studied static FC and ISFC, while future work could investigate whether dynamic FC (Xie et al., 2018b) and dynamic ISFC (Bolton et al., 2018) can be jointly studied to better characterize neural dynamics and individual differences. Finally, the current dataset utilized a ToM measure that was a composite of varied ToM abilities, which may account for the varied networks predictive of performance. While ToM is often discussed as a unitary construct, much theoretical and empirical work has demonstrated its multifaceted nature, including weak or no associations between measures (Schaafsma et al., 2015; Gernsbacher and Yergeau, 2019; Warnell and Redcay, 2019). Thus, the regions predictive of ToM should be interpreted within the context of this measure. Different measures may identify distinct patterns of predictive performance.

Conclusion

The current study investigated the potential benefits of jointly studying intra- and inter-subject connectivity, namely, FC and ISFC, respectively, and whether the two connectivity measures combined could enhance the prediction of individual differences in social cognition measured by a ToM questionnaire. Using PLS regression, we showed that the joint connectivity (ISFC + FC) model outperformed individual models even after matching the feature dimensionality. Our results suggest that intra- and inter-subject connectivity may encode unique and complementary information about the individuals’ social abilities, and we shall make full use of both connectivity measures to gain enriched insight into neural processes underlying naturalistic fMRI paradigms.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found in the article/Supplementary material.

Ethics statement

All protocols were approved by the Committee on the Use of Humans as Experimental Subjects at the Massachusetts Institute of Technology. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.

Author contributions

HX: responsible for study ideation, data analysis, manuscript writing, and manuscript reviewing. ER: responsible for manuscript writing, manuscript reviewing, and funding acquisition. Both authors contributed to the article and approved the submitted version.

Funding

This study was supported by the National Institute of Mental Health of the National Institutes of Health under Award Numbers R01MH107441 and R01MH125370.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Author disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins.2022.875828/full#supplementary-material

Footnotes

1. ^ https://www.openfmri.org/dataset/ds000228/
2. ^ https://www.nitrc.org/projects/artifact_detect/

References

Alcalá-López, D., Smallwood, J., Jefferies, E., Van Overwalle, F., Vogeley, K., Mars, R. B., et al. (2018). Computing the social brain connectome across systems and states. Cereb. Cortex 28, 2207–2232. doi: 10.1093/cercor/bhx121

PubMed Abstract | CrossRef Full Text | Google Scholar

Bai, Y., Gong, Y., Bai, J., Liu, J., Deng, H. W., Calhoun, V., et al. (2020). A joint analysis of multi-paradigm fMRI data with its application to cognitive study. IEEE Trans. Med. Imaging 40, 951–962. doi: 10.1109/TMI.2020.3042786

PubMed Abstract | CrossRef Full Text | Google Scholar

Beaty, R. E., Kenett, Y. N., Christensen, A. P., Rosenberg, M. D., Benedek, M., Chen, Q., et al. (2018). Robust prediction of individual creative ability from brain functional connectivity. Proc. Natl. Acad. Sci. U.S.A. 115, 1087–1092. doi: 10.1073/pnas.1713532115

PubMed Abstract | CrossRef Full Text | Google Scholar

Behzadi, Y., Restom, K., Liau, J., and Liu, T. T. (2007). A component based noise correction method (CompCor) for BOLD and perfusion based fMRI. Neuroimage 37, 90–101. doi: 10.1016/j.neuroimage.2007.04.042

CrossRef Full Text | Google Scholar

Bolton, T. A. W., Freitas, L. G. A., Jochaut, D., Giraud, A. L., and Van De Ville, D. (2020). Neural responses in autism during movie watching: Inter-individual response variability co-varies with symptomatology. Neuroimage 216:116571. doi: 10.1016/j.neuroimage.2020.116571

PubMed Abstract | CrossRef Full Text | Google Scholar

Bolton, T. A. W., Jochaut, D., Giraud, A. L., and Van De Ville, D. (2018). Brain dynamics in ASD during movie-watching show idiosyncratic functional integration and segregation. Hum. Brain Mapp. 39, 2391–2404. doi: 10.1002/hbm.24009

PubMed Abstract | CrossRef Full Text | Google Scholar

Cai, W., Duberg, K., Padmanabhan, A., Rehert, R., Bradley, T., Carrion, V., et al. (2019). Hyperdirect insula-basal-ganglia pathway and adult-like maturity of global brain responses predict inhibitory control in children. Nat. Commun. 10:4798. doi: 10.1038/s41467-019-12756-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Cantlon, J. F., and Li, R. (2013). Neural activity during natural viewing of sesame street statistically predicts test scores in early childhood. PLoS Biol. 11:e1001462. doi: 10.1371/journal.pbio.1001462

PubMed Abstract | CrossRef Full Text | Google Scholar

Ching Fong, A., Yoo, K., Rosenberg, M., Zhang, S., Chiang-Shan, R. L., Scheinost, D., et al. (2018). Dynamic functional connectivity during task performance and rest predicts individual differences in attention across studies. Neuroimage 188, 14–25. doi: 10.1016/J.NEUROIMAGE.2018.11.057

PubMed Abstract | CrossRef Full Text | Google Scholar

Cole, M. W., Bassett, D. S., Power, J. D., Braver, T. S., and Petersen, S. E. (2014). Intrinsic and task-evoked network architectures of the human brain. Neuron 83, 238–251. doi: 10.1016/j.neuron.2014.05.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Craddock, R. C., James, G. A., Holtzheimer, P. E., Hu, X. P., and Mayberg, H. S. (2012). A whole brain fMRI atlas generated via spatially constrained spectral clustering. Hum. Brain Mapp. 33, 1914–1928. doi: 10.1002/hbm.21333

PubMed Abstract | CrossRef Full Text | Google Scholar

Demirtaş, M., Ponce-Alvarez, A., Gilson, M., Hagmann, P., Mantini, D., Betti, V., et al. (2019). Distinct modes of functional connectivity induced by movie-watching. Neuroimage 184, 335–348. doi: 10.1016/j.neuroimage.2018.09.042

PubMed Abstract | CrossRef Full Text | Google Scholar

Finn, E. S., and Bandettini, P. A. (2021). Movie-watching outperforms rest for functional connectivity-based prediction of behavior. Neuroimage 235:117963. doi: 10.1016/j.neuroimage.2021.117963

PubMed Abstract | CrossRef Full Text | Google Scholar

Finn, E. S., Corlett, P. R., Chen, G., Bandettini, P. A., and Constable, R. T. (2018). Trait paranoia shapes inter-subject synchrony in brain activity during an ambiguous social narrative. Nat. Commun. 9:2043. doi: 10.1038/s41467-018-04387-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Finn, E. S., Glerean, E., Khojandi, A. Y., Nielson, D., Molfese, P. J., Handwerker, D. A., et al. (2020). Idiosynchrony : From shared responses to individual differences during naturalistic neuroimaging. Neuroimage 215:116828. doi: 10.1016/j.neuroimage.2020.116828

PubMed Abstract | CrossRef Full Text | Google Scholar

Fox, M. D., Snyder, A. Z., Vincent, J. L., Corbetta, M., Van Essen, D. C., and Raichle, M. E. (2005). The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proc. Natl. Acad. Sci. U.S.A. 102, 9673–9678. doi: 10.1073/pnas.0504136102

PubMed Abstract | CrossRef Full Text | Google Scholar

Gallagher, H. L., and Frith, C. D. (2003). Functional imaging of ‘theory of mind.’. Trends Cogn. Sci. 7, 77–83. doi: 10.1016/S1364-6613(02)00025-6

CrossRef Full Text | Google Scholar

Gao, S., Greene, A. S., Constable, R. T., and Scheinost, D. (2019). Combining multiple connectomes improves predictive modeling of phenotypic measures. Neuroimage 201:116038. doi: 10.1016/j.neuroimage.2019.116038

PubMed Abstract | CrossRef Full Text | Google Scholar

Gernsbacher, M. A., and Yergeau, M. (2019). Empirical failures of the claim that autistic people lack a theory of mind. Arch. Sci. Psychol. 7, 102–118. doi: 10.1037/arc0000067

PubMed Abstract | CrossRef Full Text | Google Scholar

Gobbini, M. I., Koralek, A. C., Bryan, R. E., Montgomery, K. J., and Haxby, J. V. (2007). Two takes on the social brain: A comparison of theory of mind tasks. J. Cogn. Neurosci. 19, 1803–1814. doi: 10.1162/jocn.2007.19.11.1803

PubMed Abstract | CrossRef Full Text | Google Scholar

Gratton, C., Laumann, T. O., Nielsen, A. N., Greene, D. J., Gordon, E. M., Gilmore, A. W., et al. (2018). Functional brain networks are dominated by stable group and individual factors, not cognitive or daily variation. Neuron 98, 439–452.e5. doi: 10.1016/j.neuron.2018.03.035

PubMed Abstract | CrossRef Full Text | Google Scholar

Gweon, H., Dodell-Feder, D., Bedny, M., and Saxe, R. (2012). Theory of mind performance in children correlates with functional specialization of a brain region for thinking about thoughts. Child Dev. 83, 1853–1868. doi: 10.1111/j.1467-8624.2012.01829.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Gweon, H., and Saxe, R. (2013). Developmental cognitive neuroscience of theory of mind. Amsterdam: Elsevier. doi: 10.1016/B978-0-12-397267-5.00057-1

CrossRef Full Text | Google Scholar

Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., and Malach, R. (2004). Intersubject synchronization of cortical activity during natural vision. Science 303, 1634–1640.

Google Scholar

Haxby, J. V., Guntupalli, J. S., Nastase, S. A., and Feilong, M. (2020). Hyperalignment: Modeling shared information encoded in idiosyncratic cortical topographies. eLife 9:e56601. doi: 10.7554/eLife.56601

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, D., Kay, K., Shulman, G. L., and Corbetta, M. (2018). A new modular brain organization of the bold signal during natural vision. Cereb. Cortex 28, 3065–3081. doi: 10.1093/cercor/bhx175

PubMed Abstract | CrossRef Full Text | Google Scholar

Krishnan, A., Williams, L. J., McIntosh, A. R., and Abdi, H. (2011). Partial Least Squares (PLS) methods for neuroimaging: A tutorial and review. Neuroimage 56, 455–475. doi: 10.1016/j.neuroimage.2010.07.034

PubMed Abstract | CrossRef Full Text | Google Scholar

Kurth, F., Zilles, K., Fox, P. T., Laird, A. R., and Eickhoff, S. B. (2010). A link between the systems: Functional differentiation and integration within the human insula revealed by meta-analysis. Brain Struct. Funct. 214, 519–534. doi: 10.1007/s00429-010-0255-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Lake, E. M. R., Finn, E. S., Noble, S. M., Vanderwal, T., Shen, X., Rosenberg, M. D., et al. (2019). The functional brain organization of an individual allows prediction of measures of social abilities transdiagnostically in autism and attention-deficit/hyperactivity disorder. Biol. Psychiatry 86, 315–326. doi: 10.1016/j.biopsych.2019.02.019

PubMed Abstract | CrossRef Full Text | Google Scholar

Lamm, C., Decety, J., and Singer, T. (2011). Meta-analytic evidence for common and distinct neural networks associated with directly experienced pain and empathy for pain. Neuroimage 54, 2492–2502. doi: 10.1016/j.neuroimage.2010.10.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Lynch, L. K., Lu, K. H., Wen, H., Zhang, Y., Saykin, A. J., and Liu, Z. (2018). Task-evoked functional connectivity does not explain functional connectivity differences between rest and task conditions. Hum. Brain Mapp. 39, 4939–4948. doi: 10.1002/hbm.24335

PubMed Abstract | CrossRef Full Text | Google Scholar

Marek, S., Tervo-Clemmens, B., Calabro, F. J., Montez, D. F., Kay, B. P., Hatoum, A. S., et al. (2022). Reproducible brain-wide association studies require thousands of individuals. Nature 603, 654–660. doi: 10.1038/s41586-022-04492-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Mitchell, J. P. (2008). Contributions of functional neuroimaging to the study of social cognition. Curr. Dir. Psychol. Sci. 17, 142–146. doi: 10.1111/j.1467-8721.2008.00564.x

CrossRef Full Text | Google Scholar

Moraczewski, D., Chen, G., and Redcay, E. (2018). Inter-subject synchrony as an index of functional specialization in early childhood. Sci. Rep. 8:2252. doi: 10.1038/s41598-018-20600-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Nguyen, M., Vanderwal, T., and Hasson, U. (2019). Shared understanding of narratives is correlated with shared neural responses. Neuroimage 184, 161–170. doi: 10.1016/j.neuroimage.2018.09.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Nummenmaa, L., Lahnakoski, J. M., and Glerean, E. (2018). Sharing the social world via intersubject neural synchronisation. Curr. Opin. Psychol. 24, 7–14. doi: 10.1016/j.copsyc.2018.02.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Power, J. D., Cohen, A. L., Nelson, S. M., Wig, G. S., Barnes, K. A., Church, J. A., et al. (2011). Functional network organization of the human brain. Neuron 72, 665–678. doi: 10.1016/j.neuron.2011.09.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Razi, A., Seghier, M. L., Zhou, Y., Mccolgan, P., Zeidman, P., Bottenhorn, K. L., et al. (2018). Cooperating yet distinct brain networks engaged during naturalistic paradigms: A meta-analysis of functional MRI results. Netw. Neurosci. 3, 27–48. doi: 10.1162/NETN

CrossRef Full Text | Google Scholar

Redcay, E., and Moraczewski, D. (2020). Social cognition in context: A naturalistic imaging approach. Neuroimage 216:116392. doi: 10.1016/j.neuroimage.2019.116392

PubMed Abstract | CrossRef Full Text | Google Scholar

Richardson, H. (2019). Development of brain networks for social functions: Confirmatory analyses in a large open source dataset. Dev. Cogn. Neurosci. 37:100598. doi: 10.1016/j.dcn.2018.11.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Richardson, H., Lisandrelli, G., Riobueno-Naylor, A., and Saxe, R. (2018). Development of the social brain from age three to twelve years. Nat. Commun. 9:1027. doi: 10.1038/s41467-018-03399-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Salmi, J., Metwaly, M., Tohka, J., Alho, K., Leppämäki, S., Tani, P., et al. (2020). ADHD desynchronizes brain activity during watching a distracted multi-talker conversation. Neuroimage 216:116352. doi: 10.1016/j.neuroimage.2019.116352

PubMed Abstract | CrossRef Full Text | Google Scholar

Salmi, J., Roine, U., Glerean, E., Lahnakoski, J., Nieminen-Von Wendt, T., Tani, P., et al. (2013). The brains of high functioning autistic individuals do not synchronize with those of others. Neuroimage Clin. 3, 489–497. doi: 10.1016/j.nicl.2013.10.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Schaafsma, S. M., Pfaff, D. W., Spunt, R. P., and Adolphs, R. (2015). Deconstructing and reconstructing theory of mind. Trends Cogn. Sci. 19, 65–72. doi: 10.1016/j.tics.2014.11.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Schurz, M., Radua, J., Aichhorn, M., Richlan, F., and Perner, J. (2014). Fractionating theory of mind: A meta-analysis of functional brain imaging studies. Neurosci. Biobehav. Rev. 42, 9–34. doi: 10.1016/j.neubiorev.2014.01.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Shen, X., Finn, E. S., Scheinost, D., Rosenberg, M. D., Chun, M. M., Papademetris, X., et al. (2017). Using connectome-based predictive modeling to predict individual behavior from brain connectivity. Nat. Protoc. 12, 506–518. doi: 10.1038/nprot.2016.178

PubMed Abstract | CrossRef Full Text | Google Scholar

Shen, X., Tokoglu, F., Papademetris, X., and Constable, R. T. (2013). Groupwise whole-brain parcellation from resting-state fMRI data for network node identification. Neuroimage 82, 403–415. doi: 10.1016/j.neuroimage.2013.05.081

PubMed Abstract | CrossRef Full Text | Google Scholar

Simony, E., Honey, C. J., Chen, J., Lositsky, O., Yeshurun, Y., Wiesel, A., et al. (2016). Dynamic reconfiguration of the default mode network during narrative comprehension. Nat. Commun. 7:12141. doi: 10.1038/ncomms12141

PubMed Abstract | CrossRef Full Text | Google Scholar

Sonkusare, S., Breakspear, M., and Guo, C. (2019). Naturalistic stimuli in neuroscience : Critically acclaimed. Trends Cogn. Sci. 23, 699–714. doi: 10.1016/j.tics.2019.05.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Vanderwal, T., Eilbott, J., Finn, E. S., Craddock, R. C., Turnbull, A., and Castellanos, F. X. (2017). Individual differences in functional connectivity during naturalistic viewing conditions. Neuroimage 157, 521–530. doi: 10.1016/j.neuroimage.2017.06.027

PubMed Abstract | CrossRef Full Text | Google Scholar

Warnell, K. R., and Redcay, E. (2019). Minimal coherence among varied theory of mind measures in childhood and adulthood. Cognition 191:103997. doi: 10.1016/j.cognition.2019.06.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Wold, S., Martens, H., and Wold, H. (1983). “The multivariate calibration problem in chemistry solved by the PLS method,” in Matrix pencils. Lecture notes in mathematics, Vol. 973, eds B. Kågström and A. Ruhe (Berlin: Springer), 286–293.

Google Scholar

Xiao, J., Chen, H., Shan, X., He, C., Li, Y., Guo, X., et al. (2021). Linked social–communication dimensions and connectivity in functional brain networks in autism spectrum disorder. Cereb. Cortex 31, 3899–3910. doi: 10.1093/cercor/bhab057

PubMed Abstract | CrossRef Full Text | Google Scholar

Xie, H., Calhoun, V. D., Gonzalez-Castillo, J., Damaraju, E., Miller, R., Bandettini, P. A., et al. (2018a). Whole-brain connectivity dynamics reflect both task-specific and individual-specific modulation: A multitask study. Neuroimage 180, 495–504. doi: 10.1016/j.neuroimage.2017.05.050

PubMed Abstract | CrossRef Full Text | Google Scholar

Xie, H., Gonzalez-Castillo, J., Handwerker, D. A., Bandettini, P. A., Calhoun, V. D., Chen, G., et al. (2018b). Time-varying whole-brain functional network connectivity coupled to task engagement. Netw. Neurosci. 3, 49–66. doi: 10.1162/netn_a_00051

CrossRef Full Text | Google Scholar

Xie, H., Howell, A., Schreier, M., Sheau, K., Manchanda, M., Ayub, R., et al. (2020). Finding the neural correlates of collaboration using a three-person fMRI hyperscanning paradigm. Proc. Natl. Acad. Sci. U.S.A. 117, 23066–23072. doi: 10.1101/782870

CrossRef Full Text | Google Scholar

Yoo, K., Rosenberg, M. D., Hsu, W.-T., Zhang, S., Li, C.-S. R., Scheinost, D., et al. (2017). Connectome-based predictive modeling of attention: Comparing different functional connectivity features and prediction methods across datasets. Neuroimage 167, 11–22. doi: 10.1016/j.neuroimage.2017.11.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, Y., Xiao, L., Zhang, G., Cai, B., Stephen, J. M., Wilson, T. W., et al. (2021). Multi-Paradigm fMRI fusion via sparse tensor decomposition in brain functional connectivity study. IEEE J. Biomed. Health Inform. 25, 1712–1723. doi: 10.1109/JBHI.2020.3019421

PubMed Abstract | CrossRef Full Text | Google Scholar