Toward Establishing Internal Validity for Correlated Gene Expression Measures in Imaging Genomics of Functional Networks: Why Distance Corrections and External Face Validity Alone Fall Short. Reply to “Distance Is Not Everything in Imaging Genomics of Functional Networks: Reply to a Commentary on Correlated Gene Expression Supports Synchronous Activity in Brain Networks”

The primary claim of the Richiardi et al. (2015) Science article is that a measure of correlated gene expression, significant strength fraction (SSF), is related to resting state fMRI (rsfMRI) networks. However, there is still debate about this claim and whether spatial proximity, in the form of contiguous clusters, accounts entirely, or only partially, for SSF (Pantazatos and Li, 2017; Richiardi et al., 2017). Here, 13 distributed networks were simulated by combining 34 contiguous clusters randomly placed throughout cortex, with resulting edge distance distributions similar to rsfMRI networks. Cluster size was modulated (6–15 mm radius) to test its influence on SSF false positive rate (SSF-FPR) among the simulated “noise” networks. The contribution of rsfMRI networks on SSF-FPR was examined by comparing simulated networks whose clusters were sampled from: (1) all 1,777 cortical tissue samples, (2) all samples, but with non-rsfMRI cluster centers, and (3) only 1,276 non-rsfMRI samples. Results show that SSF-FPR is influenced only by cluster size (r > 0.9, p < 0.001), not by rsfMRI samples. Simulations using 14 mm radius clusters most resembled rsfMRI networks. When thresholding at p < 10^–4, the SSF-FPR was 0.47. Genes that maximize SF have high global spatial autocorrelation. In conclusion, SSF is unrelated to rsfMRI networks. The main conclusion of Richiardi et al. (2015) is based on a finding that is ∼50% likely to be a false positive, not <0.01% as originally reported in the article (Richiardi et al., 2015). We discuss why distance corrections alone and external face validity are insufficient to establish a trustworthy relationship between correlated gene expression measures and rsfMRI networks, and propose more rigorous approaches to preclude common pitfalls in related studies.

Introduction

There is still active debate about whether brain regions comprising resting state fMRI (rsfMRI) networks exhibit uniquely high correlated gene expression or whether this effect is attributed entirely to spatial proximity in the form of contiguous clusters (Richiardi et al., 2015, 2017; Pantazatos and Li, 2017). The main claim and conclusion of the Richiardi et al. (2015) Science article is that high correlated brain gene expression [i.e., significant strength fraction (SSF)] can be at least partially explained by and is related to rsfMRI networks. Our 2017 commentary (Pantazatos and Li, 2017) found that randomly spaced clusters also generate SSFs much higher than expected by chance, indicating that SSF is not specific to rsfMRI networks. On the basis of this and other evidence presented in our 2017 commentary (Pantazatos and Li, 2017), we claimed that SSF is attributable entirely to spatial proximity, and that it is unrelated to rsfMRI networks. In their reply to our commentary, Richiardi et al. (2017) did not rebut this claim, other than to (rightly) point out that edges in these random clusters (median ∼ 24 mm) were shorter than rsfMRI networks (median ∼ 50 mm). That the SFs tend to be higher overall in these simulated networks (see Figure 1D in our 2017 commentary), which also tend to have shorter edges, is consistent with our argument that distance drives SSF. However, it is unclear, a priori, why lower average within-network (W) distances could account for the non-specificity of SSF. More rigorous and comprehensive network “noise” simulations are needed to determine the impact of spatial proximity (i.e., contiguous cluster size) vs. rsfMRI networks on SSF and whether SSF can be attributed entirely, or only partially, to spatial proximity.

Here, we generated randomly distributed networks so that they more accurately simulate rsfMRI networks. The parameters of the simulations were varied in order to examine the impact of contiguous cluster size on frequency of SSF among the simulated networks (i.e., false positive rate of SSF, or SSF-FPR). When using all samples for each network simulation, contiguous cluster centers have ∼20% chance of being a rsfMRI sample [throughout the text we refer to an “rsfMRI sample” as any of the 501 Allen Human Brain Atlas (AHBA) tissue samples that were defined in Richiardi et al. (2015) as belonging to one of the 13 resting state networks]. Note this is a rough estimate since the probability is 28% (501/1777) when the first cluster is formed and then drops slightly with subsequent clusters because they are not allowed to overlap with any previously formed clusters, see methods. Therefore, an additional variation of simulations was run which enforced non-rsfMRI cluster centers. To further the reduce the chances that simulated clusters overlapped with and included any rsfMRI samples, a third variation removed them entirely from the simulations. If rsfMRI networks are related to and contribute at all towards SSF, then the SSF-FPR should decrease with the latter two experiment variations.

In their reply to our commentary, Richiardi et al. claimed their results hold after linear distance correction (based on regression). We plotted tissue-tissue correlations vs. distance to show why this approach is insufficient to remove the effects of spatial proximity. Finally, we propose that the feature selection routine applied in the original 2015 article selects the genes that exhibit the highest global spatial autocorrelation, consistent with our argument that spatial proximity alone drives SSF. We tested this by examining and comparing measures of spatial autocorrelation for the 136 consensus features identified in the original 2015 article vs. all other genes.

Methods

A series of networks were simulated for varying sizes of contiguous clusters (i.e., 6–15 mm radius spheres) placed randomly throughout cortex. Cluster centers were comprised of cortical samples chosen at random from among the 1,777 included in the original Richiardi et al. Science article. We varied the cluster sizes from 6 to 15 mm spherical radius to systematically examine the effect of contiguous cluster size vs. SSF-FPR, and repeated this across three experiment types, in order to test and confirm our hypothesis that contiguous cluster size is the main (and only) predictor of significant SF. Thirty four clusters were first generated by randomly sampling the MNI locations of 1,777 cortical samples (or 1,276 for experiment type 3, described below). These locations defined the cluster centers, and neighboring AHBA samples within a specific spherical radius were grouped together to form a cluster. These clusters were then combined in the following manner in order to mimic the edge distance distributions of rsfMRI networks shown in Figure 2A top panel (which is essentially the same figure shown in the 2017 reply by Richiardi et al.): nine networks were generated by grouping together three clusters selected at random, three networks by grouping together two clusters at random, and one network comprised of a single cluster.

We also computed the median edge distances between all network nodes (AHBA samples) as well as between cluster centers for simulated and real networks. For the latter, the clusterdata function in MATLAB was used to group clusters in each of the 13 real rsfMRI networks when using the same number of clusters per network as used for the simulations (i.e., the nine largest networks were grouped into three clusters, the three next largest were grouped into two clusters, and the smallest network was grouped into one cluster.

For each of the 10 cluster sizes (i.e., radii varied from r = 6–15 mm), 1,000 distributed networks were simulated. For each of these 1,000 distributed networks, a “real” SF was calculated, and 1,000 “null” SFs were calculated by shuffling region labels 1,000 times as previously described in the original Richiardi et al. (2015) article. This resulted in 10 cluster sizes × 1,000 simulated networks × 1,000 shuffles = 10,000,000 total SF calculations (iterations) per experiment variation (described below). SFs were considered significant at p < 1/(# total shuffles), or 0.001 in this case. The simulations were repeated for each of the three experiment variations: (1) “All” – using all cortical samples, (2) “Z-cn” – using all cortical samples but enforcing non-rsfMRI cluster centers, and (3) “omit-RS”- using only non-rsfMRI samples. For experiment type 1 “All” (all cortical samples), cluster centers were randomly chosen from the 1,777 cortical MNI coordinates without any restrictions (other than the constraint that cluster spheres defined about each cluster center were not allowed to overlap). For experiment type 2 “Z-cn,” networks were defined as in type 1, but with the additional constraint that cluster centers were not allowed to be comprised of a rsfMRI sample (i.e., cluster centers were not allowed to include any of the 501 samples used to define rsfMRI regions in Richiardi et al., 2015). For experiment type 3 “omit-RS,” the clusters (both the centers and the neighboring samples comprising the clusters) were drawn only from the 1,276 non-rsfMRI samples. The total # of available samples for the above three experiment variations were 1,777, 1,777, and 1,276, respectively. The same procedures, including within tissue correction, etc., were applied as in the original 2015 article and in our 2017 commentary when calculating SF.

Moran’s I, a measure of spatial autocorrelation, was calculated for each of 16,906 genes’ expression levels. Moran’s I approaches +1 when the gene expression levels cluster perfectly over space (high spatial autocorrelation), and approaches -1 when the tested variable is evenly distributed (high dispersion). A measure of 0 indicates no relationship, meaning gene expression levels are randomly dispersed in space. Moran’s I was assessed with pysal v2.0 libraries (Rey and Anselin, 2010), using continuously diminishing weights over distance from each sample location up to 16 mm, beyond which all further weights were set to 0. Neither changing the 16 mm threshold to 64 mm nor using binary rather than continuous weighting substantively changed the results. The updated MATLAB code (network simulations and resulting plots and figures) and a Jupyter notebook and Python code (gene expression spatial autocorrelation analyses and resulting plots) are available at https://github.com/spiropan/ABA_functional_networks.

Results

A contiguous cluster size of 14 mm yielded within network sample size (W) closest to the rsfMRI sample size of 501 (Figure 1, top panel). The median # of W tissue samples for these simulated networks was 495 (interquartile range, IQR = 40). Across all simulated networks, the median distance among the 34 cluster centers (34^∗33/2 = 561 edges) was 84.6 mm (IQR = 2.7), indicating the clusters were widely distributed throughout cortex as expected and similar to real rsfMRI networks (median distance among cluster centroids = 80.6 mm). Note this metric is similar for all simulations since cluster size does not affect distances between the cluster centers. Varying contiguous cluster size did not significantly impact median within network (W) edge distances of the random networks, which all approximated the median edge distance of real rsfMRI networks (dashed line in Figure 1, middle panel). Contiguous cluster size almost perfectly predicted SSF-FPR (r-values > 0.98, p < 10E-6, Figure 1, bottom panel). The SSF-FPR rose from ∼0.05 at 6 mm radius to > 0.6 at 15 mm (in other words, about 50x to 600x the theoretical FPR at alpha level p = 0.001). Forcing cluster centers to lie outside rsfMRI areas or even removing rsfMRI samples all together did not impact the results In these simulations, the latter actually increased the SSF-FPR at most cluster sizes (Figure 1, bottom panel). An additional simulation at cluster size 14 mm, this time using 10,000 (vs. 1,000) shuffles for SF significant testing, yielded SSF-FPR = 0.47 (i.e., 4,700x higher than the theoretical false positive rate p < 10^–4 reported in the original 2015 Science article). Note our approach simulated 13 networks that roughly mimic the distribution of median distances of the rsfMRI networks shown in Figure 1 of the 2017 reply from Richiardi et al. (2017). Figure 2A here recapitulates that figure in boxplot form (top panel) and also shows an example set of simulated networks with cluster size = 14 mm in the current study (middle panel). The distribution of network sizes for simulated networks with cluster size 14 mm also resembled the distribution of rsfMRI network sizes (Figure 2A bottom panel) and the (ranked) network sizes for rsfMRI vs. simulated networks were tightly correlated (r = 0.97, p = 7.6E-8).

FIGURE 1

Figure 1. Plots of simulated network properties vs. contiguous cluster size. (Top panel) Median network sample size (W). Dashed line depicts the value for real rsfMRI networks (W = 501). (Middle panel) Median network edge distances (90% of the median distances across the 1,000 random networks fell within the depicted error bars). Dashed line depicts the value for real rsfMRI networks (∼50 mm). (Bottom panel) Contiguous cluster size, not rsfMRI samples, predicts percentage of SSF (SSF-FPR × 100) among the simulated “noise” networks (All: r = 0.98; Z-cn: r = 0.99; omit-RS: r = 0.99, p-values < 10E-6). All = experiment in which all 1,777 AHBA cortical samples were used; Z-cn = same as “All,” except cluster centers were forced to lie outside of rsfMRI network areas; omit-RS = only non-rsfMRI samples (1,276) were used to simulate both network and non-network regions.

FIGURE 2

Figure 2. (A) Boxplot of edge distances (y-axis) for resting state fMRI networks as defined in Richiardi et al. (2015) (top panel), and an example set of simulated networks in the current study (middle panel). The network sizes (sorted in descending order by # of AHBA tissue samples) are plotted for rsfMRI networks (black line) and simulated networks (dashed blue line, cluster size = 14 mm radius, error bars show the range for 90% of the values across the 1,000 simulations, bottom panel). (B) A plot of distance (8 mm bins, X-axis) vs. median brain gene expression tissue-tissue correlation (y-axis) along with best fit line. The plot used all edges used to calculate SF after applying within-tissue correction. (C) Consensus features that maximize high strength fraction (identified in the original 2015 article) are also more likely to cluster together in space and exhibit high global spatial autocorrelation. Top panel: Histograms and kernel density plots of Moran’s I values for each of 16,906 genes, 136 high SF genes in red, all others in gray. Bottom panel: The y-axis is relative to each sample, with different n, so heights are not comparable, but position along the x-axis is. Bottom panel: A box plot of the same Moran’s I values.

A control analysis similar to the “All” condition tested whether total # W samples (not contiguous cluster size) predicted SSF. To reduce computation time, only 200 networks were simulated using 200 shuffles for SF significance testing. The contiguous cluster size was held constant at 6 mm, while scaling factors from [1, 1.5,2…5] modulated the number of contiguous clusters comprising the networks. While this varied the median # W nodes from ∼100 to 450 among the simulated networks (median edge distances ∼57 mm), the SSF-FPR decreased slightly from 0.12 to 0.085 (r = -0.2, p = 0.6, data not shown), confirming that contiguous cluster size, and not total # W nodes, predicts SSF.

We plotted median tissue-tissue correlations (after applying within-tissue correction as in the original 2015 article) vs. distance to confirm a non-linear relationship (Figure 2B). Next, we tested whether the 136 consensus features identified in the original 2015 article exhibit high global spatial autocorrelation. The relationship between the magnitude of each gene’s expression and its spatial clustering was assessed using Moran’s I, a measure of spatial autocorrelation. Moran’s I was significantly higher in the 136 genes Richiardi deemed significant (mean I = 0.430, sd 0.142) than in remaining genes (mean I = 0.084, sd 0.100) (p < 10^–9 for each of the 136 genes; see Figure 2C).

Discussion

The main finding and claim of the original 2015 article, reflected in the title, is “that functional brain networks defined with resting-state functional magnetic resonance imaging can be recapitulated by using measures of correlated gene expression in a postmortem brain tissue data set.” The main text states “the spatial organization of functional networks corresponded to regions that have more highly correlated gene expression than expected by chance (P < 10^–4)” and that “this finding cannot emerge from spatial proximity or gross tissue similarity.” This is the most critical, specific claim under contention.

Our network simulations indicate that the spatial organization of rsfMRI networks do not correspond to regions that have more highly correlated gene expression than expected by chance. The probability of observing highly correlated gene expression for rsfMRI networks by chance is about 50%, not <0.01% as reported in the original article. Furthermore, contiguous cluster size, and not rsfMRI networks, accounts entirely for SSF. The SSF-FPR was not affected by enforcing non-rsfMRI cluster centers or removing rsfMRI samples entirely from the simulations. Taken together, these results indicate that the evidence presented in the original 2015 article is insufficient to establish a relationship between correlated brain gene expression and rsfMRI networks. In other words, SSF is not an internally valid measure. It is a highly reliable measure that is driven by artifact (spatial proximity in the form of contiguous clusters) rather than a meaningful (i.e., valid) relationship with rsfMRI networks (Zuo et al., 2019).

Why Distance Corrections Alone Are Not Enough to Establish Links Between Correlated Gene Expression and rsfMRI Networks

In their reply to our 2017 commentary, Richiardi et al. rightly point out that Euclidean distance correction will “wrongly assign ‘nearness’ to two ‘neurally distant’ regions on the crowns of adjacent gyri” (see figure in their reply). This is one reason why correcting for Euclidean distance alone is not adequate and why other types of control experiments are required to validate measures of correlated gene expression such as SSF. In their reply, Richiardi et al. also argue that their original results hold after distance corrections using linear regression and distance-aware permutation. Note that a non-linear correction for spatial proximity (Fulcher and Fornito, 2016) would have more effectively corrected for distance. However, even if the authors’ SF measure were to survive significance testing with a more effective distance correction, it would still be necessary to show that the significantly high SF is specific to rsfMRI networks. Below we discuss why parametric distance corrections alone are not enough to validate the SSF measure.

Our 2017 commentary discusses why Euclidean distance (linear regression) is an inadequate method for proximity correction due to strong model assumptions (i.e., the relationship between distance and tissue correlation is not linear, as evidenced by a plot of median tissue-correlations vs. distance, Figure 2B). The authors suggested an intrinsic contradiction in our recommendation to avoid distance correction using linear regression and then also showing a linear correlation in our original 2017 commentary Figure 1B. There was no contradiction here, because a best fit line was added only to show the first order approximate fit between distance and correlated gene expression, not to suggest linear regression be used for distance correction. A primary purpose of Figure 1B was to show that the within-network (Wi) edges (dark gray) are substantially shorter than the out of network edges (T-Wi, light gray). Our critique was also founded on Figure 1A (removing within tissue samples inadequately corrects for spatial proximity), Figure 1C (SF falls monotonically as progressively longer edges are removed) and Figure 2D (SSF is not specific to rsfMRI). In hindsight, we could have tried to overlay the curve (shown in Figure 1 here) to our 2017 commentary Figure 1B, but it would have been less straightforward to attach a p-value and the model fit might not have been readily apparent.

The distance-aware permutation testing appears to be a more valid approach than the original (non-distance) aware permutation testing. However, the authors’ results (higher p-values with more conservative distance aware corrections) illustrate and support the fact that SF is driven by distance, but in a different way that parallels our 2017 commentary Figure 1C. Whereas our 2017 commentary Figure 1C showed that “real” SF decreases as short edges are removed from the rsfMRI networks, the author’s results show that the null distribution SFs increase as long edges are removed from the “null” networks. Both scenarios will create higher p-values in SF significance testing.

Critically, the fact that the authors’ new analyses survive distance corrections and distance aware permutations, and at higher p-values than < 10^–4 as originally reported in the 2015 article, does not validate the SSF measure, since it is unrelated to rsfMRI networks to begin with. In other words, these same distance corrections and distance aware permutations would show similar results (i.e., higher SF p-values but still <0.01 or 0.05) when applied to any of the ∼50% of randomly spaced networks that started out with SF p < 10^–4.

Why Face Validity in Independent Datasets Is Not Enough to Establish a Link Between Correlated Gene Expression and rsfMRI Networks

In response to our 2017 commentary, Richiardi et al. stressed their replications (more appropriately called face validity) in independent datasets and note that we did not generate gene lists for any of our random cluster analyses and examine them in other independent datasets. There are several problems with this line of reasoning and argumentation. First, the authors’ response appears to be “moving the goalpost¹.” The authors did not rebut our 2017 specific claim that SSF is both unrelated and not specific to rsfMRI networks, other than to rightly note that the median distances of our simulated “noise” networks were half as long as rsfMRI networks. To rebut our claim, the authors would have needed to show that simulated networks with longer median distances fail to generate inflated SSF-FPR, but they did not.

Secondly, we did not claim that the set of 136 genes identified in the study were not important to functional connectivity. However, at best, the additional findings that the authors mention in their reply (i.e., in mouse connectivity and rsfMRI connectivity using their identified set of 136 genes), suggest face validity for, but not evidence that validates, their primary claim, which is that correlated gene expression is higher in and uniquely related to rsfMRI networks.

It is plausible that, even if SSF is non-specific and completely unrelated to rsfMRI networks when using all genes, the 136 consensus features identified in the original 2015 article are unique to rsfMRI networks and uniquely important to functional connectivity. To demonstrate this would require additional control experiments and comparisons. One possibility is that the consensus feature selection routine used in the original 2015 article converges on genes that tend to be more highly expressed in the largest contiguous clusters of any network. In the case of rsfMRI, these include the posterior cingulate and ventromedial PFC of the dDMN, which are major hub regions and highly connected to the rest of the brain, which could help explain the evidence for supporting connectivity in independent datasets. Critically, the independent replication (more appropriately called external face validity) noted by Richiardi et al. in their reply², used all genes as a baseline group when examining the 136 consensus features identified in the original 2015 article. A more rigorous baseline group would have been all genes with properties similar to the 136 genes (i.e., mean expression in the brain, variability and global spatial autocorrelation) in order the confirm that these generic properties alone do not account for observed relationships with connectivity.

It is also possible that the identified consensus features are not unique to rsfMRI networks. This interpretation is supported by our results that found the 2015 consensus features have high global spatial autocorrelation throughout cortex (Figure 2C). It is also possible that the 136 consensus features are relatively unique to rsfMRI networks, but that consensus features identified in random networks are also important to functional connectivity and demonstrate face validity. We did not identify consensus genes for simulated networks since it was outside the scope of the current work. Based on our conclusion that spatial proximity alone drives significant SF, and the observation that the 136 consensus genes have high global spatial autocorrelation, we would hypothesize that consensus genes for simulated networks will overlap substantially with the 136 reported consensus genes. Further work is required to determine which of the above scenarios is true. Any of the above scenarios, however, is independent of the primary claim of the original 2015 article.

Why Network Simulations Constitute an Appropriate Null Model for Testing the Claim of Higher SF in rsfMRI Networks

In the 2015 Science article, the authors claimed that “Grouping gene expression samples according to functional networks yielded a higher strength fraction than that of other groupings of samples; the spatial organization of functional networks corresponded to regions that have more highly correlated gene expression than expected by chance (P < 10^–4)” and that “this finding cannot emerge from spatial proximity or gross tissue similarity.” Evidence for this claim requires demonstrating higher correlated gene expression (SF) for rsfMRI networks compared to alternative groupings (or spatial organizations) of tissue samples. However, the authors did not attempt to do this. Instead, they used a null model in which individual samples were shuffled to generate new null networks before recomputing the SF.

This null model is invalid because it assumes tissue samples are independent and equally exchangeable with other samples, which is not true given the distance effect and spatial autocorrelation in gene expression. In other words, this approach destroys the most important driver of SF in their null models: large, spatially contiguous clusters. This is why the p-value (p < 10^–4) reported in the original article is so highly inflated. The distance aware shuffling that the authors applied in their 2017 reply mitigated this to certain extent, but not completely, since shuffling individual samples, even if they are closer together, will still tend to “break up” spatially contiguous clusters. A valid null model would instead consist of shuffling whole clusters, rather than individual samples, in order to keep the spatially contiguous structure of the networks intact. However, this is not straightforward as it would first require defining clusters of similar sizes comprised of non-rsfMRI sample to be shuffled with the rsfMRI clusters.

A next best approach, which we employ here, is to generate a null model by simulating “noise” networks comprised of contiguous clusters randomly spaced throughout cortex. This approach constitutes a valid null model for testing the claim that the spatial organization of rsfMRI networks have higher SF than expected by chance. In addition, it allowed us to rigorously quantify the impact of contiguous cluster size, as well as rsfMRI samples, on SSF FPR.

Suggestions for Future Work

When dealing with high-dimensional data, it is difficult to identify valid relationships in light of the fact that any set of features can be identified when maximizing some variable or measure, even with “noise” data. It is also important to first ensure that the measure and its interpretation is internally valid, before seeking external validations. Here, network “noise” simulations were used to test the internal validity of the SSF measure and its interpretation that it is related to rsfMRI networks. Future studies of imaging genomics and functional networks that use measures of correlated gene expression could combine non-linear or non-parametric distance correction, distance-aware permutation as well as split-half bootstrap resampling to rigorously test the validity of the measures and their interpretation and also the consistency of the features (genes) that optimize the measures.

Data Availability Statement

The datasets and code generated for this study can be found at https://human.brain-map.org and https://github.com/spiropan/ABA_functional_networks.

Author Contributions

SP conducted network simulation analyses and drafted the manuscript. MS conducted spatial autocorrelation analyses and drafted relevant portions of the text.

Funding

This work was supported by a K01MH108721 (SP).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank Paul Pavlidis for helpful comments and suggestions.

Footnotes

1. ^ Moving the goalposts is an informal [logical] fallacy in which evidence presented in response to a specific claim is dismissed and some other (often greater) evidence is demanded” (https://en.wikipedia.org/wiki/Moving_the_goalposts).
2. ^ https://arxiv.org/abs/1706.06088

References

Fulcher, B. D., and Fornito, A. (2016). A transcriptional signature of hub connectivity in the mouse connectome. Proc. Nat.l. Acad. Sci. U. S. Am 113, 1435–1440. doi: 10.1073/pnas.1513302113

PubMed Abstract | CrossRef Full Text | Google Scholar

Pantazatos, S. P., and Li, X. (2017). Commentary: BRAIN NETWORKS. Correlated gene expression supports synchronous activity in brain networks.science348, 1241-4. Front. Neurosci. 11:412. doi: 10.3389/fnins.2017.00412

PubMed Abstract | CrossRef Full Text | Google Scholar

Rey, S. J., and Anselin, L. (2010). “PySAL: a python library of spatial analytical methods,” in Handbook of Applied Spatial Analysis: Software Tools, Methods and Applications, eds M. M. Fischer and A. Getis (Berlin: Springer), 175–193.

Google Scholar

Richiardi, J., Altmann, A., and Greicius, M. (2017). Distance is not everything in imaging genomics of functional networks: reply to a commentary on Correlated gene expression supports synchronous activity in brain networks. bioRxiv [Preprint]. doi: 10.1101/132746

CrossRef Full Text | Google Scholar

Richiardi, J., Altmann, A., Milazzo, A. C., Chang, C., Chakravarty, M. M., Banaschewski, T., et al. (2015). BRAIN NETWORKS. Correlated gene expression supports synchronous activity in brain networks. Science 348, 1241–1244. doi: 10.1126/science.1255905

PubMed Abstract | CrossRef Full Text | Google Scholar

Zuo, X.-N., Xu, T., and Milham, M. P. (2019). Harnessing reliability for neuroscience research. Nat. Hum. Behav. 3, 768–771. doi: 10.1038/s41562-019-0655-x

PubMed Abstract | CrossRef Full Text | Google Scholar