Sex Differences in the Patterns and Predictors of Cognitive Function in HIV

Despite advancements in antiretroviral therapy, mild cognitive deficits persist in nearly half of people with HIV (PWH). The profile of impairment in HIV is highly variable with deficits observed in a range of cognitive domains. Despite evidence of greater cognitive impairment among women with HIV (WWH) vs. men with HIV (MWH), it is unclear how MWH and WWH differ in the type of cognitive impairment and in risk factors associated with cognitive impairment profiles. In a large and well-characterized sample of PWH, we used machine learning to identify profiles of cognitive functioning and their associated factors overall and within sex. Participants included 1,666 PWH (201 WWH; 1,465 MMH) from the HIV Neurobehavioral Research Program who completed a neuropsychological test battery at their baseline visits. Using demographically-adjusted T-scores from 13 test outcomes assessing motor skills, executive functioning, attention/working memory, episodic learning and memory, verbal fluency, and processing speed, we used Kohonen self-organizing maps to identify patterns of high-dimensional data by mapping participants to similar nodes based on T-scores (MCLUST R package). Random forest models were used to determine how sociodemographic (e.g., age, education), clinical (e.g., depressive symptoms, substance use disorder), and biological (e.g., HIV disease characteristics) factors differentially related to membership within a cognitive profile. All analyses were repeated within sex. Three cognitive profiles were identified overall and within each sex. Overall and within MWH, there were unimpaired and global weakness profiles. The third profile in the total sample demonstrated relatively weak auditory attention whereas in MWH showed relative strengths in attention and processing speed. Conversely, there was no unimpaired profile among WWH. Rather, WWH demonstrated separate profiles reflecting weakness in motor skills, a relative weakness in learning and delayed recall, and global weaknesses with spared recognition memory. Despite different cognitive profiles by sex, the most discriminative factors were similar between men and women and included reading level (cognitive reserve), current and nadir CD4 count, plasma HIV viral load, duration of HIV disease, age, depressive symptoms, and race/ethnicity. Findings fill a knowledge gap concerning sex differences in cognitive impairment in PWH and inform personalized risk reduction and therapeutic strategies.

Introduction

The Human Immunodeficiency Virus (HIV) enters the central nervous system (CNS) within days of initial infection (1), in many cases leading to neurological, cognitive, and behavioral complications. Cognitive deficits are a common feature of HIV/AIDS. While the incidence of HIV-associated dementia has considerably decreased in the era of modern ART suppressing viral replication, mild cognitive deficits with no change in everyday function persist in 24% [95% confidence interval (CI) = 20.3–26.8] of people with HIV (PWH) and mild cognitive deficits with mildly decreased everyday function persist in about 13.3% (95% CI = 10.6–16.3) of PWH (2). Although executive function and memory deficits are most common in PWH in the post-ART era, the characterization of cognitive impairment in HIV is highly variable with deficits observed in a range of cognitive domains (3). Previous studies using statistical clustering techniques have identified differing profiles of cognitive function among PWH with some profiles resembling global impairment across domains while other profiles resemble more domain-specific impairment, particularly in the domains of episodic memory and executive function (4–7). Similarly, there is also substantial variability in the risk factors associated with cognitive deficits among PWH that range from biological (e.g., CD4⁺ T-cell count, HIV viral load, comorbid health conditions), demographic (e.g., age, sex, race/ethnicity) to psychosocial factors (e.g., low education, depression, substance use/dependence). The persistence of cognitive impairment in the era of modern ART among PWH and the variability in the profiles and risk factors associated with cognitive impairment suggests that non-HIV factors associated with aging, comorbid conditions (e.g., cardiovascular disease) and psychosocial risk factors (e.g., poverty, poor education) likely contribute to cognitive impairment given the high prevalence of these factors among PWH (8, 9). With this in mind, we propose looking beyond the construct of HIV-associated neurocognitive disorders (HAND) to identify the underlying pathophysiology linked to cognitive impairment as HAND requires other comorbidities to be ruled out as primary contributing factors.

Biological sex is an important determinant of cognitive impairment among PWH. In a recent literature review of sex differences in cognitive impairment among PWH (10), seven cross-sectional (11–17) and one longitudinal analysis (18) identified sex differences on global measures of cognitive impairment among PWH. Additionally, six cross-sectional (13–15, 17, 19, 20) and one longitudinal analysis (21) also reported sex differences in domain-specific cognitive performance. The strongest available evidence of adequately-powered studies indicates that WWH show greater deficits than MWH in the domains of learning and memory followed by speed of information processing and motor functioning, with inconsistent findings in executive functioning (17, 21).

The greater vulnerability of WWH to cognitive impairment may reflect sociodemographic differences between men and women with HIV. WWH tend to have a higher prevalence of psychosocial risk factors including poverty, low literacy levels, low educational attainment, substance abuse, poor mental health, and barriers to health care services (10, 22) as compared to MWH. These psychosocial risk factors may have biological effects on the brain that lead to reduced cognitive reserve among WWH (23, 24) as evidenced by findings of greater susceptibility of cognitive function to the effects of mental health factors (e.g., depression) among WWH vs. MWH (25). Additionally, biological factors such as sex steroid hormones (e.g., estrogen, testosterone) and female-specific hormonal milieus (e.g., pregnancy, menstrual cycle, menopause transition) may contribute to sex differences in cognitive test performance in PWH. However, it remains unclear how MWH and WWH may differ in the patterns of cognitive impairment and risk factors associated with these patterns of cognitive impairment. Previous reports of impairment profiles among PWH have identified them in combined samples of men and women (4–7), masking possible sex-specific patterns of cognitive impairment among PWH. Furthermore, although a number of studies reported sex differences in the presence and pattern of cognitive impairment (14, 16, 17) and greater cognitive decline compared to MWH (18), only one study (17) was adequately powered to address meaningful sex difference in global cognitive function (10). A well-powered examination of the patterns and determinants of cognitive impairment by sex, that also controls for other demographic differences between WWH and MWH (e.g., age, education, race/ethnicity), can help to clarify the contribution of sex to heterogeneity in cognitive impairment among PWH. Such an examination could also clarify the related psychosocial vs. biological factors and, thereby, optimize risk assessments and intervention strategies in both sexes.

Leveraging comprehensive neuropsychological (NP) data from the large-scale cohort of the HIV Neurobehavioral Research Program (HNRP) at the University of California-San Diego, we used novel machine learning methods to identify differing profiles of cognitive function in PWH and to evaluate how these profiles differ between women and men in sex-stratified analyses. Rather, than using traditional cognitive domain scores, we used each of the NP test outcomes given that prior studies indicate that the correlation of NP test scores does not map to traditional domain scores in PWH. Furthermore, we determined how sociodemographic (e.g., age, education, race/ethnicity), clinical (e.g., functional status, depression, substance use disorders) and biological (e.g., measures of HIV disease severity, ART use, cardiovascular comorbid conditions, Hepatitis C co-infection) factors related to cognitive profiles within women and men. Based on previous studies among PWH (4–6), we hypothesized that the machine learning approach would identify distinct subgroups of individuals with normal cognitive function, global cognitive impairment, and domain-specific cognitive impairment. We further hypothesized that groups with domain-specific cognitive impairment would differ by sex, with WWH showing more consistent memory and processing speed impairment than MWH. Finally, we expected that similar sociodemographic/clinical/biological determinants would distinguish cognitive profiles (e.g., age, education, race, HIV viral load) among WWH and MWH; however, in line with previous research (25), we expected that depressive symptoms would be more strongly associated with cognitive impairment profiles among WWH than MWH.

Materials and Methods

Participants

Participants included 1,666 PWH (201 WWH; 1,465 MWH) enrolled in various NIH-funded research studies at the University of California, San Diego's HNRP, https://hnrp.hivresearch.ucsd.edu/. Study assessment details have been published elsewhere (3). The UCSD Institutional Review Board approved the studies. Participants provided written informed consent and were compensated for their participation. Exclusion criteria for the parent studies included history of non-HIV-related neurological, medical, or psychiatric disorders that affect brain functioning (e.g., seizure, stroke, psychosis), learning disabilities, and a first language that was not English. Inclusion in the current study required completion of neuropsychological and neuromedical evaluations at the baseline study visit. Exclusion criteria for the current study included a positive urine toxicology test for illicit drugs (excluding marijuana) or Breathalyzer test for alcohol on the day of clinic visit on the day of study visit.

NP Test Evaluation

NP test performance was assessed through a comprehensive, standardized, battery of tests that measure seven domains of cognition, including complex motor skills, executive function, attention/working memory, episodic learning, episodic memory (delayed recall and recognition), verbal fluency, and information processing speed. Motor skills were assessed by the Grooved Pegboard (GPEG) Dominant and Non-dominant Hand tests (26). Executive functioning was assessed by the Trail Making Test (TMT)-Part B (27) and the Stroop Color and Word Test interference score (28). Attention/working memory was assessed by the Paced Auditory Serial Addition Task (PASAT-50) (29, 30). Episodic learning was assessed by the Total Learning scores of the Hopkins Verbal Learning Test-Revised (HVLT-R) (31) and the Brief Visuospatial Memory Test-Revised (BVMT-R) (32). Episodic memory was assessed by the Delayed Recall and Recognition scores of the HVLT-R and BVMT-R. Verbal Fluency was assessed by the “FAS” Letter Fluency test (33). Information processing speed was assessed by the WAIS-III Digit Symbol Test (34), the TMT-Part A, and the Stroop Color and Word Test color naming score. Raw test scores were transformed into age-, education-, sex-, and race/ethnicity-adjusted T-scores based on normative samples of HIV-uninfected persons (35, 36). The use of demographically-adjusted T-scores are intended to control for these demographic effects as they occur in the general population.

Factors Associated With NP Profiles

We examined sociodemographic, clinical, and biological factors associated with cognitive impairment in the literature and available with enough participants to be adequately powered in analyses. Sociodemographic factors included age, years of education, and race/ethnicity. Although these factors were used to create the T-scores, there can still be remaining demographic associations with cognition within clinical populations such as PWH. For example, there is considerable interest in the possibility of abnormal cognitive aging PWH; also, in general, older PWH tend to have had their infections longer, may have had longer periods without benefit of suppressive ART, and more history of worse immunosuppression. Clinical factors included functional status as indicated by the number of daily activities with decreased independence from the Instrumental Activities of Daily Living questionnaire (IADL) from the modified version of the Lawton and Brody Activities of Daily Living Questionnaire (37), reading level (a proxy for cognitive reserve) based on the Wide Range Achievement Test-4 Reading subtest (WRAT-4 Reading) (38), self-reported depressive symptoms on the Beck Depression Inventory versions I (BDI-I) or II (BDI-II) (39), and diagnosis of lifetime and current major depressive disorder (MDD) as well as lifetime alcohol, cannabis, or other (i.e., amphetamine, cocaine, hallucinogen, inhalant, sedative, opioid, and PCP) substance use disorder based on the Composite International Diagnostic Interview using DSM–IV criteria (CIDI version 2.1) (40). Biological factors included HIV disease variables such as current CD4⁺ T-cell count, lowest CD4⁺ T-cell count ever recorded (nadir CD4), plasma HIV viral load, estimated duration of HIV disease, current use of ART, current use of anticholinergic-based medications (e.g., urinary incontinence and chronic obstructive pulmonary disease medications), Hepatitis C co-infection, and the cardiovascular comorbid conditions of hypertension, hyperlipidemia, and diabetes.

Statistical Analyses

All 13 NP tests were used to find groups of similar cognitive profiles within each participant subset (MWH, WWH) and in the total sample using a pipeline that consisted of dimension reduction with Kohonen self-organizing maps (SOM) followed by clustering to identify profiles based on those reduced dimensions. SOM was implemented using the Kohonen package in R (41). SOM is an unsupervised machine learning technique used to identify patterns in high-dimensional data (numerous variables) by producing a two-dimensional representation consisting of multiple nodes where each node is a group of one or more individuals with similar cognitive profiles and the location of the nodes within the 2-D representation is also a metric of similarity. Unlike probabilistic models, each individual can only be assigned to one node. The SOM grid consisted of a 10 × 10 hexagonal grid of nodes and the number of clusters for the final profiling was selected by looping over models created from 3 to 20 clusters and selecting the number that had the best fit based on entropy. Similar nodes were then clustered (grouped together) using the MClust package (42). MClust is an R Software package used for model-based clustering using finite normal mixture modeling that provides functions for parameter estimation via the Expectation-Maximization algorithm with an assortment of covariance structures which vary in distribution (spherical, diagonal, or ellipsoidal), volumes (equal or variable), shape (equal of variable), and orientation (equal or variable, only for ellipsoidal distribution). This program identifies the best model based on entropy (a model fit statistic). Once the clustering of the nodes was completed, cluster profiles were assigned to the individuals associated with that node. By using SOM and MClust in sequence, we were able to achieve fine-tuned clustering based on patterns of performance in cognitive testing.

Factors predicting profile membership between each impaired and unimpaired profile in the overall sample and within each group (MWH, WWH) were explored by creating a predictive Random Forest (RF) model using the Caret (43) package in R and then extracting variable importance (44). RF is an ensemble machine learning model based on classification trees that results in powerful prediction models based on non-linear combinations of subsets of input variables. Prior to model creation, the Synthetic Minority Over-sampling Technique (SMOTE) with the DMwR (45) package was used to control for bias due to any imbalance in the number of cases. RF models were created using internal validation using a 10-fold resampling method repeated 5 times. Pre-processing before RF creation involved removing variables as predictors if they had low variance or if they had >50% missing data. Any missing data in the remaining variables was imputed using the Multivariate Imputation by Chained Equations (46) (MICE) package in R using random forest imputations. ROC confidence intervals were calculated using the pROC package in R with 2,000 stratified bootstrap replicates (95% CI). Variable importance of all variables included in the RF models was used as the outcome metric of the predictive power of each variable. Variable importance is a scaled number [0–100] that indicates how important that variable is to the final predicted outcome in that model. For each tree in the RF model, the out-of-bag portion of the data is recorded and repeated after permuting each predictor variable. The difference between the accuracy with and without each variable is averaged over all trees and then normalized by the standard error. For visualization, all variables were plotted by relative variable importance, and attention was given to the top 10 variables in each profile. Variable importance indicates how much that variable contributes to overall prediction accuracy, but as RF is non-linear model it does not indicate directionality.

While the analysis pipeline and packages used along with the parameter inputs are stated above, we have added our code into a Supplementary Material to facilitate rigor and reproducibility.

Results

Participants

Table 1 provides sociodemographic, behavioral, and clinical factors for 1,666 PWH (1,465 men; 201 women). On average, participants were 41.8 years of age [standard deviation (SD) = 9.8] with 13.3 years of education (SD = 2.7). Fifty-eight percent were White and 18% Black. Mental health comorbidities were common. Forty-eight percent had a lifetime, and 19% had a current, diagnosis of MDD. With respect to HIV-related clinical characteristics, 60% were on combination ART and 42% were virally suppressed. Compared to MWH, WWH were less educated, had lower WRAT-4 scores, were less likely to be white, and had a shorter duration of HIV disease (P's < 0.05). Additionally, WWH reported more IADL dependence and were more likely to have HCV co-infection compared to MWH (P's < 0.05).

TABLE 1

Table 1. Demographic, behavioral, and clinical characteristics in the total sample of people with HIV and by sex.

Table 2 provides NP test performance for the total sample and by sex. In the total sample, average performance on BVMT-R delayed recall, HVLT-R (total learning, delayed recall, recognition), GPEG-non-dominant, and PASAT had T-scores <45 or 0.5 standard deviations from the general population mean (T-score of 50). WWH performed worse than MWH on BVMT-R total learning on average (P = 0.01). However, WWH performed better on the recognition measures of the BVMT-R and HVLT-R (P's < 0.01). This was also the case examining percent impairment using a T-score cutoff of 40 (P's < 0.001).

TABLE 2

Table 2. Neuropsychological test performance in the total sample of people with HIV and by sex.

Identification of Cognitive Profiles in the Total Sample

Profiles where the mean T-score on all cognitive outcomes was >45 and <55 were considered an “unimpaired average” profile. To describe the profiles, tests where the average T-scores of all participants in that cluster were <45 were considered weaknesses, and those where the average was <40 were considered impaired. An average >55 was considered a relative strength in the context of other domains being in the average range (>45 and <55).

Profiling of the 1,666 PWH resulted in three total groups using an ellipsoidal multivariate mixture model with equal orientation with an entropy of 0.982 (Figure 1A).

- Profile 1 (n = 618): Unimpaired indicated by the average T-score for all NP outcomes falling into the normal/average range between 45 and 55.

- Profile 2 (n = 461): Relatively weak auditory attention and episodic memory indicated by weaknesses in HVLT-R (learning and delayed recall), Letter Fluency, and PASAT.

- Profile 3 (n = 587): Global weaknesses indicated by average T-scores in the impaired range on all BVMT-R outcomes, HVLT-R learning and delayed recall, GPEG-non-dominant, and PASAT as well as weaknesses on TMT-Part B, Letter Fluency, GPEG-dominant, and Digit Symbol.

FIGURE 1

Figure 1. Profiling results in the (A) Total sample, (B) Men with HIV, and (C) Women with HIV. The small dotted line indicates a half of a standard deviation above and below the mean whereas the large dotted line indicates a full standard deviation above and below the mean. BVMT, Brief Visuospatial Memory Test-Revised; HVLT, Hopkins Verbal Learning Test Revised; PASAT, Paced Auditory Serial Addition Task; TMT, Trail Making Test; GPEG, Grooved pegboard.

Figure 2A provides the percent impairment on each task within each of the profiles. Supplementary Table 1 also provides T-scores and percent impairment on each task within each of the profiles for reference.

FIGURE 2

Figure 2. Percent impairment (T-score cutoff of 40) for each profile in the (A) Total sample, (B) Men with HIV, and (C) Women with HIV. BVMT, Brief Visuospatial Memory Test-Revised; HVLT, Hopkins Verbal Learning Test Revised; PASAT, Paced Auditory Serial Addition Task; TMT, Trail Making Test; GPEG, Grooved pegboard.

Identification of Cognitive Profiles in MWH and WWH Separately

Profiling of the 1,465 MWH also resulted in three groups using an ellipsoidal multivariate mixture model with equal orientation with an entropy of 0.993 (Figure 1B).

- Profile 1 (n = 753): Unimpaired indicated by the average T-score for all NP outcomes falling into the normal range between 45 and 55.

- Profile 2 (n = 286): Unimpaired with relative strength in attention and processing speed indicated by relative strengths (T-scores above 55) on TMT-Part A&B and Digit Symbol compared to Profile 1. Similar to Profile 1, all other NP outcomes fell into the normal range between 45 and 55.

- Profile 3 (n = 426): Global weaknesses indicated by impairment on all BVMT-R, HVLT-R and GPEG outcomes and weaknesses on TMT-Part A&B, Letter Fluency, and Digit Symbol.

Profiling of the 201 WWH also resulted in three groups using an ellipsoidal multivariate mixture model with equal orientation with an entropy of 0.989 (Figure 1C).

- Profile 1 (n = 64): Weakness in motor function indicated by the average T-score falling into the normal/average range (>45) or above average (>55) on all tests except for GPEG.

- Profile 2 (n = 67): Relative weaknesses in learning and memory indicated by weaknesses on learning and delayed recall on the BVMT-R and HVLT-R with the average T-scores for the other outcomes falling in the normal range (>45).

- Profile 3 (n = 70): Global weaknesses with spared verbal recognition indicated by average T-scores in the impaired range on learning and delayed recall on the BVMT-R and HVLT-R, GPEG, PASAT, and Digit Symbol, and weaknesses on BVMT-R recognition, TMT, and Letter Fluency. Notably, average recognition on the HVLT-R was in the normal range (M = 47.1, SD = 14.5).

Figure 2B provides the percent impairment on each task within each of the profiles within MWH and Figure 2C in WWH. Supplementary Tables 2, 3 also provide T-scores and percent impairment on each task within each of the profiles for reference.

Predictors of Cognitive Profiles in the Total Sample

In RF models, the top 10 variables distinguishing each of the impaired profiles—Relatively weak auditory attention and episodic memory [Profile 2; receiver operating curve (ROC) = 0.94] and Global weaknesses (Profile 3; ROC = 0.95)—from the unimpaired profile (Profile 1) were the same and included: WRAT-4, age, duration of HIV disease, nadir CD4 counts, education, BDI-II, IADL dependence, log plasma viral load, and race/ethnicity (Figure 3A). In each case, the impaired profiles or those with weaknesses (2 and 3) had lower WRAT-4 and higher BDI-II scores than the unimpaired profile (Profile 1) (Table 3). The Relatively weak auditory attention and episodic memory profile (Profile 2) group also was less educated, more likely to be Hispanic, and had a shorter duration of HIV disease compared to the unimpaired profile (Profile 1). However, the Global Weaknesses profile (Profile 3) was older, had more IADL dependence, a longer duration of HIV disease, and lower current and nadir CD4 counts as compared to the unimpaired profile.

FIGURE 3

Figure 3. Random forest variable importance from the models for (A) Total sample, (B) Men with HIV, and (C) Women with HIV. IADL, Instrumental Activities of Daily Living; WRAT, Wide Range Achievement Test-4 Reading subtest; BDI, Beck Depression Inventory; LT, lifetime; HCV, Hepatitis C co-infectious; MDD, major depressive disorder.

TABLE 3

Table 3. Demographic, behavioral, and clinical characteristics in the total sample of people with HIV by cognitive profile.

Predictors of Cognitive Profiles in MWH and WWH Separately

MWH

In RF models in MWH, the top 10 variables distinguishing the Global weaknesses profile (Profile 3) and Relative strength in attention and processing speed (Profile 2) from the unimpaired profile (Profile 1) (ROC = 0.95 and ROC = 0.92, respectively) were the same and included: IADL dependence, WRAT-4, BDI-II, nadir CD4 count, duration of HIV disease, current CD4 count, education, age, log plasma viral load, and race (Figure 3B). The Global weaknesses profile (Profile 3) was older, had higher IADL dependence, and BDI-II scores, longer duration of HIV disease, and lower current and nadir CD4 counts compared to the unimpaired profile (Profile 1) (Table 4). The Relative strength in attention and processing speed (Profile 2) was more likely to be White, had higher WRAT-4 and lower BDI-II scores, had higher current and nadir CD4 counts, and lower log plasma viral loads than the unimpaired profile (Profile 1).

TABLE 4

Table 4. Demographic, behavioral, and clinical characteristics in the sample of men with HIV by cognitive profile.

WWH

In RF models in WWH, nine out of the top 10 variables distinguishing the Relative weaknesses in learning and memory profile (Profile 2) and Global weakness with spared verbal recognition (Profile 3) from the profile only demonstrating Weakness in motor function (Profile 1; ROC = 0.95 and ROC = 0.90, respectively) were the same and included: WRAT-4, age, log plasma viral load, BDI-II, current and nadir CD4 count, duration of HIV infection, IADL dependence, and education (Figure 3C). The only unique variable distinguishing the profile demonstrating Relative weaknesses in learning and memory (Profile 2) from the profile demonstrating Weakness in motor function (Profile 1) was race. However, the unique variable distinguishing the profile demonstrating Global weaknesses with spared verbal recognition (Profile 3) from the profile demonstrating Weakness in motor function (Profile 1) was a current diagnosis of MDD. The Relative weaknesses in learning and memory profile (Profile 2) was older, less likely to be Hispanic, more likely to be non-Hispanic White, and had lower BDI-II scores compared the profile demonstrating Weakness in motor function (Profile 1) (Table 5). However, the Global weaknesses with spared verbal recognition profile (Profile 3) was more likely to be non-Hispanic White, have lower WRAT-4 scores, had more IADL dependence and higher BDI-II scores, was more likely to have a current diagnosis of MDD, have higher log plasma viral loads, lower current and nadir CD4 counts, and a shorter duration of HIV disease compared to the profile demonstrating Weakness in motor function (Profile 1).

TABLE 5

Table 5. Demographic, behavioral, and clinical characteristics in the sample of women with HIV by cognitive profile.

Discussion

In this large-scale study using a novel pipeline combination of machine learning methods, we provide further evidence in support of heterogeneity in cognitive function among PWH. Our results do not negate the heterogeneity in cognitive function in HIV-uninfected individuals but rather highlights the heterogeneity among PWH that can often be masked by a dichotomous HAND categorization. In the total sample, we identified an unimpaired profile, a profile of relatively weak auditory attention and episodic memory, and a global weakness profile. As expected, given the relative sample sizes, the cognitive patterns in the total sample were in greater alignment with those found among MWH compared to WWH. Similar to results in the overall sample, we identified an unimpaired profile and a global weakness profile in MWH; however, unlike the overall sample and inconsistent with hypotheses of domain-specific cognitive impairment profiles in both MWH and WWH, MWH demonstrated a profile with relative strengths in attention and processing speed. Conversely, there were no unimpaired, cognitive strength or global weakness profiles among WWH. Rather, as hypothesized WWH demonstrated cognitive profiles reflecting a global weakness (with spared verbal recognition) and domain-specific impairment including a weakness in learning and memory and motor skills. These findings suggest that sex and the sociodemographic factors associated with female sex within the HIV-infected population contribute to the heterogeneity in cognitive function among PWH. Studies examining cognitive function in combined samples of men and women may mask important sex differences in cognitive functioning among PWH, particularly in male-dominant samples such as the current sample. These sex differences in cognitive profiles among PWH may result from biological sex differences and/or the psychosocial factors that tend to characterize WWH more than MWH (e.g., low education, poverty). Biological sex differences include those seen in the general population such as sex steroid hormones (e.g., estrogen, progesterone, testosterone), female-specific reproductive events (e.g., parity, reproductive span, hormone therapies) and genetic factors or previously-reported sex differences specifically in HIV disease characteristics unmeasured herein (e.g., size of viral reservoirs, CD4 cell count at seroconversion) (47, 48). Regardless of the underlying mechanism, characterizing these sex differences in cognitive functioning among PWH can provide inroads to identifying mechanisms of cognitive dysfunction and optimizing risk assessments and diagnostic and therapeutic strategies for each sex.

A notable sex difference in profiles was the lack of the unimpaired or cognitive strength profile among WWH that was observed among MWH. Our cognitive profile analyses are in line with prior studies that suggests that WWH are often but not always, more likely to demonstrate cognitive deficits than MWH (10). Our analysis suggests that the impairment manifests more often as domain-specific impairment (i.e., learning, memory, motor) in women than in men that may not be revealed in a more cross-domain summary measure like GDS or global T-scores. This female vulnerability to cognitive deficits is thought to reflect sociodemographic differences whereby low education and socioeconomic status and their associated psychosocial risk factors (e.g., depression, poverty, early-life trauma, barriers to health care, co-infections) are more prevalent among WWH vs. MWH (10, 22, 49). These psychosocial risk factors can have adverse effects on the brain that lower cognitive reserve (23, 24, 50, 51), suggesting that interventions geared toward addressing these psychosocial factors should be a priority for WWH and/or for women who are at increased risk of HIV. In support of these studies, Sundermann et al. (17) found that the higher rates of cognitive impairment in WWH vs. MWH were eliminated after adjusting for the lower reading level (i.e., WRAT-4 score) that characterized WWH compared to MWH. Biological differences may also contribute to sex differences in the pattern and magnitude of cognitive impairment in PWH including disease characteristics, brain structure/function, sex steroid hormones and female-specific hormonal milieus (e.g., pregnancy, menstrual cycle, menopause transition). There is also evidence to suggest that WWH may be more cognitively susceptible than MWH to the effects of mental health factors (25).

As mentioned, only women demonstrated more domain-specific cognitive profiles including weakness in motor functioning and relative weakness in learning and memory. Similarly, previous studies report that learning, memory, and motor functioning are among the domains in which cognitive impairment is more common among WWH vs. MWH (10) and these differences persisted after adjusting for HIV RNA and CD4 counts (21). These sex differences in domain-specific impairment may reflect psychosocial factors (e.g., cognitive reserve, mental health), biological factors (sex steroid hormones, genetic), or interactions among them. Although women in general demonstrate relative advantages in verbal memory and fine motor function compared men (52–57) likely due, at-least in part, to the effects of estrogen on the developing brain and the neuroprotective effects of circulating estradiol (58–60), the menopause transition has been associated with declines in verbal memory and motor function (61–63). The mean age of women in our study was 41 (SD = 9.6; 33% >45 years of age) suggesting that a portion of women may be experiencing cognitive deficits associated with reproductive aging. Germane to the learning/memory impairment in WWH, women are more vulnerable to the negative effects of stress hormones on hippocampal-dependent tests compared to men (64). This finding may be particularly relevant to the current sample considering the high prevalence of psychosocial stressors among WWH including childhood trauma and domestic violence (65).

Unlike MWH, WWH demonstrated a global impairment profile with spared verbal recognition. Consistently, previous findings regarding memory impairment among PWH found this impairment to be more dependent on frontal and subcortical structures with relatively normal memory retention but impaired memory retrieval (recall but not recognition deficits) (66–68). Even in the female-specific profile of relative weakness in learning and memory, recognition was less impaired compared to learning and recall. We can only speculate as to why the sparing of recognition in the global impairment profile was specific to WWH and to verbal vs. visual memory. It is possible that, in the context of cognitive impairment in HIV, the female advantage in verbal memory may be most salient for the least cognitively-taxing memory component, recognition performance, and this advantage is not fully adjusted for in our demographically-corrected T-scores.

Despite the heterogeneity in cognitive profiles by sex, the sociodemographic/clinical/biological factors associated with these cognitive profiles were similar for MWH and WWH suggesting that, although the same factors confer increased vulnerability to cognitive dysfunction, the adverse effects of these factors impact brain function differently in men and women. In both MWH and WWH, WRAT-4 had the greatest discriminative value of profile class followed by HIV disease variables (e.g., CD4 count, viral load and estimated duration of HIV disease), depressive symptoms, age, race/ethnicity and years of education. WRAT-4 scores have been consistently identified as an important determinant of cognitive function among PWH, with lower WRAT-4 scores conferring risk for cognitive impairment (17, 69). WRAT-4 performance may be particularly salient in this population, given that reading level may reflect education quality, above and beyond years of education, especially in lower socioeconomic populations because of the many factors impacting education quality (e.g., ability to attend school, economic disadvantages in schools within low SES districts) (69). Additionally, reading level is associated with health outcomes including hospitalizations and outpatient doctor visits (70) and, thus, may be a proxy for biopsychosocial factors underlying general health (e.g., socioeconomic status, self-efficacy).

HIV disease variables were also strong determinants of cognitive profiles in both men and women. Aside from some instances of a shorter duration of HIV disease relating to more cognitive impairment in WWH and in the total sample, the more biologically-based HIV disease variables were associated with cognitive impairment in the expected direction; higher current and nadir CD4 count and lower viral load were protective against cognitive impairment. It is curious that the global weakness with spared verbal recognition profile in women was associated with more severe HIV-related variables (i.e., higher viral loads, lower current, and nadir CD4 counts) yet with shorter duration of HIV infection. We speculate that the shorter HIV infection in WWH may reflect CNS effects of untreated and/or early-course HIV infection. Alternatively, the self-reported shorter duration of infection may not have been accurate, to the extent that WWH lived longer with untested/undetected infections. Findings are consistent with a wealth of literature relating proxies of HIV disease burden and severity to cognitive function (71–73) and suggests that, even in the era of effective ART when viral suppression is common, HIV disease burden can have adverse effects on the brain possibly due to poor penetration of ARTs into the CNS, ART resistance, poor medication adherence (74), and/or the establishment of viral reservoirs in the CNS reservoir (75, 76).

In line with hypotheses of mental health factors relating to cognitive impairment profiles more strongly in women, current diagnosis of MDD was a predictor of cognitive profiles only among WWH. Although the prevalence of a current or lifetime diagnosis of MDD did not differ between WWH and MWH, MDD was an important risk factor of demonstrating Global weaknesses with spared verbal recognition (Profile 2) compared to the profile demonstrating only Weakness in motor function (Profile 1). This finding aligns with our work demonstrating that MDD may have a greater impact in women compared to men (25). Our work indicates that HIV comorbid with depression affects certain cognitive domains including cognitive control, and that these effects are largest in women. Specifically, WWH with elevated depressive symptoms had 5 times the odds of impairment on Stroop Trial 3, a measure of behavioral inhibition, compared to HIV-uninfected depressed women, and 3 times the odds of impairment on that test compared to depressed MWH. In a recent meta-analysis, small to moderate deficits in declarative memory and cognitive control were documented not only in individuals with current MDD but also in individuals with remitted MDD, leading to the conclusion that these deficits occur independently of episodes of low mood in individuals with “active” MDD (77). Together these lines of work suggest that MDD would exacerbate (or co-occur with factors that cause) cognitive difficulties in PWH, particularly in the cognitive domains of declarative memory and cognitive control in WWH.

Our study has limitations. Although we were adequately powered within both WWH and MWH (10), the magnitude of power was discrepant by sex considering that women represented 20% of our sample. Larger-scale studies in WWH only are currently underway. The generalizability of our findings also warrant additional study as the profiles identified here may not represent the profiles among all PWH. Due to the unavailability of data, we were unable to explore certain psychosocial factors (e.g., early life trauma, perceived stress) as potential determinants of cognitive profiles. Our analyses were cross-sectional which allows us to identify determinants associated with cognitive profiles but precludes us from determining the temporal relationships between these factors and cognitive function. Although many of the related factors may be risk factors for cognitive impairment, reverse causality is possible with some of the factors resulting from cognitive impairment (e.g., depression, IADL). Additionally, interpretation of the machine learning results should be done with care as RF is an ensemble model that is inherently non-linear in nature. This means that the importance and predictive power of every variable is specified in the context of other variables. This can lead to situations where an important predictive variable in the RF model has no significant difference in the overall comparison but has dramatic differences when included with other variables in the model. As such, this model should be interpreted as hypothesis-generating and identifies variables in need of further investigation. Lastly, because our study was focused on sex differences in cognitive profiles within PWH, we did not include a HIV-seronegative comparison group. Thus, we cannot determine the degree to which HIV contributes to sex differences in cognitive profiles. However, the independent HIV-related predictors does suggest that HIV has a role. Despite these limitations, we selected RF over linear models such as lasso and ridge regression because RF models had more predictive power and higher accuracy in this data compared to the linear models, even linear models with tuning parameters such as ridge and lasso that can used for feature selection. The results from these models mirror the P-values for the univariate comparisons (see Tables 1–5), which is expected since analysis of variance and t-tests are also linear models. Moreover, RF models are more optimal for handling missing data, the inclusion of categorical predictor variables, and the use of categorical outcome measures which was the case in the present study. RF models also account for the complexity in the data that can arise from multicollinearity often seen in large feature sets.

In conclusion, our results also suggest that sex is a contributor to the heterogeneity in cognitive profiles among PWH and that cognitive findings from MWH or male-dominant samples cannot be wholly generalized to WWH. Whereas, MWH showed an unimpaired profile and even a cognitively advantageous profile, WWH only showed impairment profiles that included global and more domain-specific impairment, which supports previous findings of greater cognitive impairment in WWH than in MWH (10). Although the strongest determinants of cognitive profiles were similar in MWH and WWH including WRAT-4, HIV disease characteristics, age and depressive symptoms, the direction of these associations sometimes differed. This suggests that the effects of certain biological, clinical, or demographic factors on the brain and cognition may manifest differently in MWH and WWH and that sex may contribute to heterogeneity not only in cognitive profiles but in their determinants although studies with larger numbers of WWH are needed to more definitively test these hypotheses. It is important to detect these differing cognitive profiles and their associated risk/protective factors as this information can help to identify differing mechanisms contributing to cognitive impairment and whether these mechanisms are related to HIV disease, neurotoxic effects of ART medications, and/or comorbidities that are highly prevalent among PWH (e.g., depression, substance abuse, hyperlipidemia). Given the longer lifespan of PWH in the era of effective antiretroviral therapy, cognitive profiling will also inform aging-related effects on cognition in the context of HIV and perhaps early clinical indicators of age-related neurodegenerative disease. By identifying cognitive profiles and their underlying mechanisms, we can ultimately improve our ability to treat by tailoring and directing intervention strategies to those most likely to benefit. Overall, our results stress the importance of considering sex differences in studies of the pathogenesis, clinical presentation, and treatment of cognitive dysfunction in HIV.

Data Availability Statement

The data analyzed in this study is subject to the following licenses/restrictions: data from our study are available upon request. Persons with HIV are highly stigmatized, and even in the presence of strong de-identification, the risk of re-identification is real. Protecting persons with HIV remains our top priority. The Data Access Committee whom imposed these restrictions and to whom data requests should be made is the HNRP Data Resource Committee. Requests to access these datasets should be directed to hnrpresource@ucsd.edu.

Ethics Statement

The studies involving human participants were reviewed and approved by the Institutional Review Board of the University of California, San Diego. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

ES aggregated the data. LR, RD, and EP conducted statistical analyses. LR and ES have primary responsibility for final content and wrote the paper. All authors contributed to the project conception, design, manuscript editing, statistical review, and read and approved the final manuscript.

Funding

This research was funded by the Johns Hopkins University NIMH Center for novel therapeutics for HIV-associated cognitive disorders (P30MH075773) and by NIH RF1AG061070. The HIV Neurobehavioral Research Center (HNRC) was supported by Center award P30MH062512 from NIMH.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor is currently organizing a Research Topic with one of the authors ES.

Acknowledgments

The San Diego HIV Neurobehavioral Research Center (HNRC) group is affiliated with the University of California, San Diego, the Naval Hospital, San Diego, and the Veterans Affairs San Diego Healthcare System, and includes: Director: RH, Co-Director: IG; Associate Directors: J. Hampton Atkinson, M.D., RE, and Scott Letendre, M.D.; Center Manager: Jennifer Iudicello, Ph.D.; Donald Franklin, Jr.; Melanie Sherman; NeuroAssessment Core: RE, Scott Letendre, M.D., Thomas D. Marcotte, Ph.D., Christine Fennema-Notestine, Ph.D., Debra Rosario, M.P.H., Matthew Dawson; NeuroBiology Core: Cristian Achim, M.D., Ph.D. (P.I.), Ana Sanchez, Ph.D., Adam Fields, Ph.D.; NeuroGerm Core: Sara Gianella Weibel, M.D. (P.I.), David M. Smith, M.D., Rob Knight, Ph.D., Scott Peterson, Ph.D.; Developmental Core: Scott Letendre, M.D. (P.I.), J. Allen McCutchan; Participant Accrual and Retention Unit: J. Hampton Atkinson, M.D. (P.I.) Susan Little, M.D., Jennifer Marquie-Beck, M.P.H.; Data Management and Information Systems Unit: Lucila Ohno-Machado, Ph.D. (P.I.), Clint Cushman; Statistics Unit: Ian Abramson, Ph.D. (P.I.), Florin Vaida, Ph.D. (Co-P.I.), Anya Umlauf, M.S., Bin Tang, M.S. The views expressed in this article are those of the authors and do not reflect the official policy or position of the Department of the Navy, Department of Defense, nor the United States Government. We would like to thank all the participants for their continued involvement in the study. Without you this work would not be possible.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2020.551921/full#supplementary-material

References

1. Valcour V, Chalermchai T, Sailasuta N, Marovich M, Lerdlum S, Suttichom D, et al. Central nervous system viral invasion and inflammation during acute HIV infection. J Infect Dis. (2012) 206:275–82. doi: 10.1093/infdis/jis326

CrossRef Full Text | Google Scholar

2. Wang Y, Liu M, Lu Q, Farrell M, Lappin JM, Shi J, et al. Global prevalence and burden of HIV-associated neurocognitive disorder: a meta-analysis. Neurology. (2020). doi: 10.1212/WNL.0000000000010752. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Heaton RK, Franklin DR, Ellis RJ, Mccutchan JA, Letendre SL, Leblanc S, et al. HIV-associated neurocognitive disorders before and during the era of combination antiretroviral therapy: differences in rates, nature, and predictors. J Neurovirol. (2011) 17:3–16. doi: 10.1007/s13365-010-0006-1

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Dawes S, Suarez P, Casey CY, Cherner M, Marcotte TD, Letendre S, et al. Variable patterns of neuropsychological performance in HIV-1 infection. J Clin Exp Neuropsychol. (2008) 30:613–26. doi: 10.1080/13803390701565225

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Gomez D, Power C, Gill MJ, Koenig N, Vega R, Fujiwara E. Empiric neurocognitive performance profile discovery and interpretation in HIV infection. J Neurovirol. (2018) 25:72–84. doi: 10.1007/s13365-018-0685-6

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Pasipanodya EC, Montoya JL, Campbell LM, Hussain MA, Saloner R, Paolillo EM, et al. Metabolic risk factors as differential predictors of profiles of neurocognitive impairment among older HIV+ and HIV- adults: an observational study. Arch Clin Neuropsychol. (2019). doi: 10.1093/arclin/acz040. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

7. May PE, Heithoff AJ, Wichman CS, Phatak VS, Moore DJ, Heaton RK, et al. Assessing cognitive functioning in people living with HIV (PLWH): factor analytic results from CHARTER and NNTC cohorts. J Acquir Immune Defic Syndr. (2020) 83:251–9. doi: 10.1097/QAI.0000000000002252

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Nightingale S, Winston A, Letendre S, Michael BD, Mcarthur JC, Khoo S, et al. Controversies in HIV-associated neurocognitive disorders. Lancet Neurol. (2014) 13:1139–51. doi: 10.1016/S1474-4422(14)70137-1

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Milanini B, Catella S, Perkovich B, Esmaeili-Firidouni P, Wendelken L, Paul R, et al. Psychiatric symptom burden in older people living with HIV with and without cognitive impairment: the UCSF HIV over 60 cohort study. AIDS Care. (2017) 29:1178–85. doi: 10.1080/09540121.2017.1281877

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Rubin LH, Neigh GN, Sundermann EE, Xu Y, Scully EP, Maki PM. Sex Differences in neurocognitive function in adults with HIV: patterns, predictors, and mechanisms. Curr Psychiatry Rep. (2019) 21:94. doi: 10.1007/s11920-019-1089-x

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Vassallo M, Durant J, Lebrun-Frenay C, Fabre R, Ticchioni M, Andersen S, et al. Virologically suppressed patients with asymptomatic and symptomatic HIV-associated neurocognitive disorders do not display the same pattern of immune activation. HIV Med. (2015) 16:431–40. doi: 10.1111/hiv.12246

CrossRef Full Text | Google Scholar

12. Foca E, Magro P, Motta D, Compostella S, Casari S, Bonito A, et al. Screening for neurocognitive impairment in HIV-infected individuals at first contact after HIV diagnosis: the experience of a large clinical center in Northern Italy. Int J Mol Sci. (2016) 17:434. doi: 10.3390/ijms17040434

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Kabuba N, Menon JA, Franklin DR Jr, Heaton RK, Hestad KA. HIV- and AIDS-associated neurocognitive functioning in Zambia - a perspective based on differences between the genders. Neuropsychiatr Dis Treat. (2016) 12:2021–8. doi: 10.2147/NDT.S105481

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Royal W 3rd, Cherner M, Burdo TH, Umlauf A, Letendre SL, Jumare J, et al. Associations between cognition, gender and monocyte activation among HIV infected individuals in Nigeria. PLoS ONE. (2016) 11:e0147182. doi: 10.1371/journal.pone.0147182

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Burlacu R, Umlauf A, Luca A, Gianella S, Radoi R, Ruta SM, et al. Sex-based differences in neurocognitive functioning in HIV-infected young adults. AIDS. (2018) 32:217–25. doi: 10.1097/QAD.0000000000001687

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Gascon MRP, Vidal JE, Mazzaro YM, Smid J, Marcusso RMN, Capitao CG, et al. Neuropsychological assessment of 412 HIV-infected individuals in Sao Paulo, Brazil. AIDS Patient Care STDS. (2018) 32:1–8. doi: 10.1089/apc.2017.0202

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Sundermann EE, Heaton RK, Pasipanodya E, Moore RC, Paolillo EW, Rubin LH, et al. Sex differences in HIV-associated cognitive impairment. AIDS. (2018) 32:2719–26. doi: 10.1097/QAD.0000000000002012

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Heaton RK, Franklin DR Jr, Deutsch R, Letendre S, Ellis RJ, Casaletto K, et al. Neurocognitive change in the era of HIV combination antiretroviral therapy: the longitudinal CHARTER study. Clin Infect Dis. (2015) 60:473–80. doi: 10.1093/cid/ciu862

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Behrman-Lay AM, Paul RH, Heaps-Woodruff J, Baker LM, Usher C, Ances BM. Human immunodeficiency virus has similar effects on brain volumetrics and cognition in males and females. J Neurovirol. (2016) 22:93–103. doi: 10.1007/s13365-015-0373-8

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Do TC, Kerr SJ, Avihingsanon A, Suksawek S, Klungkang S, Channgam T, et al. HIV-associated cognitive performance and psychomotor impairment in a Thai cohort on long-term cART. J Virus Erad. (2018) 4:41–7. doi: 10.1016/S2055-6640(20)30243-0

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Maki PM, Rubin LH, Springer G, Seaberg EC, Sacktor N, Miller EN, et al. Differences in cognitive function between women and men with HIV. J Acquir Immune Defic Syndr. (2018) 79:101–7. doi: 10.1097/QAI.0000000000001764

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Maki PM, Martin-Thormeyer E. HIV, cognition and women. Neuropsychol Rev. (2009) 19:204–14. doi: 10.1007/s11065-009-9093-2

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Singer M. AIDS and the health crisis of the U.S. urban poor; the perspective of critical medical anthropology. Soc Sci Med. (1994) 39:931–48. doi: 10.1016/0277-9536(94)90205-4

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Tsai AC, Burns BF. Syndemics of psychosocial problems and HIV risk: a systematic review of empirical tests of the disease interaction concept. Soc Sci Med. (2015) 139:26–35. doi: 10.1016/j.socscimed.2015.06.024

CrossRef Full Text | Google Scholar

25. Rubin LH, Springer G, Martin EM, Seaberg EC, Sacktor NC, Levine A, et al. Elevated depressive symptoms are a stronger predictor of executive dysfunction in HIV-infected women than men. J Acquir Immune Defic Syndr. (2019) 81:274–83. doi: 10.1097/QAI.0000000000002029

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Klove H. Clinical Neuropsychology. New York, NY: Saunders (1963).

27. Reitan R, Wolfson D. The Halstead-Reitan Neuropsychological Test Battery: Theory and Clinical Interpretation. Tuscon, AZ: Neuropsychology Press (1985).

28. Golden CJ. The Stroop Color and Word Test: A Manual for Clinical and Experimental Use. Chicago, IL: Stoelting (1978).

29. Gronwall DMA. Paced auditory serial-addition task: a measure of recovery from concussion. Perceptual Motor Skills. (1977) 44:367–73. doi: 10.2466/pms.1977.44.2.367

CrossRef Full Text | Google Scholar

30. Diehr MC, Heaton RK, Miller W, Grant I. The Paced Auditory Serial Addition Task (PASAT): norms for age, education, and ethnicity. Assessment. (1998) 5:375–87. doi: 10.1177/107319119800500407

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Brandt J, Benedict R. Hopkins Verbal Learning Test-Revised: Professional Manual. Lake Magdalene, FL: Psychological Assessment Resources (2001).

32. Benedict RHB. Brief Visuospatial Memory Test - Revised: Professional Manual. Lutz, FL: Psychological Assessment Resources, Inc (1997).

33. Borkowski JG, Benton AL, Spreen O. Word fluency and brain damage. Neuropsychologia. (1967) 5:135–40. doi: 10.1016/0028-3932(67)90015-2

CrossRef Full Text | Google Scholar

34. Wechsler D. Wechsler Adult Intelligence Scale, Vol. 4. San Antonio, TX: Pearson (2008).

Google Scholar

35. Heaton RK, Miller SW, Taylor MJ, Grant I. Revised Comprehensive Norms for an Expanded Halstead-Reitan Battery: Demographically Adjusted Neuropsychological Norms for African American and Caucasian Adults Scoring Program. Odessa, FL: Psychological Assessment Resources (2004).

36. Norman MA, Moore DJ, Taylor M, Franklin D Jr, Cysique L, Ake C, et al. Demographically corrected norms for African Americans and Caucasians on the Hopkins Verbal Learning Test-Revised, Brief Visuospatial Memory Test-Revised, Stroop Color and Word Test, and Wisconsin Card Sorting Test 64-Card Version. J Clin Exp Neuropsychol. (2011) 33:793–804. doi: 10.1080/13803395.2011.559157

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Lawton MP, Brody EM. Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist. (1969) 9:179–86. doi: 10.1093/geront/9.3_Part_1.179

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Wilkinson GS, Robertson GJ. WRAT 4: Wide Range Achievement Test. Lutz, FL: Psychological Assessment Resources (2006).

Google Scholar

39. Beck AT, Steer RA, Brown GK. Beck Depression Inventory-II. San Antonio, TX: PsychCorp (1996).

Google Scholar

40. Wittchen HU. Reliability and validity studies of the WHO–Composite International Diagnostic Interview (CIDI): a critical review. J Psychiatr Res. (1994) 28:57–84. doi: 10.1016/0022-3956(94)90036-1

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Wehrens R, Buydens L. Self- and super-organizing maps in R: the kohonen package. J Stat Softw. (2007) 21:1–19. doi: 10.18637/jss.v021.i05

CrossRef Full Text | Google Scholar

42. Fraley C, Raftery A, Scrucca L. mclust: Normal Mixture Modeling for Model-Based Clustering, Classification, and Density Estimation (2014).

Google Scholar

43. Kuhn M. Building predictive models in R using the caret package. J Stat Softw. (2008) 28:1–26. doi: 10.18637/jss.v028.i05

CrossRef Full Text | Google Scholar

44. Kuhn M. Variable Importance Using the Caret Package (2012).

Google Scholar

45. Torgo L, Torgo M. Package 'DMwR'. Comprehensive R Archive Network (2013).

Google Scholar

46. Van Buuren S, Groothuis-Oudshoorn K. mice: multivariate imputation by chained equations in R. J Stat Softw. (2011) 45:1–67. doi: 10.18637/jss.v045.i03

CrossRef Full Text | Google Scholar

47. Hewitt RG, Parsa N, Gugino L. Women's health. The role of gender in HIV progression. AIDS Read. (2001) 11:29–33.

Google Scholar

48. Griesbeck M, Scully E, Altfeld M. Sex and gender differences in HIV-1 infection. Clin Sci. (2016) 130:1435–51. doi: 10.1042/CS20160112

CrossRef Full Text | Google Scholar

49. Maki PM, Rubin LH, Valcour V, Martin E, Crystal H, Young M, et al. Cognitive function in women with HIV: findings from the Women's Interagency HIV Study. Neurology. (2015) 84:231–40. doi: 10.1212/WNL.0000000000001151

CrossRef Full Text | Google Scholar

50. Basso MR, Bornstein RA. Estimated premorbid intelligence mediates neurobehavioral change in individuals infected with HIV across 12 months. J Clin Exp Neuropsychol. (2000) 22:208–18. doi: 10.1076/1380-3395(200004)22:2;1-1;FT208

CrossRef Full Text | Google Scholar

51. Farinpour R, Miller EN, Satz P, Selnes OA, Cohen BA, Becker JT, et al. Psychosocial risk factors of HIV morbidity and mortality: findings from the Multicenter AIDS Cohort Study (MACS). J Clin Exp Neuropsychol. (2003) 25:654–70. doi: 10.1076/jcen.25.5.654.14577

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Ruff RM, Parker SB. Gender- and age-specific changes in motor speed and eye-hand coordination in adults: normative values for the Finger Tapping and Grooved Pegboard Tests. Perceptual Motor Skills. (1983) 76:1219–30. doi: 10.2466/pms.1993.76.3c.1219

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Bleecker ML, Bolla-Wilson K, Agnew J, Meyers DA. Age-related sex differences in verbal memory. J Clin Psychol. (1988) 44:403–11. doi: 10.1002/1097-4679(198805)44:3<403::AID-JCLP2270440315>3.0.CO;2-0

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Kramer JH, Delis DC, Daniel M. Sex differences in verbal learning. J Clin Psychol. (1988) 44:907–15. doi: 10.1002/1097-4679(198811)44:6<907::AID-JCLP2270440610>3.0.CO;2-8

CrossRef Full Text | Google Scholar

55. Kramer JH, Delis DC, Kaplan E, O'donnell L, Prifitera A. Developmental sex differences in verbal learning. Neuropsychology. (1997) 11:577–84. doi: 10.1037/0894-4105.11.4.577

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Barrett-Connor E, Kritz-Silverstein D. Gender differences in cognitive function with age: the Rancho Bernardo study. J Am Geriatr Soc. (1999) 47:159–64. doi: 10.1111/j.1532-5415.1999.tb04573.x

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Schmidt SL, Oliveira RM, Rocha FR, Abreu-Villaca Y. Influences of handedness and gender on the grooved pegboard test. Brain Cogn. (2000) 44:445–54. doi: 10.1006/brcg.1999.1204

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Joffe H, Hall JE, Gruber S, Sarmiento IA, Cohen LS, Yurgelun-Todd D, et al. Estrogen therapy selectively enhances prefrontal cognitive processes: a randomized, double-blind, placebo-controlled study with functional magnetic resonance imaging in perimenopausal and recently postmenopausal women. Menopause. (2006) 13:411–22. doi: 10.1097/01.gme.0000189618.48774.7b

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Arevalo MA, Azcoitia I, Garcia-Segura LM. The neuroprotective actions of oestradiol and oestrogen receptors. Nat Rev Neurosci. (2015) 16:17–29. doi: 10.1038/nrn3856

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Engler-Chiurazzi EB, Brown CM, Povroznik JM, Simpkins JW. Estrogens as neuroprotectants: estrogenic actions in the context of cognitive aging and brain injury. Prog Neurobiol. (2017) 157:188–211. doi: 10.1016/j.pneurobio.2015.12.008

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Epperson CN, Sammel MD, Freeman EW. Menopause effects on verbal memory: findings from a longitudinal community cohort. J Clin Endocrinol Metab. (2013) 98:3829–38. doi: 10.1210/jc.2013-1808

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Weber MT, Rubin LH, Maki PM. Cognition in perimenopause: the effect of transition stage. Menopause. (2013) 20:511–7. doi: 10.1097/gme.0b013e31827655e5

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Weber MT, Maki PM, Mcdermott MP. Cognition and mood in perimenopause: a systematic review and meta-analysis. J Steroid Biochem Mol Biol. (2014) 142:90–8. doi: 10.1016/j.jsbmb.2013.06.001

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Seeman TE, Mcewen BS, Singer BH, Albert MS, Rowe JW. Increase in urinary cortisol excretion and memory declines: MacArthur studies of successful aging. J Clin Endocrinol Metab. (1997) 82:2458–65. doi: 10.1210/jc.82.8.2458

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Cohen M, Deamant C, Barkan S, Richardson J, Young M, Holman S, et al. Domestic violence and childhood sexual abuse in HIV-infected women and women at risk for HIV. Am J Public Health. (2000) 90:560–5. doi: 10.2105/AJPH.90.4.560

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Peavy G, Jacobs D, Salmon DP, Butters N, Delis DC, Taylor M, et al. Verbal memory performance of patients with human immunodeficiency virus infection: evidence of subcortical dysfunction. The HNRC Group. J Clin Exp Neuropsychol. (1994) 16:508–23. doi: 10.1080/01688639408402662

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Becker JT, Caldararo R, Lopez OL, Dew MA, Dorst SK, Banks G. Qualitative features of the memory deficit associated with HIV infection and AIDS: cross-validation of a discriminant function classification scheme. J Clin Exp Neuropsychol. (1995) 17:134–42. doi: 10.1080/13803399508406588

PubMed Abstract | CrossRef Full Text | Google Scholar

68. White DA, Taylor MJ, Butters N, Mack C, Salmon DP, Peavy G, et al. Memory for verbal information in individuals with HIV-associated dementia complex. HNRC Group. J Clin Exp Neuropsychol. (1997) 19:357–66. doi: 10.1080/01688639708403864

CrossRef Full Text | Google Scholar

69. Manly JJ, Touradji P, Tang MX, Stern Y. Literacy and memory decline among ethnically diverse elders. J Clin Exp Neuropsychol. (2003) 25:680–90. doi: 10.1076/jcen.25.5.680.14579

PubMed Abstract | CrossRef Full Text | Google Scholar

70. Baker DW, Parker RM, Williams MV, Clark WS. Health literacy and the risk of hospital admission. J Gen Intern Med. (1998) 13:791–8. doi: 10.1046/j.1525-1497.1998.00242.x

PubMed Abstract | CrossRef Full Text | Google Scholar

71. Stankoff B, Calvez V, Suarez S, Bossi P, Rosenblum O, Conquy L, et al. Plasma and cerebrospinal fluid human immunodeficiency virus type-1 (HIV-1) RNA levels in HIV-related cognitive impairment. Eur J Neurol. (1999) 6:669–75. doi: 10.1046/j.1468-1331.1999.660669.x

PubMed Abstract | CrossRef Full Text | Google Scholar

72. Valcour V, Yee P, Williams AE, Shiramizu B, Watters M, Selnes O, et al. Lowest ever CD4 lymphocyte count (CD4 nadir) as a predictor of current cognitive and neurological status in human immunodeficiency virus type 1 infection–The Hawaii Aging with HIV Cohort. J Neurovirol. (2006) 12:387–91. doi: 10.1080/13550280600915339

PubMed Abstract | CrossRef Full Text | Google Scholar

73. Ellis RJ, Badiee J, Vaida F, Letendre S, Heaton RK, Clifford D, et al. CD4 nadir is a predictor of HIV neurocognitive impairment in the era of combination antiretroviral therapy. AIDS. (2011) 25:1747–51. doi: 10.1097/QAD.0b013e32834a40cd

PubMed Abstract | CrossRef Full Text | Google Scholar

74. Cysique LA, Brew BJ. Neuropsychological functioning and antiretroviral treatment in HIV/AIDS: a review. Neuropsychol Rev. (2009) 19:169–85. doi: 10.1007/s11065-009-9092-3

PubMed Abstract | CrossRef Full Text | Google Scholar

75. Kramer-Hammerle S, Rothenaigner I, Wolff H, Bell JE, Brack-Werner R. Cells of the central nervous system as targets and reservoirs of the human immunodeficiency virus. Virus Res. (2005) 111:194–213. doi: 10.1016/j.virusres.2005.04.009

PubMed Abstract | CrossRef Full Text | Google Scholar

76. Spudich SS, Ances BM. Neurologic complications of HIV infection: highlights from the 2013 Conference on Retroviruses and Opportunistic Infections. Top Antivir Med. (2013) 21:100–8.

PubMed Abstract | Google Scholar

77. Rock PL, Roiser JP, Riedel WJ, Blackwell AD. Cognitive impairment in depression: a systematic review and meta-analysis. Psychol Med. (2014) 44:2029–40. doi: 10.1017/S0033291713002535

PubMed Abstract | CrossRef Full Text | Google Scholar