A novel method for quantitative analysis of subjective experience reports: application to psychedelic visual experiences

Introduction: Psychedelic compounds such as LSD, psilocybin, mescaline, and DMT can dramatically alter visual perception. However, the extent to which visual effects of psychedelics consistently vary for different substances is an open question. The visual effects of a given psychedelic compound can range widely both across and within individuals, so datasets with large numbers of participants and descriptions of qualitative effects are required to adequately address this question with the necessary sensitivity.

Methods: Here we present an observational study with narrative self-report texts, leveraging the massive scale of the Erowid experience report dataset. We analyzed reports associated with 103 different psychoactive substances, with a median of 217 reports per substance. Thirty of these substances are standardly characterized as psychedelics, while 73 substances served as comparison substances. To quantitatively analyze these semantic data, we associated each sentence in the self-report dataset with a vector representation using an embedding model from OpenAI, and then we trained a classifier to identify which sentences described visual effects, based on the sentences’ embedding vectors.

Results: We observed that the proportion of sentences describing visual effects varies significantly and consistently across substances, even within the group of psychedelics. We then analyzed the distributions of psychedelics’ visual effect sentences across different categories of effects (for example, movement, color, or pattern), again finding significant and consistent variation.

Discussion: Overall, our findings indicate reliable variation across psychedelic substances’ propensities to affect vision and in their qualitative effects on visual perception.

Introduction

Visual perceptual effects are hallmark features of psychedelic substances. LSD and psilocybin are two “classical psychedelics” that have been administered to human research volunteers (Carhart-Harris et al., 2016; Griffiths et al., 2006), and a great diversity of other psychedelic compounds serve as valuable research tools as well (Shulgin and Shulgin, 1997; Shulgin and Shulgin, 1991).

Based on their particular chemical structures, different psychedelic compounds have distinct effects in the brain. To the extent that different psychedelics also alter visual perception in characteristic ways, a comparison of physiological and qualitative effects across many psychedelics would be a powerful way to study the neural processes underlying conscious visual experience. However, how psychedelic visual effects vary for different substances is a largely open question. Anecdotally, visual effects often range widely across individuals even for a specific psychedelic compound, so a controlled experiment involving administration of psychedelics to human participants would likely not have the sensitivity to effectively answer this question. Here we address this question by conducting an observational study of narrative self-report texts, leveraging the massive scale of the Erowid dataset.

Erowid Center archives narrative self-reported texts of experiences with psychoactive substances, submitted by users and accessible to the public at its website.¹ In our study, we analyzed experience reports associated with 103 different substances, with a median of 217 reports per substance. Thirty of the substances are generally recognized as psychedelics (some of which are also referred to as hallucinogens), while 73 served as comparison substances in our analyses and include sedatives, stimulants, herbs, and other drug classes. We designated substances in our study as psychedelic if they are characterized as such in the Erowid experience report dataset and if they either belong to tryptamine, phenethylamine, or lysergamide chemical classes or include compounds from these classes.

To quantitatively analyze these semantic data, we associated each sentence with a text embedding vector representation, mapping its semantic information to a mathematical form. The text-embedding-ada-002 model (OpenAI) (Greene et al., 2022) was used to generate vectors for each of the 2.2 million sentences in our text dataset. We then employed logistic regression to identify sentences describing visual subjective effects based on their vector representations. This analytical approach differs from previous text processing studies of the Erowid data set that relied primarily on word frequency-based or occurrence-based analysis methods (Mooseder et al., 2022; Nayak et al., 2021; Sanz et al., 2018; Zamberlan et al., 2018).

We observed that the proportion of sentences describing visual effects varies substantially and systematically across substances, even within the subset of psychedelic compounds. Next, we manually identified a group of categories of visual experiences by surveying the full set of visual effect sentences. For each substance, we calculated the proportion of visual effect sentences within each defined experience category, and we found that psychedelic compounds consistently differ in their profiles of visual effects.

Overall, our analyses demonstrate significant variation in psychedelic substances’ propensities to affect visual experience and other qualitative effects. Our findings also establish a new method for quantitative analysis and categorization of visual effects of psychoactive substances and other altered states of consciousness. The analysis method we describe here can be utilized in future studies to systematically characterize differences among psychedelic substances for various aspects of subjective experience. Our method also provides a foundation for future studies of psychoactive substances that relate physiological and biological measures to quantitative metrics of subjective experience, and our results indicate that the neurotransmitter receptor activity patterns that mediate psychedelic visual phenomena are multifactorial.

Methods

Erowid experience vault: subjective experience report dataset

Erowid Center is a nonprofit organization whose mission is to provide accurate and unbiased information about psychoactive substances freely to the public via its website, Erowid.org (Erowid, 1995–2023). Part of this mission involves informing website visitors about the acute effects of different psychoactive substances. The existing scientific literature on physiological and subjective effects is available on the Erowid website for a limited set of substances, but there are many more substances that people use recreationally, ceremonially, and medicinally whose effects have not been well characterized in experimentally controlled settings. Therefore, Erowid Center collects and maintains a publicly available archive of user-submitted text reports of subjective experiences with psychoactive substances.

Any Erowid website user may freely submit a text report describing their experiences with a psychoactive substance. Users may also submit reports of experiences that are related to psychoactive substances or other altered states of consciousness, such as dreams, meditation, drug testing, and law enforcement encounters. For our study, we excluded these reports from the analysis.

Erowid provides minimal guidelines and instructions for user-submitted reports, emphasizing well-written descriptive information about the user’s experiences, including mindset and setting, dosage and timing, physical and mental effects, preparation and intention, insights gained, and problems encountered. Erowid Center volunteers review submissions and screen out reports that are obviously fictional or exaggerated and/or do not provide useful information.

Researchers partnering with Erowid Center can access report text and metadata via an application programming interface (API). For our study, we used the Erowid API to download complete subjective report text for all substances associated with at least 100 distinct reports that were published between June 13, 1995 and May 22, 2023.

Text preprocessing

We first preprocessed the text dataset in the Python programming language to prepare it for analysis. Initially, we employed the Beautiful Soup package (Richardson, 2007) to remove markup text that was used to format the reports for presentation on the Erowid website. Next, we separated each report into its constituent sentences by splitting the text at period characters. This procedure included exceptions for period characters between two numeral characters that we assumed denoted decimal points and for adjacent period characters that we assumed denoted ellipses. We associated each resulting sentence with a sentence identification number and a report identification number, along with a substance label. Reports that were associated with more than one substance in the Erowid metadata were excluded from the dataset.

The final dataset comprised reports for 103 substances (total of 39,586 reports; median of 217 reports per substance). The median number of sentences per report was 40, with a minimum of 1 and a maximum of 1,396. Table 1 summarizes the dataset at the level of individual substances. Substance names are presented in Table 1 and the following figures as they were entered in the Erowid experience report database verbatim. Supplementary Table 1 associates these Erowid database substance names with chemical names or other identifiers.

Table 1

Table 1. Summary of Erowid experience report text dataset.

Text embedding vectors

To quantitatively analyze the semantic content of the experience report texts, we associated each sentence in the dataset with a text embedding vector. A text embedding is a mapping from character strings to vectors such that the semantic similarity of two strings is related to the mathematical similarity of the associated vectors. For sentence embedding, two sentences with similar meanings (e.g., “I sauteed the tofu” and “I braised the bean curd”) would be separated by a smaller Euclidian distance in vector space than two sentences with more different meanings (“I sauteed the tofu” and “The president enjoyed my cooking”).

For each sentence in the text dataset, we generated a corresponding embedding vector using the text-embedding-ada-002 model (OpenAI) (Greene et al., 2022). Text embeddings were computed with the OpenAI API over a period from July 10, 2023, to July 23, 2023. The text-embedding-ada-002 model associates any input text string with a 1,536-dimensional vector, and at the time of our analysis, it was the highest performing text embedding model available from OpenAI (Greene et al., 2022).

Visual effect sentence classifier

Erowid experience reports are unconstrained narrative reports that often contain information about a psychoactive substance user’s mindset and setting, context and motivation, methods of preparation and administration, etc., along with descriptions of various subjective effects. In order to systematically compare visual subjective effects across substances, we needed to identify descriptions of visual subjective effects in narrative text, but our dataset was too large to allow manual evaluation of every sentence. We therefore developed a logistic regression classifier model to detect sentences that describe visual effects, based on their vector embeddings.

First, we created a set of labeled sentences to train the logistic regression model. We randomly sampled 10,000 sentences, without replacement, from the report dataset. For each sentence, we performed an OpenAI GPT-4 language model API call to label visual effect sentences. This GPT-4 API call used the following system prompt: “You are a model that identifies effects of psychoactive substances on visual experience (1) or not (0). Classify the following sentence. Respond only with 1 or 0. A 1 means that the sentence explicitly describes a visual effect.”

The user prompt for each API call was one of the 10,000 sampled sentences, generating a 1 or a 0 response for each of the sampled sentences. The API call temperature was set to 0.0 to minimize the possibility of unexpected (“creative”) responses to the prompt. Of the 10,000 sampled sentences submitted to GPT-4 for labeling, 1,231 (12.31%) were labeled as visual effect sentences, and 8,769 (87.69%) were labeled as not visual effect sentences.

To check the quality of the automated labeling procedure, we performed a manual review on a random sample of 100 of the sentences labeled by GPT-4 as 0 and 100 of the sentences labeled as 1. One of the investigators (S.N.) made a subjective judgment about whether each of these sentences were visual effect sentences or not, without knowledge of the labels assigned by GPT-4. All 100 of the sentences that were labeled as “not visual effect” by GPT-4 were classified the same way by the manual review. However, 18 of the sentences that were labeled as “visual effect” by GPT-4 were classified as “not visual effect” by the manual review. We therefore manually reviewed all 1,231 sentences labeled as “visual effect” by GPT-4, correcting the labels where necessary to reduce the false positive rate of our subsequent classifier training on the full data set. This resulted in a change of labels of 244 “visual effect” sentences to “not visual effect,” with a final count of 987 “visual effect” sentences and 9,013 “not visual effect” sentences.

We then used the Scikit-learn (Pedregosa et al., 2011) Python package to build a logistic regression classifier for our labeled training data. First, we added the corresponding embedding vectors to the labeled sentence dataset. We then trained the logistic regression on the embedding vectors, with each of the 1,536 dimensions as an input variable, and the corresponding sentence label (“not visual effect” or “visual effect”) as the outcome variable.

We used an 80–20% train-test split procedure and undersampled the training set such that the number of “not visual effect” sentences matched the number of “visual effect” sentences (987 sentences). The prediction accuracy of this training procedure was 89.3%. Prediction accuracy is defined as the number of sentences for which the labels produced by the logistic regression classifier matched the labels in the training set, divided by the number of sentences in the test set.

We then used the trained classifier to predict the probability that each sentence in our full Erowid sentence dataset was a visual effect sentence. We manually reviewed a sample of classified sentences to assess whether the predicted probabilities generated by the classifier corresponded to our own judgments of “visual effect” versus “not visual effect” sentences.

Example sentences and their associated classifier probabilities are displayed in Table 2. We observed that the prediction probability was generally well correlated with our own certainty about whether a sentence described a visual effect. Based on our review, we chose a prediction probability threshold of 0.75 to classify a sentence as a visual effect sentence, aiming to limit false positives. The total number of visual effect sentences above this prediction probability threshold was 143,520, corresponding to 6.52% of the total number of sentences in the full data set.

Table 2

Table 2. Examples of Erowid subjective report sentences and visual effect logistic regression prediction probabilities.

In Figure 1, we display the prediction probability distribution for all sentences in the dataset and for a selection of individual substances. In general, most of the sentences have a prediction probability below 0.5. This was expected, given that Erowid reports often describe many more aspects of a psychoactive substance experience than just visual subjective effects. We display the prediction probability distribution for the top four and bottom four number of reports (substances needed to have at least 100 reports in the data set to be included in our analysis) (Figure 1). We observed that the general shape of the probability distributions is similar across substances, with most variation across substances occurring in the upper end of the distribution. This variation is further explored below.

Figure 1

Figure 1. Histograms of the relative frequency distribution of prediction probability values for selected substances. For each subplot, prediction probability values are represented along the horizontal axis, and relative frequency is represented along the vertical axis. Because the different substances have different numbers of sentences in their total experience report data, the vertical axis scales are normalized so that the maximum frequency value is plotted at the top of each subplot. The red line in each subplot depicts the overall distribution of prediction probabilities for the entire dataset. Prediction probability values were calculated by the trained logistic regression classifier model for each sentence in the dataset.

Categorizing visual effects

In this study, we assessed whether experience reports from users of psychedelic and other psychoactive substances consistently vary in their descriptions of visual effects. To conduct this analysis, we first extracted those sentences describing visual effects from the original subjective reports using the trained logistic regression classifier. We created a new visual effect sentences dataset consisting of all sentences whose classifier prediction probability was at least 0.75, along with their substance labels and their embedding vectors. We then used the embedding vectors to quantitatively analyze the distributions of types of visual effects across substances.

Embedding vectors translate semantic similarity of sentences into mathematical distance in vector space. We therefore used the embedding vectors to quantify, for each substance, the proportion of sentences that describe particular categories of visual effects. To determine the categories of visual effects to be analyzed, we surveyed the full visual effect semantic space by projecting the embedding vector dataset into two dimensions using uniform manifold approximation and projection (UMAP), a dimensionality reduction method optimized to maintain the global and local structure of high-dimensional data that have nonlinear variation (McInnes et al., 2018).

The UMAP 2-D projection of the embedding vectors facilitates assessment of how different visual effect sentences are semantically related to one another and identification of prototypical visual effect sentences representing distinct regions of the visual effect sentence space. Specifically, one of the authors (SN) surveyed the 2-D UMAP of visual effect sentences by densely sampling and reading sentences across the projection to explore the semantic space. Based on this survey, the same author (SN) then manually generated prototypical sentences to reflect common types of visual effect descriptions encountered. This process is illustrated in Figure 2, and the prototypical sentences we generated as seed sentences for further analysis are listed in Table 3.

Figure 2

Figure 2. (A) Two-dimensional UMAP projection of all visual effect sentence embedding vectors. Each point corresponds to a single visual effect sentence, and colors denote different substances. The locations of some example sentences are indicated with arrows. (B) Illustration of visual effect categorization and calculation method. Seed sentences (listed in Table 3) were created to represent different regions of the visual effect vector space. Each visual effect was defined as the area within a threshold distance of a seed sentence vector. The proportions of visual effect sentences that fell within each category were then compared across substances. The black circles overlaid on the 2-dimensional point clouds demonstrate our procedure for categorizing visual effect sentences by calculating distances from seed sentence vectors (the center of each circle) and setting a distance threshold. Note that the analysis was conducted in the original 1,536-dimensional vector space, and the two-dimensional projection with overlaid circles/visual effect categories shown here is for illustration purposes.

Table 3

Table 3. Seed sentences for visual effect categorization analysis.

To quantify any given substance’s likelihood of causing different visual effects, we calculated the proportion of that substance’s visual effect sentences that contained embedding vectors within a distance threshold of each seed sentence. We defined a distance threshold of 0.55, based on a survey of subjective report sentences and their vector distances from the seed sentences. We determined that this threshold value is the approximate distance above which sentences are no longer semantically similar enough to the seed sentence to justify being labeled as an instance of that category of visual effect. Figure 2 illustrates this distance calculation method for a two-dimensional UMAP projection (although the quantitative analysis involves computing distance over all 1,536 dimensions). Figure 3 illustrates the full analysis method with a flowchart.

Figure 3

Figure 3. Flowchart of method for calculating visual effect distributions across substances from the Erowid experience report dataset.

To test for statistical significance of differences among psychedelic substances in their profiles of visual effect sentence categories, we performed a permutation test using a statistic that measures the differences across substances in their respective associations with seed sentences. This test statistic T quantifies the average inter-substance variability in relation to each seed sentence. Specifically, for each seed sentence, the mean absolute difference between the sentence proportion values was computed for all pairwise comparisons of substances, thereby quantifying the dispersion of each substance’s association with that seed sentence. This process was repeated for each seed sentence, and the value of the test statistic is the overall mean of these differences. This measure summarizes the variation among substances in their associations with the spectrum of visual effect categories.

Formally, T is defined as:

$T=\frac{1}{\left|S\right|}{\displaystyle \sum }_{s\in S}\left(\frac{2}{\left|D\right|\left(\left|D\right|-1\right)}{\displaystyle \sum }_{d1,d2\in D;d1\ne d2}|V\left(s,d1\right)-V\left(s,d2\right)|\right)$

Where:

– S is the set of all seed sentences,

– D is the set of all substances,

– V(s,d) represents the value (the proportion of subjective report sentences below the distance threshold) for seed sentence s and substance d,

– ∣S∣ and ∣D∣ represent the number of seed sentences and substances, respectively.

In this equation, the inner sum represents the mean absolute difference between all pairwise comparisons of substances for seed sentence s. The outer sum is the average of these mean differences across all seed sentences.

We created seed sentences that reflected categories of visual effect sentences that we frequently encountered while manually reviewing the list of visual effect sentences, arranged by their semantic similarity in the 2-D UMAP projection. Therefore, the generation of seed sentences was a top-down, rather than data-driven, process. In future work, a data-driven method for identifying visual effect categories without manual classification would allow for a more objective analysis of systematic differences among psychedelic substances’ visual effects. However, for the present study, we reasoned that manually identified visual effect categories that exhibited statistically significant differences would allow testing for differences in the visual subjective effects among psychedelic substances.

Results

We observed that the proportion of visual effect sentences varies significantly by substance (Figure 4). Using a chi-square procedure, we rejected the null hypothesis that across all sentences in the report dataset, substance and prediction probability vary independently, X² (114, N = 2,246,254) = 72,912, p < 0.001. The strength of association between substance and prediction probability, measured by Cramér’s V, was 0.180, indicating a small-to-moderate association. Psychedelic compounds tend to have a much higher proportion of visual effect sentences than any other drug category. We performed a two-sample t-test comparing the proportion of visual effect sentences for psychedelic versus non-psychedelic substances and found that psychedelic compounds have a significantly higher proportion of visual effect sentences (p < 0.001).

Figure 4

Figure 4. Proportion of visual effect sentences for each psychoactive substance. Substances are color coded by drug class.

The proportion of visual effect sentences also varied significantly by substance when analyzing only psychedelic substances (Figure 5), X² (29, N = 931,858) = 5,771, p < 0.001. The strength of the association between psychedelic and prediction, measured by Cramér’s V, was 0.078, indicating a small association.

Figure 5

Figure 5. Proportion of visual effect sentences for each psychedelic substance. Psychedelic substances are color coded by drug class.

The analyses displayed in Figures 4, 5 demonstrate reliable variation of the proportion of visual effects sentences across all substances and across psychedelic substances. In addition, we conducted a categorization analysis (see Methods) that showed that the proportions of different categories of visual effects vary across psychedelic substances. These results are displayed as a clustermap of proportions of categories of visual effect sentences, with seed sentences and substances arranged according to their dendrogram distances (Figure 6). Each leaf of the dendrogram represents one observation (a row or a column of the heatmap). Branches connect the leaves, and the number of branch crossings that must be traveled between two leaves represents the dissimilarity between those two leaves within the category (sentences or psychedelic substances) (Figure 6).

Figure 6

Figure 6. Clustermap of visual effect sentence proportions for each psychedelic substance. For each seed sentence, we calculated the proportion of visual effect sentences for each psychedelic substance that fell within a distance threshold in embedding vector space. This proportion quantifies the strength of association between report sentences and a given seed sentence and is visualized using the color map on the right of the figure. This map combines heatmap and dendrogram methods, displaying the hierarchical clusters among substances and visual effect sentences. In the dendrograms, the total branch distance between two leaves (seed sentences or substances) indicates dissimilarity between those two leaves. Substance labels are color coded by psychedelic chemical class, and seed sentence labels are color coded by sentence category.

We defined dissimilarity between two substances or two seed sentences as the Euclidean distance between their proportion vectors. The resulting clustermap reveals three distinct groups of seed sentences over all substances: relatively high, moderate, and low proportion values (Figure 6). These three clustermap groups do not map directly onto the seed sentences categories that we manually determined. For example, the high proportion clustermap group contains Color, Movement, and Pattern seed sentences, but these sentence categories are also found in the other two groups (Figure 6).

By visual inspection, the longest branches in the dendrogram for the seed sentences can be used to distinguish three groups. The clustering of seed sentences into these three groups suggests that variation in the overall proportion values across substances is the most important factor for clustering the seed sentences. In contrast, the dendrogram for the psychedelics does not reveal clearly distinct substance groups to the same extent as the dendrogram for the seed sentences (Figure 6). The substance dendrogram suggests that there are three major clusters of psychedelic substances, but the lengths of the branches that separate the three substance clusters are shorter than the corresponding branches for the three seed sentence clusters, suggesting that the dissimilarity of proportion values in the three clusters is less for the substances than it is for the seed sentences.

We defined a test statistic T that quantifies the extent to which psychedelic substances differ from one another in the strength of their associations with seed sentences (see Methods). For the dataset of visual effect sentences, T = 0.0165. We then created a null distribution by shuffling the substance labels in this dataset and recalculated T for the proportion values derived from the shuffled-label dataset.

We performed this shuffling and T calculation procedure 10,000 times to generate a distribution of T values that could be expected under the null distribution that there is no association between substance and visual effect profile. We then calculated the p-value as the proportion of permuted T values that were greater than or equal to the observed T value. This p-value was less than 0.001, indicating significant consistent variation across psychedelic substances in their relationships with seed sentences.

Discussion

We found that experience reports for different psychedelic substances vary in their proportions of different categories of visual effect sentences (Figure 6), suggesting that the profiles of visual effects for a given psychedelic are multifactorial. This would provide support for the patterns of receptor activation underlying the perceptual effects of psychedelics also being multifactorial, rather than being reducible to action at a single type or subtype of neurotransmitter receptor.

The 2A subtype of serotonin (5-hydroxytryptamine, or 5-HT) receptors (5-HT_2A receptors) has been proposed to mediate the subjective effects of psychedelics. Classical psychedelics like DMT, LSD, psilocybin, and mescaline are sometimes termed serotonergic psychedelics because of their affinity for 5-HT receptors, particularly the 2A subtype (Nichols, 2016). The 5-HT_2A receptor subtype has been described as necessary for some effects of classical psychedelics, including subjective ratings of complex visual imagery and increased visual cortical excitability (Kometer et al., 2013; Preller et al., 2018; Quednow et al., 2012). In addition, 5-HT_2A receptors are densely expressed in visual cortex (Beliveau et al., 2017).

Our results are consistent with the notion that activation of the 5-HT_2A receptor does not solely dictate psychedelic visual phenomenology. If psychedelics varied only in their binding affinities or activation levels of 5-HT_2A receptors, the intensities of their effects could be expected to vary across psychedelic substances as a function of dose. However, in this case, the profiles of subjective effects would be more consistent across different psychedelic substances, especially for two substances with similar binding affinities and functional activations of the 5-HT_2A receptor. In contrast, our results suggest that different psychedelics have characteristically different distributions of visual subjective effects and that there is a statistically significant association between these profiles of effects and the corresponding substances.

Results from studies employing the 5-HT_2A receptor antagonist ketanserin have been interpreted to mean that activation of this receptor subtype may be necessary for the characteristic psychoactive effects of psychedelics (Kometer et al., 2013; Preller et al., 2018; Quednow et al., 2012). However, several non-psychedelic 5-HT_2A receptor agonists have been identified. For example, lisuride and ergotamine are analogs of LSD that have been previously described as non-psychedelic because their subjective effects do not resemble those typically associated with psychedelics (González-Maeso et al., 2007; Pieri et al., 1978). However, these drugs are 5-HT_2A receptor agonists at levels comparable to those of their psychedelic congeners, as assessed by functional measures of receptor activation (Bonhaus et al., 1997; Egan et al., 1998). The existence of non-psychedelic 5-HT_2A receptor agonists indicates either that 5-HT_2A receptor activation alone is insufficient to cause visual effects, or that there is a threshold of receptor activation that must be reached to produce psychedelic effects in a particular signaling pathway or population of neurons (Wallach et al., 2023).

Furthermore, recent evidence indicates that ketanserin, in addition to being an antagonist for the 5-HT_2A receptor subtype, also has moderate affinity for the 5-HT_2C receptor subtype, as well as moderate to high affinity for several adrenergic and histamine receptors (Casey et al., 2022). Therefore, the profile of receptor activation across multiple serotonin receptor subtypes and other types of neurotransmitter receptors, rather than just the level of activation of 5-HT_2A receptors, is likely to determine the full set of subjective effects of any given psychedelic compound.

We observed that the proportion of visual effect sentences in the full dataset of experience reports varies significantly across all substances (Figure 4) and for the category of psychedelic substances (Figure 5). This variation represents differences across substances in their propensity to cause visual effects relative to other types of effects. Notably, psychedelics had significantly greater proportions of visual effect sentences than non-psychedelic substances, even though they also generally tended to have greater number of sentences per experience report in general (Table 1). Erowid experience reports are unconstrained and unprompted, and it is therefore likely that description of an effect in a given report reflects how salient or memorable that effect was for the report’s author.

Our findings of multifactorial visual effect profiles and varying propensities to cause visual effects across psychedelics, together with pharmacological evidence of different psychedelic compounds’ varying binding affinities across neurotransmitter receptor types (Jensen and Roth, 2008), suggest the importance of activation of multiple types of neurotransmitter receptors to account for the phenomenology of any given psychedelic compound. This possibility has been previously described in the literature. For example, it has been hypothesized that activation of 5-HT_1A receptors may have larger effects on central visual processes than 5-HT_2A/_2C receptor activation (Nichols, 2000). However, there is currently no clearly established link between receptor types/subtypes and propensities for visual psychedelic effects or types of visual effects. Other possible mechanisms that could lead 5-HT_2A agonists to have differing visual effects include agonist-directed trafficking (Berg et al., 1998), various forms of biased agonism, and activity of metabolites of ingested psychedelic substances.

Based on Cramér’s V values, there was a small-to-moderate effect size for the variation of proportion of visual effect sentences across all substances, and a small effect size when the analysis was limited to psychedelic substances. The larger effect size in the all-substance analysis is driven by the varying propensities of different drug classes to cause visual effects, with psychedelics being the most likely, and opioids being the least likely, to cause visual effects (Figure 4).

Our study demonstrates the utility of analysis of large-scale narrative self-report data in the study of the phenomenology of psychoactive substances. Individual substances can cause wide ranging and highly variable visual effects both within and across individuals, so large and high-quality data sets (such as Erowid’s Experience Vaults) are needed to accurately derive the visual effect probability distributions for different substances and to have the statistical power to meaningfully compare them to one another. To our knowledge, our study is the first to quantitatively demonstrate that different psychedelic substances result in different types of visual experiences.

The approach of quantitatively studying semantic text data that we describe here represents a methodological advance, combining the strengths of natural language processing and qualitative analysis of text. We used a text embedding model (text-embedding-ada-002; OpenAI) (Greene et al., 2022) to map sentences to vectors in a way that translates semantic similarity among sentences to mathematical similarity among vectors, and we used the GPT-4 large language model (OpenAI) to automate the creation of a labeled training set for our sentence embedding vector classifier.

By combining text embedding and large language models, we identified visual effect sentences in the Erowid dataset at scale, with minimal research costs, and without concerns about human interrater reliability. This text analysis pipeline—associating all of the text units (e.g., sentences) in a dataset with embedding vectors, automating the creation of a training set, and performing classification to label all text units—has many potential applications in the analysis of psychoactive substance experience reports and in other fields.

It is possible that non-pharmacological differences between drug experiences could have contributed to differences in the visual experience reports we analyzed. First, the relative salience of different aspects of the visual effects could vary according to non-visual influences, such as cognitive alterations. In future work, the analysis method we describe in this study can be extended to non-visual aspects of the psychedelic experience to examine this possibility. Second, contextual factors and population differences also contribute to the experience reports. If different substances are statistically associated with different sets and settings associated with the experience, or different populations of experience report authors, such differences might manifest as variation in visual effects in our results. These challenges are inherent to observational studies conducted with self-reported descriptions of self-administered psychedelic substances. In future work, our analysis method could be applied to research studies in which set, setting, and population are more controlled.

In future work, our method for analyzing the visual subjective effect profiles of psychedelics can be used in conjunction with biochemical and physiological measures like receptor binding affinity, functional activation of receptors, and brain activity. In general, effects of psychedelics on conscious experience and global brain activity cannot be directly reduced to binding affinity measures. Different compounds binding to the same receptor can cause different patterns of neural activations for a variety of reasons, including affinity differences, nonlinear effects, threshold effects, and whether the binding is agonistic or antagonistic. Nonetheless, the method of quantitatively measuring psychedelics’ subjective effects that we describe here can be combined with biochemical, anatomical, physiological, and psychological measures to further investigate the actions of psychedelics in the brain and the biological bases of conscious visual experience.

Conclusion

Much remains to be understood about how activation levels of different neurotransmitter receptor types/subtypes contribute to visual perception and how psychedelics interact with these classes of receptors to affect subjective experience. Moreover, at the level of neural circuits and brain networks, the mechanisms of action of psychedelic substances are still largely unknown. Even so, patterns of receptor binding affinities or actions on individual neurons are insufficient to fully characterize the perceptual and cognitive effects of psychedelics. A compelling explanation of how psychedelics affect brain activity must not only describe their actions on individual neurons but also how changes in the patterns of activity across populations of neurons are linked to perceptual and cognitive effects.

This latter question remains a mystery, but there are several theories being developed that seek to explain the acute effects of psychedelics on perception and cognition and their long-term effects on personality, worldview, and mental health status (Carhart-Harris and Friston, 2019; Vollenweider and Preller, 2020). Our study contributes to these efforts by highlighting the need to account for the roles that multiple types of neurotransmitter receptors play in brain activity.

Finally, our study suggests that psychedelics can be used as effective tools to study basic questions in psychology and neuroscience, such as how patterns of activity in visual cortex relate to different features of visual experience. Future studies can correlate measures such as receptor binding, brain activity, and phenomenological reports across psychedelic substances to disentangle heterogenous biological contributions to the panoply of conscious visual experience.

Data availability statement

The data analyzed in this study are subject to the following licenses/restrictions: researchers partnering with Erowid Center may access the Erowid Experience Vaults database for research purposes. The Erowid Experience Vaults are publicly viewable at the following URL: https://erowid.org/experiences/. Requests to access these datasets should be directed to cmVzZWFyY2hAZXJvd2lkLm9yZw==.

Ethics statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent from the participants or participants legal guardian/next of kin was not required to participate in this study in accordance with the national legislation and the institutional requirements.

Author contributions

SN: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Supervision, Visualization, Writing – original draft, Writing – review & editing. MSh: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Visualization, Writing – review & editing. EE: Conceptualization, Data curation, Project administration, Resources, Writing – review & editing. FE: Conceptualization, Data curation, Project administration, Resources, Writing – review & editing. MSi: Conceptualization, Funding acquisition, Project administration, Resources, Supervision, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This study was supported by funds provided to Michael Silver by the University of California, Berkeley, by the UC Berkeley Center for the Science of Psychedelics, and by National Eye Institute Training Grant T32 EY007043. Additionally, Erowid Center funded the Experience Report data collection from their general budget as a 501(c)(3) non-profit educational organization.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2024.1397064/full#supplementary-material

Footnotes

1. ^Erowid.org

References

Beliveau, V., Ganz, M., Feng, L., Ozenne, B., Højgaard, L., Fisher, P. M., et al. (2017). A high-resolution in vivo atlas of the human brain’s serotonin system. J. Neurosci. 37, 120–128. doi: 10.1523/JNEUROSCI.2830-16.2016

Crossref Full Text | Google Scholar

Berg, K. A., Maayani, S., Goldfarb, J., Scaramellini, C., Leff, P., and Clarke, W. P. (1998). Effector pathway-dependent relative efficacy at serotonin type 2A and 2C receptors: evidence for agonist-directed trafficking of receptor stimulus. Mol. Pharmacol. 54, 94–104. doi: 10.1124/mol.54.1.94

PubMed Abstract | Crossref Full Text | Google Scholar

Bonhaus, D. W., Weinhardt, K. K., Taylor, M., DeSouza, A., McNeeley, P. M., Szczepanski, K., et al. (1997). RS-102221: a novel high affinity and selective, 5-HT2C receptor antagonist. Neuropharmacology 36, 621–629. doi: 10.1016/S0028-3908(97)00049-X

PubMed Abstract | Crossref Full Text | Google Scholar

Carhart-Harris, R. L., and Friston, K. J. (2019). REBUS and the anarchic brain: toward a unified model of the brain action of psychedelics. Pharmacol. Rev. 71, 316–344. doi: 10.1124/pr.118.017160

PubMed Abstract | Crossref Full Text | Google Scholar

Carhart-Harris, R. L., Muthukumaraswamy, S., Roseman, L., Kaelen, M., Droog, W., Murphy, K., et al. (2016). Neural correlates of the LSD experience revealed by multimodal neuroimaging. Proc. Natl. Acad. Sci. U. S. A. 113, 4853–4858. doi: 10.1073/pnas.1518377113

PubMed Abstract | Crossref Full Text | Google Scholar

Casey, A. B., Cui, M., Booth, R. G., and Canal, C. E. (2022). “Selective” serotonin 5-HT2A receptor antagonists. Biochem. Pharmacol. 200:115028. doi: 10.1016/j.bcp.2022.115028

PubMed Abstract | Crossref Full Text | Google Scholar

Egan, C. T., Herrick-Davis, K., Miller, K., Glennon, R. A., and Teitler, M. (1998). Agonist activity of LSD and lisuride at cloned 5HT2A and 5HT2C receptors. Psychopharmacology 136, 409–414. doi: 10.1007/s002130050585

PubMed Abstract | Crossref Full Text | Google Scholar

Erowid (1995–2023). About Erowid: mision, vision, and crew. Available at: Erowid.org/general/about/about.shtml

Google Scholar

González-Maeso, J., Weisstaub, N. V., Zhou, M., Chan, P., Ivic, L., Ang, R., et al. (2007). Hallucinogens recruit specific cortical 5-HT2A receptor-mediated signaling pathways to affect behavior. Neuron 53, 439–452. doi: 10.1016/j.neuron.2007.01.008

PubMed Abstract | Crossref Full Text | Google Scholar

Greene, R., Ted, S., Weng, L., and Neelakantan, A. (2022). New and improved embedding model [Online]. Available: https://openai.com/blog/new-and-improved-embedding-model (Accessed July 27, 2023).

Google Scholar

Griffiths, R. R., Richards, W. A., McCann, U., and Jesse, R. (2006). Psilocybin can occasion mystical-type experiences having substantial and sustained personal meaning and spiritual significance. Psychopharmacology 187, 268–283. doi: 10.1007/s00213-006-0457-5

Crossref Full Text | Google Scholar

Jensen, N. H., and Roth, B. L. (2008). Massively parallel screening of the receptorome. Comb. Chem. High Throughput Screen. 11, 420–426. doi: 10.2174/138620708784911483

PubMed Abstract | Crossref Full Text | Google Scholar

Kometer, M., Schmidt, A., Jäncke, L., and Vollenweider, F. X. (2013). Activation of serotonin 2A receptors underlies the psilocybin-induced effects on alpha oscillations, N170 visual-evoked potentials, and visual hallucinations. J. Neurosci. 33, 10544–10551. doi: 10.1523/JNEUROSCI.3007-12.2013

Crossref Full Text | Google Scholar

McInnes, L., Healy, J., Melville, J., Saul, N., and Großberger, L. (2018). UMAP: Uniform manifold approximation and projection. J. Open Source Softw. 3:861. doi: 10.21105/joss.00861

Crossref Full Text | Google Scholar

Mooseder, A., Malik, M. M., Lamba, H., Erowid, E., Thyssen, S., and Pfeffer, J. (2022). Glowing experience or bad trip? A quantitative analysis of user reported drug experiences on Erowid.org. Proc. Int. AAAI Conf. Web Soc. Media 16, 675–686. doi: 10.1609/icwsm.v16i1.19325

Crossref Full Text | Google Scholar

Nayak, S. M., Gukasyan, N., Barrett, F. S., Erowid, E., Erowid, F., and Griffiths, R. R. (2021). Classic psychedelic coadministration with lithium, but not lamotrigine, is associated with seizures: an analysis of online psychedelic experience reports. Pharmacopsychiatry 54, 240–245. doi: 10.1055/a-1524-2794

Crossref Full Text | Google Scholar

Nichols, D. E. (2000). “Role of serotoninergic neurons and 5-HT receptors in the action of hallucinogens” in Serotoninergic neurons and 5-HT receptors in the CNS (Berlin, Heidelberg: Springer Berlin Heidelberg), 563–585.

Google Scholar

Nichols, D. E. (2016). Psychedelics. Pharmacol. Rev. 68, 264–355. doi: 10.1124/pr.115.011478

Crossref Full Text | Google Scholar

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al. (2011). Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830.

Google Scholar

Pieri, L., Keller, H. H., Burkard, W., and Da Prada, M. (1978). Effects of lisuride and LSD on cerebral monoamine systems and hallucinosis. Nature 272, 278–280. doi: 10.1038/272278a0

Crossref Full Text | Google Scholar

Preller, K. H., Burt, J. B., Ji, J. L., Schleifer, C. H., Adkinson, B. D., Stämpfli, P., et al. (2018). Changes in global and thalamic brain connectivity in LSD-induced altered states of consciousness are attributable to the 5-HT2A receptor. eLife 7:e35082. doi: 10.7554/eLife.35082

PubMed Abstract | Crossref Full Text | Google Scholar

Quednow, B. B., Kometer, M., Geyer, M. A., and Vollenweider, F. X. (2012). Psilocybin-induced deficits in automatic and controlled inhibition are attenuated by ketanserin in healthy human volunteers. Neuropsychopharmacology 37, 630–640. doi: 10.1038/npp.2011.228

PubMed Abstract | Crossref Full Text | Google Scholar

Richardson, L. (2007). Beautiful soup documentation. Available at: https://beautiful-soup-4.readthedocs.io/en/latest/

Google Scholar

Sanz, C., Zamberlan, F., Erowid, E., Erowid, F., and Tagliazucchi, E. (2018). The experience elicited by hallucinogens presents the highest similarity to dreaming within a large database of psychoactive substance reports. Front. Neurosci. 12:7. doi: 10.3389/fnins.2018.00007

PubMed Abstract | Crossref Full Text | Google Scholar

Shulgin, A. T., and Shulgin, A. (1991). Pihkal: a chemical love story. Transform Press. Available at: https://books.google.com/books?id=O8AdHBGybpcC

Google Scholar

Shulgin, A., and Shulgin, A. (1997). Tihkal: the continuation. Transform Press. Available at: https://books.google.com/books?id=jl_ik66IumUC

Google Scholar

Vollenweider, F. X., and Preller, K. H. (2020). Psychedelic drugs: neurobiology and potential for treatment of psychiatric disorders. Nat. Rev. Neurosci. 21, 611–624. doi: 10.1038/s41583-020-0367-2

Crossref Full Text | Google Scholar

Wallach, J., Cao, A. B., Calkins, M. M., Heim, A. J., Lanham, J. K., Bonniwell, E. M., et al. (2023). Identification of 5-HT2A receptor signaling pathways associated with psychedelic potential. Nat. Commun. 14:8221. doi: 10.1038/s41467-023-44016-1

PubMed Abstract | Crossref Full Text | Google Scholar

Zamberlan, F., Sanz, C., Martínez Vivot, R., Pallavicini, C., Erowid, F., Erowid, E., et al. (2018). The varieties of the psychedelic experience: a preliminary study of the association between the reported subjective effects and the binding affinity profiles of substituted phenethylamines and tryptamines. Front. Integr. Neurosci. 12:54. doi: 10.3389/fnint.2018.00054

PubMed Abstract | Crossref Full Text | Google Scholar