Opportunities and Challenges of Extracting Values in Autobiographical Narratives

Fischer, Ronald; Karl, Johannes; Fetvadjiev, Velichko; Grener, Adam; Luczak-Roesch, Markus

doi:10.3389/fpsyg.2022.886455

ORIGINAL RESEARCH article

Front. Psychol., 02 August 2022

Sec. Personality and Social Psychology

Volume 13 - 2022 | https://doi.org/10.3389/fpsyg.2022.886455

This article is part of the Research Topic The Interconnectedness of Personality and Language View all 10 articles

Opportunities and Challenges of Extracting Values in Autobiographical Narratives

$\nRonald Fischer,$ Ronald Fischer^1,2^*

Johannes Karl^1,3

Velichko Fetvadjiev^4,5

Adam Grener⁶

Markus Luczak-Roesch⁷

¹School of Psychology, Victoria University of Wellington, Wellington, New Zealand
²Unidade de Neurociências Cognitiva e Neuroinformática (Cognitive Neuroscience and Neuroinformatics Unit), Instituto D'Or de Pesquisa e Ensino, Rio de Janeiro, Brazil
³School of Psychology, Dublin City University, Dublin, Ireland
⁴Faculty of Social and Behavioural Sciences, University of Amsterdam, Amsterdam, Netherlands
⁵North-West University, Potchefstroom, South Africa
⁶School of English, Film, Theatre, Media Studies, and Art History, Victoria University of Wellington, Wellington, New Zealand
⁷School of Information Management, Victoria University of Wellington, Wellington, New Zealand

We report three studies in which we applied a value dictionary to narratives. Our objective was to test a theory-driven value dictionary for extracting valuable information from autobiographical and narrative texts. In Studies 1 (N = 106) and 2 (N = 152), participants wrote short autobiographical narratives and in Study 3 (N = 150), participants wrote narratives based on ambiguous stimuli. Participants in all three studies also completed the Portrait Value Questionnaire as a self-report measure of values. Overall, our results demonstrate that it is possible to extract value-relevant information from these narratives. Extracted values from autobiographical narratives showed average correlations of 0.07 (Study 1) and 0.12 (Study 2) with self-reports compared to an average correlation of 0.01 for the extracted values from implicit motive tasks (Study 3). The correlations with self-reports were in line with previous validation studies. The most salient values in narratives diverged somewhat, with a stronger emphasis on achievement values compared to self-reports, probably due to the nature of salient episodes within one's life that require demonstrating success according to social standards. Benevolence values were consistently most important in both self-ratings and text-based scoring. The value structure emerging from narratives diverged from the theoretically predicted structure, yet broad personally vs. socially focused value dimensions were qualitatively discernible. We highlight opportunities and challenges for future value research using autobiographical stories.

Introduction

The study of values is fundamental for understanding the goals and motivations of individuals because values refer to trans-situationally important goals in a person's life that guide the thoughts and behavior and provide standards for evaluating actors, events, and situations (Schwartz and Bilsky, 1987). Some of the earliest attempts to identify human values have been based on linguistic data (Allport and Odbert, 1936), but the study of values have been dominated by self-report measures, most notably those based on Schwartz's seminal theory of values (Schwartz, 1992; Schwartz et al., 2001, 2012). Using these surveys, values have been shown to relate to personality traits, attitudes, moral issues, and behavior across a range of contexts (Boer and Fischer, 2013; Fischer and Boer, 2015; Fischer, 2017; Roccas and Sagiv, 2017). The reliance of value research on self-reports certainly limits the opportunities for analyzing values at a distance, across time and space, which is increasingly becoming possible through data that are being produced in social media contexts (Ponizovskiy et al., 2020; Boyd et al., 2021). There have been manual coding approaches in which human coders analyzed texts at a deep level to infer salient values expressed by the authors (Suedfeld et al., 2010, 2011; Portman, 2014). Ponizovskiy et al. (2020) recently developed a theory-based value dictionary that overcomes two major limitations of such previous indirect measures of values, that is, the lack of validated dictionaries and the effort that is involved in manual coding of texts. This theory-based dictionary has been developed and validated in the context of social media texts, using various types of blogs, Facebook updates and value-focused texts are produced by MTurk workers and therefore offers a promising tool for further value research.

We expand this study in a number of ways. First, we tested to find whether values can be extracted from short autobiographic stories with this recently developed theory-based dictionary. Autobiographical research has suggested that individuals build coherent narratives around their values and beliefs (McAdams et al., 2004; McAdams and Pals, 2006). Therefore, it is worth exploring to what extent salient values of individuals can be extracted from autobiographical stories.

Second, an older tradition of research attempted to extract person-relevant information from individuals' responses to ambiguous stimuli as an indication of implicitly held motives (Schultheiss and Pang, 2007). Previous studies suggested that implicit motive measures do not converge strongly with explicit value ratings (Hofer et al., 2006). We have been examining whether values extracted from stories elicited using implicit assessment techniques correlate with self-reports to a comparable degree as values extracted from autobiographical stories. Using both autobiographical stories and implicit stories based on ambiguous stimuli material, we can provide some benchmarks on possible text-based value analyses with a theory-based dictionary.

A final contribution is a closer examination of the values and their associations as identified through textual analysis. To what extent are values organized in the same circular space based on motivational compatibilities and conflicts as has been described by Schwartz? Recent studies examining values as situational states (instead of more stable trait-like structures as is common in value research) have suggested that values may be organized differently when considered in the context of specific situations compared to values as relevant to life in general (Skimina et al., 2021). Narratives or stories are often about the descriptions of specific events; hence, they may be more like states than traits. At the same time, the request to reflect on salient episodes within one's life may motivate individuals to provide more abstract reflections on specific events which may be compatible with value traits. Hence, it is theoretically interesting to explore the structure of values in the context of personally meaningful life events.

Schwartz's Nearly Universal Theory of Values

Schwartz (1992) presented a descriptive theory of human values based on their motivational compatibilities and conflicts. Individual values are thought to be motivational goals that show mutual conflicts and congruence which result in a two-dimensional structure when considering the overall levels of compatibilities and conflicts across individuals. The first dimension has been called Openness to Change vs. Conservation and contrasts an orientation toward hedonistic goals and pursuit of independent thoughts and actions vs. an endorsement of goals that highlight preserving the social and cultural traditions and order, restraining personal actions not to upset social conventions, and a concern with the security of one's family and country. The second dimension was labeled self-transcendence vs. self-enhancement. This dimension differentiates more prosocial and altruistic concerns with the wellbeing of close and distant others, maintaining fairness within one's society and a concern with the state of the social and natural environment in general from more self-centered concerns that focus on advancing one's power, wealth, and status as well as demonstrating one's abilities and dominance in ways that conform to socially acceptable standards.

Figure 1 offers a conceptual representation of the two major dimensions and the 10 value types that are encapsulated within them. Finer theoretical distinctions are possible (Schwartz et al., 2012), but here we keep with the more common focus on the 10 value types within the two major dimensions (Fischer, 2017; Schwartz, 2017). Specifically, Benevolence captures the importance of caring about the wellbeing of individuals emotionally close to the person; Universalism captures the appreciation, tolerance, and protection of the wellbeing of all people and nature, independent of the emotional closeness; Conformity values emphasize the restraint of actions or impulses that may upset others or violate social norms and expectations; Tradition values relate to the preservation, respect, and commitment to norms and customs of one's traditional culture or religion; Security concerns the safety and stability of one's close relationships as well as society; Power values emphasize control and dominance of other people and resources; Achievement values focus on demonstrating success and competence according to socially accepted standards; Hedonism values capture the striving for pleasure and gratification of personal desires; Stimulation values highlight excitement, seeking challenges, and novelty in life and finally, Self-Direction values emphasize independence in both thought and action.

FIGURE 1

Figure 1. Schematic representation of the Schwartz Value Theory. SD, Self direction; ST, Stimulation; HE, Hedonism; AC, Achievement; PO, Power; SE, Security; TR, Tradition; CO, Conformity; BE, Benevolence; UN, Universalism.

This value structure has been transformed into a number of self-report measures (Schwartz, 1992; Schwartz et al., 2001, 2012; Lindeman and Verkasalo, 2005; Lee et al., 2008; Verkasalo et al., 2009; Schwartz and Cieciuch, 2021), which appear to capture these two major dimensions in line with the theory. Demonstrating the conceptual validity, the two major dimensions have been identified using reaction time and neuroscience methodologies (Pakizeh et al., 2007; Brosch et al., 2011; Leszkowicz et al., 2017), and can also be measured in a picture format (Döring et al., 2010).

Predictions based on the theory have so far mostly been tested with data that focus on differences between individuals. On the other hand, a recent set of studies focusing on values as motivational goals within specific situations, that is, value states instead of more trans-situationally stable value traits, has suggested that values expressed within specific situations may relate to other values differently when compared to their relationship across situations and between individuals (Skimina et al., 2021). In other words, previous research has focused on value importance ratings and compared these across individuals, effectively generalizing across situations.

When focusing on specific situations, slightly different associations may emerge because different combinations of values may be salient within a situation to achieve a goal. Individuals are able to pursue two values that are conceptually conflicting in different everyday situations, which may mean that across situations, both values are salient. A good example is the importance of self-enhancement values of achievement within work or academic settings, but the importance of self-transcendence values of benevolence in family settings. In line with the reasoning, Skimina et al. (2021) reported that Power and Hedonism values are positively correlated when examining individual differences across situations as predicted by the theory, but within specific situations, the two value types are not simultaneously activated, leading to an orthogonal relationship between these two value types in situ. As we are focusing on narratives, it becomes relevant to ask whether descriptions of specific events may result in different structures. Such distinctions are conceptually important as we describe in the next section.

Narratives, Autobiographies, and Implicit Values

Understanding a person requires paying attention to different levels or principles of what it means to be a person (McAdams, 1995; McAdams and Pals, 2006). At the most basic level, it is possible to describe individuals as belonging to the human species that emphasizes our communalities. At the most specific level, it is possible to focus on the unique aspects of each person's life. At this level, we can distinguish individual elements of the biography and identity one individual from all other individuals. Despite these differences in abstraction and detail, work on life stories has suggested that these different levels and perspectives are interlinked and integrated. At the lowest level, according to McAdams, are basic biologically driven predispositions. The most widely used descriptions at this level use personality traits, such as the Big Five or the approach vs. avoidance motivational systems (Corr and Cooper, 2016; Soto and John, 2017). At an intermediate level in this framework are aspirations, values, and beliefs that link basic biological predispositions to the wider social and cultural environment. Values are often thought to be operating at this intermediate level. At the most person-specific level, there are rich lived stories that weave together the strands across these different levels, linking basic predispositions to the goals, motivations, and values encountered byan individual in specific situations. From this perspective, autobiographical life stories build on and explicate the individual's choices in the past. These stories are meaningful from a person-focused level of analysis because they provide unity to a person's experience, and provide purpose and meaning.

These stories rely on verbal descriptions of events in a person's life. A person has choices in describing these events, drawing upon a vast vocabulary. The specific choices of words used to describe one's life story can be analyzed and provide insights into the major concerns and preoccupations of the individual. Linguistic choices are behavioral expressions that are worth exploring, an idea that traces back to the linguistic hypothesis first sketched in 1884 by Galton (1949). Such a cognitive approach has been extensively used within the personality domain: language can provide information on what a person attends to, what they associate with objects or events and how they interpret their world, and thereby allowing a careful listener to gain relevant information on the personality of the person (Chung and Pennebaker, 2008).

We propose that the analysis of linguistic choices can provide meaningful information on the values that implicitly guide the individuals. Extracting value information adds an additional layer as values are evaluative standards, that is, they go beyond descriptions of what a person attends to or has done (as typically presumed in personality trait research) and provide insights into how individuals interpreted situations, individuals, and events in their lives. By definition, values are evaluation standards that help individuals make sense of their world (Schwartz and Bilsky, 1987). To the extent that values also reflect motivational goals, extracting information on values can theoretically provide a deeper layer of a person's psychological make-up. By analyzing the choices of words out of a universe of words when describing an event in one's life, it should be possible to derive information about both the evaluative standards and motivations of an individual, that is, their salient values. This is an extension of previous lexical analyses of values of speeches, newspaper texts, and blogs (Bardi et al., 2008; Portman, 2014; Boyd et al., 2015; Ponizovskiy et al., 2020) to the context of autobiographical narratives. We believe that the study of values in autobiographies has much potential because autobiographical narratives have a rich conceptual foundation as a source of personal information (McAdams, 1995).

Our approach shares some similarity with an older tradition focusing on implicit motives (Schultheiss and Pang, 2007). Recent psychometric work in this tradition has also focused on narratives, but in response to standardized ambiguous material to elicit salient motives (McClelland et al., 1989; Schultheiss and Pang, 2007). The basic idea within this research tradition is that the presentation of standardized, but ambiguous stimuli will arouse specific motivational content of importance to the individual and results in a projection of this personally important content onto the generation of the narrative. It is possible to use both content-based coding options (Winter, 2016) and dictionaries (Schultheiss, 2013) to extract salient motives. Although motives are not identical to values, they are conceptually related. Furthermore, an alignment of implicit motives and explicit values appears to increase the wellbeing (Hofer et al., 2006).

This related line on implicit motives offers an interesting additional perspective on lexical analyses of values within narratives. Studying both personally experienced autobiographical stories and stories elicited from ambiguous stimulus materials, these two methods provide an interesting cross-check on our ability of analyzing word choices to index value information about an individual. In summary, we will use both autobiographical life stories and stories produced in response to ambiguous stimuli to examine how well they correlate with explicit self-ratings on value scales.

Dictionary Approaches to Value Measurement

The focus so far has been on the linguistic choices, that is, the verbal behavior of the actor (Ponizovskiy et al., 2020). These behaviors need to be coded in objective, valid, and reliable ways. Manual coding procedures for value content are available, but are relatively subjective and highly labor-intensive (Suedfeld et al., 2010, 2011; Portman, 2014). A first value dictionary was very brief and included only 3 value terms per value type (Bardi et al., 2008). This will lead to very sparse data matrices when based on naturally occurring language. The most comprehensive dictionary to date has over 1,000 entries and was developed using both theoretical criteria and previous value lists (Ponizovskiy et al., 2020). This list was validated against different types of blogs, text containing fiction from a large American English corpus of texts, Facebook status updates, and short essays in which participants had to describe their values and behavior. The validation was primarily focused on internal relations between the individual value terms. Overall, the higher order motivational domains were well replicated across the five different data sets. An analysis of the position of the 10 value types within a two-dimensional representation suggested greater variability. Notably, universalism values tended to merge with Conservation values and Hedonism and Benevolence values tended to be located quite close to each other, suggesting greater motivational complexity than predicted by Schwartz's model.

A subset of participants from Ponizovskiy et al. (2020) Facebook study and essay study provided self-ratings of their values using the Schwartz Value Survey (Schwartz, 1992). The results showed that 7 out of the 10 correlations between self-reports and value dictionary-derived scores were significant in the essay study and 6 out of the 10 in the Facebook study. The correlations between conceptually matched scores varied between 0.10 and 0.16 in the Facebook study and between 0.12 and 0.31 in the essay study. Maybe this latter finding is not surprising given that the task for the essay was to describe one's values. These analyses therefore suggest it is possible to extract value information with this theory-based value dictionary from texts produced by individuals in different formats and platforms.

To the extent that such analyses are part of an analysis from a distance, it may be interesting to describe the most salient values of an individual or a group. Within self-report measures, there is a relatively consistent hierarchy of values in terms of importance across representative, student, and teacher samples (Schwartz and Bardi, 2001): Benevolence values are typically ranked most important, followed by self-direction values and then universalism values. Similarly, these values and some of the achievement values show the greater consensus within nations and the weakest cross-cultural differences (Fischer and Schwartz, 2011). This suggests that there is a relatively universal ordering of values in terms of their importance in people's life and it would be informative to see whether values extracted in autobiographical or ambiguous stories (which are thought to activate salient motives) show a similar pattern on average.

Across three studies that differed in the content and format of freely generated texts, we aimed to examine how well we can extract personal values from these texts using a pretested theory-driven value dictionary. We addressed the following research questions: First, what are the overall value priorities extracted from texts, and to what extent do they agree with value priorities derived from responses to standard self-rating questionnaires? Second, what is the structure of values derived from texts and to what extent does it correspond to the theoretical model and to the theory-based? Third, to what extent are value patterns consistent across texts generated in different narrative tasks?

Study 1

Our first study aims to test whether value-relevant information can be extracted from two short autobiographical stories using a recently proposed theory-based value dictionary. This is the first test to check whether value information extracted from two short autobiographical statements using this dictionary correlates with value self-reports. Furthermore, we compare the value importance scores extracted from short autobiographical stories with ordering of self-rated values. We also aim to examine the internal structure of values within autobiographical narratives, that is whether value terms in relation to this specific autobiographical event cluster as expected by the theory.

Methods

We recruited young adults enrolled in an introductory psychology course in New Zealand. A total of 106 individuals participated. Mean age was 19.6 years (SD = 3.32, Min = 17, Max = 46) and 31 participants (29.2%) were males. Thirteen participants (12.3%) were non-native English speakers. All individuals received partial course credit in return for participating in this study. The study was conducted online in 2019. The study was approved by the School of Psychology Human Ethics Committee under delegated authority of Victoria University of Wellington's Human Ethics Committee (ID0000023640).

Procedure

We gave individuals two tasks based on the life narrative writing tasks (Rubin et al., 2009). We always first asked individuals to write about a positive experience: “Think of an episode in your life which you consider a POSITIVE EXPERIENCE, and which has influenced you. Describe in detail WHAT has happened, WHERE, WHEN and WHO was involved. Describe what you THOUGHT or FELT.” We then asked about a challenging episode: Looking back over your whole life so far, select an episode which you would consider a GREAT CHALLENGE that you had to face. Describe in detail WHAT has happened, WHERE, WHEN and WHO was involved. Describe how you ADDRESSED the challenge. Describe what you THOUGHT or FELT.” We always encouraged individuals to first think for a minute before they started writing. We also requested full sentences and text that avoided slang. As a guide, we suggested individuals to write between 20 and 30 lines.

Self-reported values were measured with an adapted gender-neutral version of the Portrait Value Questionnaire (Schwartz et al., 2012). The scale consists of 57 items that describe individuals with specific value priorities. Participants have to respond whether the statements involving various value-driven actions describe them using a scale varying from 1 (Not like me at all) to 6 (Very much like me). Example items were “It is important to me to form my views independently” and “It is important to me that my country is secure and stable.” We used the 10 value types for the present study and calculated Cronbach's alpha using the raw data. Refer to Table 1 for more information. Achievement, Stimulation and Hedonism values had reliabilities below 0.70 [for a discussion of reliability issues with this measure, refer to Schwartz and Cieciuch (2021)]. To examine the theoretical structure, we conducted a multidimensional scaling analysis with the smacof package (Mair et al., 2021) and Euclidean distances based on Pearson's correlations, ordinal data structure, and extracting two dimensions. Compared to the theory-predicted positions (Bilsky et al., 2011), our data showed acceptable conceptual similarity (congruence = 0.88) in line with minimum standards for conceptual replication (Fischer and Fontaine, 2011; Fischer and Karl, 2019). Since there is an ongoing debate in the literature about the need to ipsatize value scores prior to analysis (Fischer, 2004; Rudnev, 2021), we also computed ipsatized values by subtracting the overall value mean from each individual value item and then averaged these ipsatized scores per value type.

TABLE 1

Table 1. Mean scores for the text-based and rating-based value scores.

Dictionary

We used the refined value dictionary developed by Ponizovskiy et al. (2020). The refined value dictionary contains theory-driven selections of adjectives, verbs, and nouns that have been shown to correlate with value priorities across a range of textual sources. The refined dictionary contains 1,068 entries. The values are not equally represented in the dictionary: the smallest word set captures Security values (85 words) and the largest word set is the Self-direction value list (140 words; Ponizovskiy et al., 2020).

Data Treatment

To facilitate analyses, we first converted the texts into a format that facilitates subsequent analyses, using tokenization and part-of-speech (PoS) tagging from the R version of spaCy (Benoit et al., 2020), a wrapper around the natural language processing library in Python (Van Rossum and Drake, 2011). Tokenization automatically splits the texts into single word units, and PoS tagging annotated each word with word type, such as noun, adjectives, and verbs. We then matched words in each person's text with the dictionary. If a person used a word from the dictionary, this was counted as a hit. All hits were summed per dictionary value category for each person. No spelling correction or lemmatization was used. Similar to value ratings, it is possible to use an adjusted value score which adjusts the frequency for each value type for the overall number of value terms used by the participant. Therefore, this score (which we refer to as ipsatized text-based value score henceforth) could be interpreted similarly to ipsatized rating scores. It is also possible to compute an adjusted score as reflecting the overall text length (dividing each text-based value score by the overall text length). We do not report these scores here, because the results were qualitatively very similar to the ipsatized scores, which are more commonly reported in the value literature. All analyses are based on Pearson's product moment correlations.

To examine the overall content of both writing prompts, we fitted a bi-term topic model (Yan et al., 2013) using the BTM package (Wijffels, 2021a) in R on each separate text. Initially, we used UDPipe (Wijffels, 2021b) to tokenize and annotate the text for each participant. For the analysis, we focused on the lemmas of verbs, nouns, and adjectives. To achieve a robust estimation of term -topic relationship, we repeated the Gibbs sampling process 1,000 times allowing the extraction of up to eight topics. The graphical representations of the extracted topics are available in the Supplementary Material.

Results

First, we examined basic descriptive information. On average, participants produced 490 words (SD = 4.25; min = 190, max = 1,150). This is above the average word count in the essay study in the original validation study of the value dictionary (206 words), and our lower limit is aligned with the lower limit of 200 words set in the Facebook validation study (Ponizovskiy et al., 2020).

Second, on average, participants used 30.52 words from the value dictionary (SD = 10.4, min = 8, max = 67, median = 28.0). This means that on average, 6.38% of the terms used were entries in value dictionary used (SD = 1.69, min = 2.34, max = 11.34, and median = 6.21). This suggests that value terms are relatively common in autobiographical stories.

Third, focusing on the overall value profiles, the most important value type using the rating scale was Benevolence, followed by Hedonism and Universalism (using both raw and ipsatized scores). When using the text-based value scores, the most important values were Benevolence, followed by Achievement and Self-Direction (for both raw scores and ipsatized scores). The least important value types in ratings were Power, Tradition, and then Conformity. For the text-based value scores, the least important value types were Tradition, Security, and then Power. This suggests that most and least important values differed somewhat across rating vs. text-based analyses. The correlation of the value profiles with the raw data was 0.62.

To examine the similarity in greater detail at the level of individual differences, we next analyzed the similarity between the self-reports and the text-based value scores. Table 2 shows the summary of the motivation-matched correlations and Table 3 shows the full correlation table across methods. A few results are noteworthy. First, the average correlations suggested the strongest average relations for the raw text scores with the raw rating scores (mean r = 0.07). The next strongest average correlation was between ipsatized rating scores and raw text-based scores (mean r = 0.04). There have been strong arguments for and against ipsatization to strengthen value profiles (Schwartz, 1992; Schwartz et al., 2001; Fischer, 2004; He and van de Vijver, 2015; Rudnev, 2021); our data showed that raw scores matched with motivational content for both self-ratings and text-based data correlated on average slightly stronger (Δr = 0.03). Second, these average correlations were similar to the average correlation between text-based and rating-based value scores in Ponizovskiy et al. (2020) when using Facebook status updates as input (mean correlation was 0.07). The average correlation for texts in which students had to describe their values was higher (mean r = 0.16), as might be expected. Therefore, our mean correlations were weak, but in line with previous research that used text that is not explicitly value focused. Third, the highest correlations for matching value types were observed for Tradition (r = 0.16), followed by Conformity (r = 0.15) and Power (r = 0.13), when using the raw scores for both types of variables. The pattern was identical when examining the correlations between raw ratings and adjusted scores using text-based values. When using ipsatized ratings, the associations were somewhat different. For ipsatized ratings with text-based scores, the strongest correlation now was observed for Hedonism (r = 0.24 and 0.08, for raw and ipsatized text-based scores, respectively), followed by Power (r = 0.12 and 0.11, for raw and ipsatized text-based scores, respectively) and Tradition (r = 0.07 and 0.10 for raw and ipsatized text-based scores, respectively). Hence, Tradition and Power showed relatively consistent positive correlations, irrespective of the specific type of score adjustment used. A final observation is that the off-diagonal elements in a majority of cases were of similar or larger absolute value, suggesting that the text-based value scores correlated more strongly with other value types rather than the intended value type. In some cases, these larger values were found with adjacent values, in some cases with motivationally opposing values (which is appropriate as long as the correlation is negative, indicating that motivationally conflicting values are negatively correlated with each other), but a considerable number of absolute correlations on the off-diagonal were larger for motivationally incongruent values. In other words, no clear pattern indicating discriminant validity was observable.

TABLE 2

Table 2. Overall similarity (correlation) of matching value types in Study 1.

TABLE 3

Table 3. Correlation matrix across types of score adjustment in Study 1.

To provide a more holistic examination of the similarity of the correlation matrices, we computed Mantel correlations to examine the overall similarity of the matrices (refer to Table 4). As can be seen there, the raw rating correlation matrix and the raw text-based value dictionary matrix showed a significant and positive correlation, compared to the adjusted scores. Therefore, the raw data-based correlation matrices showed some overall and significant similarity in the correlation patterns.

TABLE 4

Table 4. Overall similarity of the correlation matrices (using Mantel test) in Study 1.

Finally, we conducted a multidimensional scaling analysis of the raw data of text-based scores, using the smacof package (Mair et al., 2021). We specified ordinal data structure and plotted two dimensions. Stress 1 for this two-dimensional solution was 0.13, which is comparable to previous MDS stress values. In order to interpret the matrix, it is necessary to rotate the empirical structure to similarity with the expected structure. We used the theory-predicted positions as the target matrix (Bilsky et al., 2011). Compared to ideal theory-based matrix, the congruence coefficient of the solution was 0.878 (alienation coefficient: 0.479), suggesting a conceptual similarity in structure (Fischer and Fontaine, 2011; Fischer and Karl, 2019). When focusing on the individual dimensions, the estimates of the individual dimensions suggested lower replicability; for dimension 1, we found r = 0.41 and for dimension 2, r = 0.46. Examining the position of the individual value types (Figure 2), achievement values were positioned in between power values toward the conservative value end (positive end of dimension 1). Achievement and Power were located nearly at opposite ends of dimension 1, which suggests lower motivational congruence based on the extracted value scores in these autobiographical stories. Security values emerged close to Hedonism values, which is theoretically unexpected. Stimulation and self-direction values emerged in the center of the two-dimensional representation. In line with the theory, tradition and conformity values emerged close together, similar to benevolence and universalism values.

FIGURE 2

Figure 2. The two-dimensional representation of raw text-based value scores after rotation to maximal similarity with theoretical predictions in Study 1. SD, Self direction; ST, Stimulation; HE, Hedonism; AC, Achievement; PO, Power; SE, Security; TR, Tradition; CO, Conformity; BE, Benevolence; UN, Universalism.

Discussion

Our first exploration of the value-based scores in two short autobiographical stories suggested mixed evidence. On the one hand, the average correlation was similar to the correlations observed in Facebook updates reported in the original validation study (Ponizovskiy et al., 2020), which is encouraging. The raw data matrices also showed some significant similarity overall and the individual value correlations suggested comparable patterns to previous studies. The structure showed reasonable similarity with the overall expected structure, even though individual dimensions were not all positioned as expected by theory. On the other hand, the salient values in this sample diverged between the rating and text-based scores, although the profiles were somewhat similar. The correlation patterns showed that there were relatively low levels of convergent and discriminant validity.

One interesting observation was the emergence of achievement values as one of the most salient values in the autobiographical narratives. Participants in this study wrote about a positive and a challenging experience. It seems that these achievement values are among the salient values that occur more when individuals describe and reflect on these events, which appear valid as challenges often involving the need for overcoming obstacles that show performance according to social standards (achievement values according to Schwartz's definition). Hence, the divergence of the value priorities between self-ratings and values extracted from these narratives may be explicable by the nature of the autobiographical memories that individuals were asked to share.

Of relevance to value researchers are the ipsatized scores that did not improve the patterns in line with theory. When examining the overall patterns via Mantel correlations, the ipsatized matrices showed qualitatively divergent patterns of correlations between ratings and text-based scores. Given the debates about the interpretation of adjusted scores and their relative performance compared to the raw data in the current data, it seems more prudent to work with raw scores, both for ratings and for the text-based value scores, because raw scores do not entail the same assumptions as adjusted scores.

However, the first study was based on a sample of 106 individuals which was asked to describe only two autobiographical stories. These two stories were also particularly focused on emotionally salient events, which may not necessarily strongly activate values. It would be useful to extend these tasks and examine whether a stronger pattern can be observed when asking individuals to provide autobiographical stories at a broader level.

Study 2

This study was intended to replicate and extend the first study. First of all, we recruited a larger sample and asked participants to report a larger number of autobiographical stories. Instead of focusing individuals on stories with a specific emotional content, we asked for a larger number of general autobiographical memories that are salient for individuals. Therefore, the goal of the second study was to extend Study 1 by using a more elaborate autobiographical narrative task.