Xavier Bronlet ^*

• Integral Transpersonal Institute, Milan, Italy

The emergence of Artificial Intelligence and widespread availability of Large Language Model open the door to text analysis at scale leveraging on complex classification instructions. This case study explores the usage of Large Language Models to measure Ego Development at scale and establish a methodology that can be applied to other classification instructions. Ego consists of the traits that influence how a person perceives and engages with the world, while Ego development is a crucial aspect of adult personality growth, influencing behaviors and decisions in both personal and professional contexts. This case study investigates the agreement between expert and automated classifications of Ego Development STAGES, aiming to evaluate the potential of automation in this domain leveraging on Large Language Models. Cohen's Kappa statistic has been used to measure the agreement between classifications made by experts and the automated classiification. The inter-rater agreement yielded a weighted Kappa value of 0.779, indicating a substantial level of agreement that is statistically meaningful and unlikely to be due to chance. Notably, we observed low variability in aggregated values, demonstrating that the automated process functions effectively at scale. The robustness in aggregated data is particularly evident when calculating ego development scores for individuals, groups, corporate units, and entire corporations. This capability underscores the utility of the automated system for high-level evaluations and decision-making leveraging on a solid indicator.Our findings emphasize the importance of continuous improvement in the automated classification process through algorithm refinement. Future research should address this area and focus on enhancing the contextual adaptability of the system to better capture nuanced judgments. The rapid evolution of Large Language Models may support the refinement process. While the classification system developed in this case study shows promise, targeted enhancements may help to achieve a level of accuracy and reliability that aligns with expert evaluations for single sentences. The methodology used in this case study appears to be useful to support other evaluation at scale that leverage on Large Language Models using other classifications' maps.