AUTHOR=Kievit Rogier , Frankenhuis Willem E., Waldorp Lourens , Borsboom Denny TITLE=Simpson's paradox in psychological science: a practical guide JOURNAL=Frontiers in Psychology VOLUME=4 YEAR=2013 URL=https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2013.00513 DOI=10.3389/fpsyg.2013.00513 ISSN=1664-1078 ABSTRACT=

The direction of an association at the population-level may be reversed within the subgroups comprising that population—a striking observation called Simpson's paradox. When facing this pattern, psychologists often view it as anomalous. Here, we argue that Simpson's paradox is more common than conventionally thought, and typically results in incorrect interpretations—potentially with harmful consequences. We support this claim by reviewing results from cognitive neuroscience, behavior genetics, clinical psychology, personality psychology, educational psychology, intelligence research, and simulation studies. We show that Simpson's paradox is most likely to occur when inferences are drawn across different levels of explanation (e.g., from populations to subgroups, or subgroups to individuals). We propose a set of statistical markers indicative of the paradox, and offer psychometric solutions for dealing with the paradox when encountered—including a toolbox in R for detecting Simpson's paradox. We show that explicit modeling of situations in which the paradox might occur not only prevents incorrect interpretations of data, but also results in a deeper understanding of what data tell us about the world.