Navigating the Road Ahead: Using Concept Mapping to Assess CTSA Program Goals

Cathleen T. Kane ^1* William Michael Trochim ² Haim Yehuda Barr ³ Andie Vaught ⁴ Heather L. Baker ⁴ Munziba T. Khan ⁴ Robin Marian Wagner ⁴ Keith Alan Herzog ⁵ Kristi Holmes ⁵ Jamie Mihoko Doyle ⁴

• ¹ Institute of Clinical & Translational Science,School of Medicine, New York University, New York, United States
• ² Cornell Jeb E. Brooks School of Public Policy, Cornell University, Ithaca, New York, United States
• ³ Department of Statistics, College of Liberal Arts and Sciences, University of Connecticut, Storrs, Connecticut, United States
• ⁴ National Center for Advancing Translational Sciences (NIH), Bethesda, Maryland, United States
• ⁵ Northwestern University Feinberg School of Medicine, Northwestern University Clinical and Translational Sciences (NUCATS) Institute, Chicago, United States

Evaluating large-scale programs designed to transform public health demands innovative approaches for navigating their complexity and scope. The Clinical and Translational Science Awards (CTSA) Program, supported by the NIH's National Center for Advancing Translational Sciences (NCATS), represents a significant national investment with over 60 sites or "hubs" spread across the country. Assessing an initiative of this size and complexity requires measures that balance local flexibility with national coherence. To that end, this study used concept mapping, a mixed-methods approach integrating qualitative brainstorming and sorting with quantitative multidimensional scaling and cluster analysis. Participation across the CTSA was unprecedented. Over 100 evaluation stakeholders were engaged across the network of hubs, leading to the identification of more than 80 measures, which were then organized into thematic clusters that reflect a logical progression from CTSA activities to outcomes and impacts, as well as critical foundational factors such as collaboration and education. The results also revealed a pattern where long-term impacts were ranked among the highest in importance but among the lowest in feasibility, particularly for measures tied to the Translational Science Benefits Model (TSBM), a new evaluation framework gaining popularity across the CTSA. The findings of this study underscore the efficacy of concept mapping in incorporating wide-ranging perspectives, identifying areas of consensus, and informing leadership in the development of unified, data-driven evaluation frameworks -such as TSBM and/or a CTSA logic model-critical to maximizing the CTSA's transformative potential for public health.