94% of researchers rate our articles as excellent or good
Learn more about the work of our research integrity team to safeguard the quality of each article we publish.
Find out more
Alejandra Cid-Vega1,2*
Chynere Best3
Kendall Pfeffer1,4Manaswi Sangraula3Janus Wong1Wilfred Gwaikolo3James Caracoglia3Sauharda Rai3
Adam D. Brown1,5*
Brandon Kohrt3- 1Department of Psychology, The New School for Social Research, New York, NY, United States
- 2Department of Psychology, University of Miami, Coral Gables, FL, United States
- 3Department of Psychiatry and Behavioral Health, Center for Global Mental Health Equity, George Washington University, Washington, DC, United States
- 4World Trade Center Mental Health Program, Icahn School of Medicine at Mount Sinai, New York, NY, United States
- 5Department of Psychiatry, New York University School of Medicine, New York, NY, United States
Data quality is critical in mental health research, yet there is considerable variability in the training and preparation of those involved in data collection. Individuals with limited research training often play a central role in collecting data and, to date, no systematic means of evaluating researcher competencies exists. The Assessment of Key Interviewing factors for Research Assistants (AKIRA), a competency-based tool for researchers conducting interview-based mental health data collection with communities, aims to address this gap. AKIRA provides a systematic framework for the training, monitoring and evaluation of data collectors, by identifying key interviewing behaviors across 10 domains, and assessing research assistants for mastery, areas for improvement, and displays of harmful behaviors in these domains. With an emphasis on cross-cultural interactions, the AKIRA is designed to be easily adapted to various implementation contexts. Given the information gaps in mental health research, and the importance of monitoring and evaluation in the development and guidance of mental health interventions, AKIRA may enhance data quality and research especially in community mental health settings where the experiences and training of individuals involved in research and data collections may vary considerably and would benefit from competency-based trainings to support this complex work.
A major service gap exists globally where individuals' and communities' mental health needs far exceed the availability of specialized care. For instance, only 23% of individuals with depression receive adequate treatment in high-income countries, a rate that drops to 3% for populations receiving treatment for depression in lower-to-middle income countries (LIMCs) (World Health Organization, 2022). Addressing these service gaps in mental health throughout the world requires the development, cultural and contextual adaptation, implementation, monitoring, and evaluation of evidence-based interventions (Jordans and Brandon, 2020; Malekzadeh et al., 2020; Wainberg et al., 2017; World Health Organization, 2022). The World Health Organization (2022) recently emphasized the need for greater integration of research and evaluation into existing programs as a critical goal for reducing gaps in equity and improving care, yet the generation of solid evidence through research for mental health interventions continues to remain a challenge globally. Published, peer-reviewed research on various domains in existing programs is sparse in areas such as reach, effectiveness, barriers and facilitators, or impact among others. These gaps may be influenced by a wide range of factors such as resource gaps, training variations in data collection processes, and the complex contexts in which programs are being delivered (Thornicroft et al., 2017; World Health Organization, 2022; Patel et al., 2018). For instance, although 82% of the nearly one billion people with a mental health disorder live in LIMCs (World Health Organization, 2022), only 3.57% of health-related publications and 6% of open access literature on mental health derive from these contexts (Saxena et al., 2006). Moreover, despite the urgent public health concern that mental health challenges pose for communities around the world, mental health care often accounts for only 2% of national budgets (World Health Organization, 2022). Thus, a stronger evidence base is necessary to shape mental health programming, policy and investments in early prevention and healthcare systems to feasibly, effectively address the culturally unique needs of a given community or setting (Collins et al., 2011; Proctor et al., 2009).
Among the challenges and barriers that exist for carrying out research, especially in lower resourced or emergency settings, is the preparedness of individuals to collect data related to mental health programs. The lack of data collector readiness may result in decreased data collection activities and potential loss of important data, particularly from under-resourced communities, needed for a reliable, replicable and generalizable evidence base (Hossen, 2022). Several reasons may account for gaps in research and monitoring and evaluation carried out in mental health programs globally. Among mental health programs being delivered throughout the world, little is reported about the standards and consistency in which data is collected and the extent to which those involved have received training in data collection processes (Macfarlane and Carla, 2019). Additionally, the extent to which mental health programs consider methodological designs as part of their implementation strategies remains largely unknown. Furthermore, stakeholders, such as non-profit and community-based organizations are often funded to deliver mental health interventions with little to no support dedicated to research or monitoring and evaluation. As such, the resources needed to carry out these efforts may be perceived as an additional burden for which staff may not be adequately supported.
Moreover, variability in data collector competency may discourage lack of participant engagement in research or bias in data collected. Among individuals receiving mental health services, data collection challenges may also exist due to mistrust in research processes (Holden et al., 2015; Cokley and Awad, 2013; Singh et al., 2018). Underrepresentation of ethnic minorities and other minoritized groups in research reduces the ability to generalize findings and may widen the gap in health inequities (Waheed et al., 2015). Mistrust, stigma around research, and the scarcity of culturally sensitive researchers are major deterrents to willing participants (Holden et al., 2015; Singh et al., 2018). Such mistrust derives in part from histories of exploitation and deceit in which minoritized and vulnerable communities were either forced to participate in studies, or included in research without adequate information and consent (Cokley and Awad, 2013; Singh et al., 2018). Importantly, an explicit commitment to social justice, cultural sensitivity and more inclusive data collection in research practices, is critical to foster more equitable approaches to research and information that captures a greater diversity of lived experiences (Cokley and Awad, 2013).
Competency training, particularly at a cross-cultural level, linguistic adaptation, capacity building and frameworks that aid in the standardization of participant-researcher interactions, are pivotal next steps toward developing a strong evidence base reflective of the contexts in which work is carried out. The “interviewer effect” describes the potential bias introduced when respondents' perceptions of the interviewer influence an answer, resulting in potential measurement errors, particularly found in data collection processes in public health research (Davis et al., 2010). This highlights the importance of training interviewers in cross-cultural data collection among other critical interpersonal and professional competencies. Furthermore, there is a prominent gap in availability of training addressing data collection practices in addition to the standardization of frameworks to conduct these processes, particularly in lower-to-middle income contexts and community settings (Holden et al., 2015; Wainberg et al., 2017; Collins and Beverly, 2016). Capacity building in mental health research from the level of field data collection requires a targeted strategy that begins to bridge research, development, implementation, and policy (Wainberg et al., 2017; Collins and Beverly, 2016).
Despite the many factors that contribute to information gaps across global and community mental health programs, one potential strategy to promote the readiness of those involved in data collection and/or program evaluation is the use of a competency-based framework. This may be especially helpful when working with non-specialist teams, where training and exposure to research and monitoring and evaluation may vary considerably. A competency-based approach also builds on other programs developed to support non-specialist readiness to deliver scalable mental health interventions. For instance, the Enhancing Assessment of Common Therapeutic Factors (ENACT) is a globally implemented tool which forms the basis for the World Health Organization's (WHO) Ensuring Quality in Psychological Support (EQUIP) platform (Kohrt et al., 2015a,b, 2020). ENACT is a role-play-based assessment approach that provides a competency evaluation scale for trainers', supervisors', and peers' use when working with non-specialist providers of mental health support (Kohrt et al., 2015a,b). This tool, along with the EQUIP platform, was designed to set the minimum competency criteria to assure safety and enhance intervention effectiveness. For instance, EQUIP's competency-based training was more effective compared to standard training methods, which resulted in better skill acquisition and practical implementation of task-sharing mental health interventions in a study conducted in Lebanon (Jordans and Brandon, 2020). Competency based frameworks also have emerged in the field of public health education with institutions like the Centers for Disease Control (CDC) outlining competencies for public health professionals in various fields, including community health. However, these competencies largely emphasize practice as opposed to research skills (Core Competencies for Public Health Professionals, 2023). Nonetheless, the field would greatly benefit from specific tools that identify, measure and ensure specific data collection competencies in mental health research settings.
To begin to address this gap, we developed a novel competency-based framework for the assessment and training of data collectors interacting directly with participants in interview-based encounters named the Assessment of Key Interviewing Factors in Research Assistants (AKIRA). AKIRA is an adaptable, 10-item competency-based assessment tool specifically designed to facilitate self-report interviews. AKIRA was developed in the context of the National Institutes of Mental Health (NIMH)-funded research study “Restoring mental health after COVID-19 through community-based psychological services in NYC (RECOUP-NY)”. RECOUP-NY aims to evaluate the effectiveness of Problem Management Plus (PM+) (World Health Organization, 2016), a task-sharing manualized psychological intervention for distressed adults developed by the WHO, in improving the mental health of COVID-19-impacted communities in New York City. The project focuses on training staff at community-based organizations to deliver PM+ and integrate culturally sensitive, evidence-based mental health support into existing systems of care, reducing barriers and empowering communities (RECOUP-NY, 2021).
The goal in conceptualizing AKIRA was to create a competency-based framework to ensure consistency and quality among individuals involved in data collection processes given the variability of data collectors' experiences in research and participant interactions prior to working on this trial. The development of AKIRA is guided by the multi-step process used in the development of EQUIP and ENACT (Kohrt et al., 2015a,b, 2020). First, a careful review of the ENACT scale and EQUIP resources was conducted to ensure that the competencies found in AKIRA were adapted to be specific to data collection interviews, not overlapping with those of therapeutic nature from EQUIP tools. Second, an extensive literature review was carried where we further identified the gap in specific tools and frameworks to train non-specialists in data collection within the health field generally, and in community mental health research particularly. Third, an iterative process was used in which the conceptual development of AKIRA was informed through feedback and interviews with Key Informants and users. Fourth, AKIRA was pilot tested for feasibility and acceptability and is currently undergoing further evaluation. Once complete, a psychometric evaluation will be conducted with AKIRA.
AKIRA may be used as a framework to augment best practices, foster cross-cultural and trauma-informed communication skills, ensure fidelity in data collection, and identify harmful behaviors in the context of data collection. Moreover, it spotlights specific areas of improvement which helps to focus training, thus contributing to the overall quality and reliability of the data collected. Competencies and content included in the tool were informed by pilot testing, focus group discussions, interviews, and adaptations from other competency frameworks (e.g., ENACT; Kohrt et al., 2015a,b), as well as evidence-based and informed practices extensively documented in public and global mental health literature (e.g., Bedoya et al., 2016; Hoge et al., 2015; Malekzadeh et al., 2020; Matovu et al., 2013; Pedersen et al., 2022).
The AKIRA competency evaluation is conducted in the context of a semi-structured role-play in which an individual acts as a simulated research participant, while the data collector administers a questionnaire in a mimicked real-life interaction with an actor. Although the term “actor” is used, the individual simulating the research participant can be anyone with enough knowledge or experience working with the community they are representing in portraying this research participant. The role play is recorded and a “rater,” who receives basic training in the tool's framework and identification of competencies, then scores the interaction and provides feedback to the data collector accordingly.
Pilot implementation results will become available in future publications. A current version of the tool will be available for use and downloadable in English and Spanish. The AKIRA tool will undergo iterative development, adaptation and refinement based on the feedback and results from further research and implementation in the United States, Uganda, and Nepal. We aim to make the tool, lessons learned, adaptable versions, and further results available as a resource to teams aiming to enhance data collection quality. A focus on competencies may also lead to improved usability and frequency of data collection, as well as greater confidence in the quality of information acquired in the context of mental health programs. Importantly, such data can then be used to compare outcomes across diverse contexts and help to guide the delivery of interventions with greater precision.
Moreover, a competency-based approach may facilitate important discussions among stakeholders involved in mental health programming about ethical practices and potential concerns communities have about sharing information. As the AKIRA framework continues to evolve, it can be tailored and adapted to reflect specific histories or cultural factors to increase awareness and sensitivity among those involved in research and monitoring and evaluation in particular contexts. Community members should be invited to partner with those delivering and researching mental health programs. Participatory contribution to this development, capacity building and adaptation work, ensures the generation and dissemination of programming that are acceptable, relevant and aligned with the needs, and unique characteristics of served communities (Coulter et al., 2020). Along with the WHO's recommendations that mental health gaps be reduced through the training of non-mental health specialists, tools like AKIRA may also help to strengthen a research and monitoring and evaluation workforce that can complement and inform the important delivery of care.
Author contributions
AC-V: Conceptualization, Investigation, Project administration, Writing – original draft, Writing – review & editing. CB: Conceptualization, Methodology, Project administration, Supervision, Writing – review & editing, Writing – original draft. KP: Conceptualization, Methodology, Writing – review & editing, Writing – original draft. MS: Conceptualization, Methodology, Supervision, Writing – original draft, Writing – review & editing. JW: Investigation, Project administration, Writing – review & editing, Writing – original draft. WG: Conceptualization, Methodology, Writing – review & editing, Writing – original draft. JC: Conceptualization, Methodology, Writing – review & editing, Writing – original draft. SR: Investigation, Supervision, Writing – review & editing, Writing – original draft. AB: Conceptualization, Methodology, Supervision, Writing – review & editing, Writing – original draft. BK: Conceptualization, Methodology, Supervision, Writing – review & editing, Writing – original draft.
Funding
The author(s) declare that financial support was received for the research and/or publication of this article. Funding for the RECOUP-NY trial is provided the National Institute of Mental Health (R01MH127767).
Acknowledgments
We would like to thank our research assistants at The New School and George Washington University for their contributions to this work and Wallington Lin for supporting with manuscript formatting.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.
Generative AI statement
The author(s) declare that no Gen AI was used in the creation of this manuscript.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Bedoya, C. A., Sannisha, K. D, and Peter, P. E. (2016). “Cultural competence within behavioral medicine: culturally competent CBT with diverse medical populations,” in The Massachusetts General Hospital Handbook of Behavioral Medicine, eds. A. M. Vranceanu, J. Greer, and S. Safren (Cham: Humana Press), 321–334. doi: 10.1007/978-3-319-29294-6_15
Cokley, K., and Awad, G. H. (2013). In defense of quantitative methods: using the “Master's Tools” to promote social justice. J. Soc. Action Couns. Psychol. Available at: https://openjournals.bsu.edu/jsacp/article/view/487/465 (accessed May 2023).
Collins, P. Y., and Beverly, A. P. (2016). Building a global mental health research workforce: perspectives from the national institute of mental health. Academic Psychiatry 40, 723–726. doi: 10.1007/s40596-015-0453-3
Collins, P. Y., Vikram, P., Sarah, S. J., Dana, M., Thomas, R. I., Abdallah, S. D., et al. (2011). Grand challenges in global mental health. Nature 475, 27–30. doi: 10.1038/475027a
Core Competencies for Public Health Professionals (2023). Available at: https://www.phf.org/resourcestools/pages/core_public_health_competencies.aspx (accessed September 22, 2024).
Coulter, K., Maia, I., Deborah, J. M, and Abby, L. (2020). Positionality of community health workers on health intervention research teams: a scoping review. Frontiers in Public Health 8:208. doi: 10.3389/fpubh.2020.00208
Davis, R. E., Couper, M. P., Janz, N. K., Caldwell, C. H., and Resnicow, K. (2010). Interviewer effects in public health surveys. Health Edu. Res. 25, 14–26. doi: 10.1093/her/cyp046
Hoge, M. A., Jessica, W., Scott, M., Elisabeth, C., and Francis, X. G. (2015). Workforce development and mental health transformation: a state perspective. Community Mental Health J. 52, 323–331. doi: 10.1007/s10597-015-9953-6
Holden, R. J., Amanda, M. M. S., Peter, L. T. H., Ann, S. H., and Pascale, C. (2015). Data collection challenges in community settings: insights from two field studies of patients with chronic disease. Quality of Life Res. 24, 1043–1055. doi: 10.1007/s11136-014-0780-y
Hossen, M. A. (2022). “Fieldwork for data collection: preparation and challenges,” in Principles of Social Research Methodology (Singapore: Springer), 463–470. doi: 10.1007/978-981-19-5441-2_33
Jordans, M. J. D., and Brandon, A. K. (2020). Scaling up mental health care and psychosocial support in low-resource settings: a roadmap to impact. Epidemiology and Psychiatric Sci. 29:e189. doi: 10.1017/S2045796020001018
Kohrt, B. A., Mark, J. D. J., Sauharda, R., Pragya, S., Nagendra, P. L., Megan, K. R., et al. (2015b). Therapist competence in global mental health: development of the enhancing assessment of common therapeutic factors (ENACT) rating scale. Behav. Res. Ther. 69, 11–21. doi: 10.1016/j.brat.2015.03.009
Kohrt, B. A., Ramaiya, M. K. S., Rai, A. B., and Jordans, M. J. D. (2015a). Development of a scoring system for non-specialist ratings of clinical competence in global mental health: a qualitative process evaluation of the enhancing assessment of common therapeutic factors (ENACT) scale. Glob. Ment. Health 2:e23. doi: 10.1017/gmh.2015.21
Kohrt, Brandon A., Alison, S., Ann, W., Edith, H., Gloria, A. P., Sarah, W., et al. (2020). Ensuring quality in psychological support (WHO EQUIP): developing a competent global workforce. World Psychiatry 19, 115–116. doi: 10.1002/wps.20704
Macfarlane, S. B., and Carla, A. (2019). The Palgrave Handbook of Global Health Data Methods for Policy and Practice. 1st Edn. London: Palgrave Macmillan UK.
Malekzadeh, A., Kathleen, M., Celia, W., Nalini, A., and Rachel, S. (2020). Strengthening research capacity in LMICs to address the global ncd burden. Glob. Health Action 13, (1846904). doi: 10.1080/16549716.2020.1846904
Matovu, J. K. B., Rhoda, K. W., Susan, M., Olico, O., William, B., and David, S. (2013). Strengthening health workforce capacity through work-based training. BMC International Health and Human Rights 13:8. doi: 10.1186/1472-698X-13-8
Patel, V., Shekhar, S., Crick, L., Graham, T., Florence, B., Paul, B., et al. (2018). The lancet commission on global mental health and sustainable development. Lancet 392, 1553–1598. doi: 10.1016/S0140-6736(18)31612-X
Pedersen, G. A., Kendall, A. P., Adam, D. B., Kenneth, C., Ann, W., Alison, S., et al. (2022). Identifying core competencies for remote delivery of psychological interventions: a rapid review. Psychiatric Services 74, 292–304. doi: 10.1176/appi.ps.202100677
Proctor, E. K., John, L., Gregory, A., David, C., Charles, G., and Brian, M. (2009). Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health 36, 24–34. doi: 10.1007/s10488-008-0197-4
RECOUP-NY (2021). Restoring mental health after COVID-19 through community-based psychological services in NYC. Available at: https://www.recoupny.com/
Saxena, S., Guillermo, P., Pratap, S., Ghassan, K., and Ritu, S. (2006). The 10/90 divide in mental health research: trends over a 10-year period. British Journal of Psychiatry 188, 81–82. doi: 10.1192/bjp.bp.105.011221
Singh, S., Granski, M., Victoria, M. D. P., and Javdani, S. (2018). The praxis of decoloniality in researcher training and community-based data collection. Am. J. Community Psychol. 62, 385–395. doi: 10.1002/ajcp.12294
Thornicroft, G., Somnath, C., Sara, E.-L., Michael, G., Nancy, S., Sergio, A.-G., et al. (2017). Undertreatment of people with major depressive disorder in 21 countries. British Journal of Psychiatry 210, 119–124. doi: 10.1192/bjp.bp.116.188078
Waheed, W., Hughes-Morley, A., Woodham, A., Allen, G., and Bower, P. (2015). Overcoming barriers to recruiting ethnic minorities to mental health research: a typology of recruitment strategies. BMC Psychiatry. 15:101. doi: 10.1186/s12888-015-0484-z
Wainberg, M. L., Pamela, S., James, M. S., Liat, H., Jennifer, J. M., Karen, A. J., et al. (2017). Challenges and opportunities in global mental health: a research-to-practice perspective. Current Psychiatry Rep. 19:28. doi: 10.1007/s11920-017-0780-z
World Health Organization (2016). Problem Management Plus (PM+): Individual Psychological Help for Adults Impaired by Distress in Communities Exposed to Adversity. Geneva: World Health Organization.
Keywords: mental health research, monitoring and evaluation, competency assessment, global mental health, cross-cultural, capacity building, equity
Citation: Cid-Vega A, Best C, Pfeffer K, Sangraula M, Wong J, Gwaikolo W, Caracoglia J, Rai S, Brown AD and Kohrt B (2025) Assessment of key interviewing factors for research assistants (AKIRA): development of a novel training and evaluation competency-based tool for mental health data collection in community settings. Front. Educ. 10:1539124. doi: 10.3389/feduc.2025.1539124
Received: 03 December 2024; Accepted: 05 February 2025;
Published: 12 March 2025.
Edited by:
Mona Hmoud AlSheikh, Imam Abdulrahman Bin Faisal University, Saudi ArabiaReviewed by:
Ingrid Vargas-Huicochea, National Autonomous University of Mexico, MexicoCopyright © 2025 Cid-Vega, Best, Pfeffer, Sangraula, Wong, Gwaikolo, Caracoglia, Rai, Brown and Kohrt. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Alejandra (Alesha) Cid-Vega, YS5jaWR2ZWdhQG5ld3NjaG9vbC5lZHU=; Adam D. Brown, YnJvd25hZEBuZXdzY2hvb2wuZWR1