Predictive power of selection tests on the academic performance of medical students

This study analyzes the predictive power of various selection tests for admission to the bachelor’s degree in medical surgery at the Autonomous University of Tamaulipas. A retrospective cohort study with an analytical approach was conducted, evaluating 167 students of the 2022 cohort. Their performance in the first two school periods was compared with the results of admission tests such as the National Entrance Exam II of the National Evaluation Center for Graduate Education (CENEVAL EXANI-II), and other exams of the propaedeutic course. It was revealed that the CENEVAL EXANI-II test has a medium-high correlation (r = 0.588; p = 0.000) with academic performance in the first two terms although its predictive power is relatively low (R² = 0.138, p = 0.001). Histology modules (r = 0.518; p = 0.000), and Module Final Exam (r = 0.472; p = 0.000) showed significant correlations but with lower predictive power. Differences in academic performance were identified according to gender and type of admission (selected vs. conditional). Although screening tests can predict academic success to some extent, their effectiveness is limited. Improved assessment methods are recommended to enhance the development of students in their medical training. This study provides the basis for future research on optimizing selection processes in medical education.

Introduction

The admissions process for applicants to medical school is complex and varies greatly from country to country (Eveland et al., 2022). It is highly competitive, with very few places available, allowing only the best students to be accepted through a rigorous admissions process. The selection of accepted students is a difficult task, as many cognitive and non-cognitive factors influence the selection process (Hefny et al., 2024).

Universities and health sciences educational institutions decide within their processes how best to select the applicants they will accept into their academic programs. This will depend on the national legal context, the admission and exit characteristics of the educational program, and the society in which these institutions are located (Gatica-Lara et al., 2010). One way to guide selection processes is through reports on the prior knowledge that applicants must have to enter the bachelor’s program in medical surgery (Association of American Medical Colleges and Howard Hughes Medical Institute, 2009; Finnerty, 2010), which are evaluated in admissions exams through various processes.

Selection processes vary widely among educational institutions around the world in terms of the type of assessment, the number of tests, the number of applicants, and how the results are considered (Li et al., 2019). These processes represent a hurdle that applicants must overcome to gain admission to any medical school in the world (Norman et al., 2002). The best selection processes consider the philosophy of the university or educational institution, the context of the society to which the future health professionals will provide their services, and the health system in which these graduates will be integrated (Patterson et al., 2018).

In Mexico, one of the selection processes for students who graduate from High Schools and intend to be registered in universities and health institutions is that of the National Evaluation Center for Graduate Education (CENEVAL), with the possibility of taking the National Admission Examination II (EXANI-II) (Centro Nacional de Evaluación para la Educación Superior, 2020), and other specific exams designed to evaluate competencies relevant to medical training. Selection processes are designed to determine which applicants are most likely to succeed in their studies, but they do not necessarily predict academic performance during their studies (Tamimi et al., 2023). The academic performance of medical students is crucial for their future professional success, and the quality of health services (Althewini and Al Baz, 2022).

The Autonomous University of Tamaulipas, through its School of Medicine in Tampico, has adopted a comprehensive approach to student selection using a combination of psychometric and academic tests. However, the effectiveness of these tests in predicting academic performance during early years has not yet been fully explored. This study seeks to fill that gap by analyzing the predictive power of screening tests on the academic performance of students in the 2022 cohort of the Medical Surgeon bachelor’s degree. Specifically, it examines the correlations between admission test scores and grades obtained in the first and second periods of the degree. The findings of this research could have significant implications for the improvement of selection processes and the development of educational strategies to optimize student performance.

Materials and methods

Study design

A retrospective cohort study with an analytical approach was conducted to evaluate the predictive power of screening tests on the academic performance of medical students.

Participants

Data from 167 students were considered for the 2022 cohort, who are currently in the 5th period of the Surgeon Medical Bachelor’s Degree of the School of Medicine in Tampico of the Autonomous University of Tamaulipas, who had previously applied to the call for admission to the career from a total of 1,200 applicants.

Sampling and variables

Data collection was non-probabilistic (by convenience), and the independent variables identified as independent variables were the applicant selection tests of the propaedeutic course, comprising the subjects: Module 1 Anatomy, Module 2 Histology, Module 3 Physiology, Module 4 Sexual and Reproductive Health, the Module Final Exam, the CENEVAL EXANI-II Exam, and the Psychometric Test, and the dependent variable was the average academic performance of the 1st and 2nd periods of the Surgeon Medical Bachelor’s Degree.

The main source of data corresponded to the records of the results of the selection tests, as well as the evaluations, and grades provided by the databases of the School Control Department of the School at the time of application, and at the end of the 1st and 2nd periods of the course. The study was approved by the Research Ethics Committee of the Alberto Romo Caballero MD of the School of Medicine of the Autonomous University of Tamaulipas. The data collected were coded to protect the students’ information, and only the researchers themselves had access to the database.

Statistical analysis

Phase 1

Descriptive statistics were used for sociodemographic variables such as gender, and geographic origin of the applicant.

Phase 2

Determination of changes or statistically significant differences in the results of the selection tests according to dichotomous variables such as gender, the accreditation of the psychometric test, and whether admission to the program was through a selection process or conditional, using the Mann–Whitney U test for independent samples with a confidence level of 95%.

Phase 3

Estimation of statistically significant differences with the results of the selection tests according to the group to which they were assigned (distributed from A to G, where the latter was the group of students with conditional admission) by means of an analysis of variance (ANOVA) (EL ANOVA, 2018) and Tukey’s post-hoc test.

Phase 4

Characterization of the degree of correlation between the selection variables for admission and the academic performance variables of the students once in the program. For this purpose, a Spearman correlation analysis was performed.

Phase 5

Identification of the factors associated with academic performance through multiple linear regression, using the stepwise method to find the model, and the variables with the greatest power of association.

For all purposes, a p-value of <0.05 was considered statistically significant, and statistical analyses were performed in the IBM SPSS v.26 and JASP v.0.12.2.0 statistical programs. Similarly, power and effect size were calculated in each case using G*Power software.

Results

Table 1 shows the results of the CENEVAL EXANI-II test corresponding to the 167 applicants accepted for enrollment in the first period of the Medical Surgeon Bachelor’s Degree, out of a population of 1,200 applicants for the 2022 selection process. The accepted applicants are represented by 35.3% (n = 59) men, and 64.7% (n = 108) women, it is important to note that in this admission process 84.4% (n = 141) applicants were selected by passing all the selection exams, and 15.6% (n = 26) did not pass all the exams but were conditionally accepted. Of the accepted applicants, 7.78% (n = 13) obtained an unsatisfactory performance, 79.05% (n = 132) satisfactory performance, and 13.17% (n = 22) an outstanding performance.

TABLE 1

Table 1. Gender, state of origin of the applicants, and CENEVAL EXANI-II test result.

The means and standard deviations of the grades of the four modules were compared: Anatomy, Histology, Physiology, Sexual and Reproductive Health, as well as the final module averages, and the CENEVAL EXANI-II examination results between men (1), and women (2). A statistically significant difference was found (at a level α = 0.05) only in the grades of the module corresponding to the Sexual and Reproductive Health subject, which obtained lower means (p < 0.05) with respect to the means of Anatomy, Histology, Physiology, Sexual and Reproductive Health, Module Final Exam, and CENEVAL EXANI-II Test, which showed means (p > 0.05) (Table 2). Likewise, the means of the selection tests were contrasted with the result of the psychometric test by means of the dichotomous question Yes approved (1) and Not approved (2), obtaining statistically significant differences (at a level α = 0.05) with a p < 0.05 in the results of Anatomy, Histology, Sexual and Reproductive Health Modules, Final Exam of Modules, and CENEVAL EXANI-II Test, opposite case in the Physiology Module showing a p < 0.05 (Table 3).

TABLE 2

Table 2. Differences, according to gender, in the results of the selection tests of applicants to the medical surgeon degree.

TABLE 3

Table 3. Differences, according to the psychometric test, in the results of the selection tests of the applicants to the medical surgeon degree.

Comparing the grades of students with (1) selected and (2) conditional criterion, it was observed that the mean of conditional entry grades was significantly lower than the mean of selected grades (at a level α = 0.05), all selection tests indicated means (p < 0.05) (Table 4).

TABLE 4

Table 4. Differences in the results of the selection tests of the applicants to the medical surgeon degree considering whether they were selected or conditional admission.

The 167 accepted applicants were enrolled as the class of 2022, they were assigned to groups A to G. With the results of the modules of the selection process, it was found that the conditional students assigned to groups G were the ones who showed significant differences in grade point averages (at a level α = 0.05) compared to groups A to F (Table 5).

TABLE 5

Table 5. Differences in academic performance according to assigned group.

Table 6 shows the concurrence and prediction processes, the results of the academic performance were considered, and the mean of the 1st and 2nd periods of the course were obtained, which were contrasted with each of the test results of the applicant selection process. Given the non-parametric nature of the data, Spearman’s rho correlation coefficients were used, yielding a high mean correlation between the academic performance of the 1st and 2nd periods of the course and the CENEVAL EXANI-II Test (r = 0.588; p = 0.000), the Histology Module (r = 0.518; p = 0.000), and the Final Module Exam (r = 0.472; p = 0.000).

TABLE 6

Table 6. Correlation between the selection tests and the academic performance of the average of the 1st and 2nd period of the bachelor’s degree.

By means of linear regression the predictive level of the independent variables was determined, in the first order the variable CENEVAL EXANI-II Exam was analyzed with respect to the dependent variable academic performance of the 1st and 2nd period, a low value was observed (R² = 0.138; p = 0.001), although the correlation of the variables is medium high (r = 0.588; p = 0.000), this model indicates that the academic performance of an applicant to the Medicine Bachelor’s Degree is predicted by the CENEVAL EXANI-II test by 14% (Figure 1).

FIGURE 1

Figure 1. Predictive model between the CENEVAL EXANI-II test and academic performance in the 1st and 2nd periods. (Author’s own source.)

Discussion

In quantitative terms, the productivity of an educational institution is often measured by its graduates and alumni, specifically the number of students who have successfully completed the educational program and fulfilled the requirements established by the institution (Miranda-Saucedo et al., 2023); however, there are other qualitative and quantitative variables that must be evaluated to determine the efficiency and quality of educational programs. For the medical surgeon degree, in addition to the importance of its graduates, the applicants represent a human input of great value that deserves to be evaluated from its selection process.

The complexity of the medical education programs, the knowledge prior to entering the profession, the teaching performance (Flores-Hernández et al., 2016; García-Domínguez, 2016; Muñoz-Comonfort et al., 2014) are some of the factors that show relevance in the studies to know the predictive power of the selection tests for the entrance to a medical degree, and to contrast it with the academic performance. According to the purpose of the study, the main findings refer that there is a significant difference in the mean scores in the Sexual and Reproductive Health Module with respect to gender, which is supported by the contributions of Ortiz-Castillo (2016), who states that the medical student populations show a ratio of 1 male for 1.9 females. This observation suggests that gender may influence academic performance in specific subjects, warranting further investigation into the underlying causes.

In accordance with the findings on the differences in selection test results of applicants to the medical surgeon degree in relation to the psychometric test, it is necessary to comment that there is no report of any other study that has addressed it as such, although Carrillo-Avalos et al. (2024) suggested that psychometric assessments do not significantly impact early academic success. Our results imply that while psychometric tests may provide insights into certain cognitive abilities, they may not be as predictive of academic performance in the initial stages of medical education.

The CENEVAL EXANI-II test emerged as the primary predictor of academic performance among medical students in their fifth period, which coincides with the research of Hernández-Madrigal et al. (2021), who also identified a strong correlation between this test and academic success. However, our study contrasts with the results of (García-Domínguez’s (2016) work, who reported that the average scores obtained in propaedeutic courses were the most significant predictors of academic performance. This discrepancy may stem from differences in study populations, methodologies, or the specific educational contexts in which these studies were conducted.

Furthermore, Martínez-González et al. (2021) highlighted that mathematical thinking, a component evaluated by the EXANI-II test, had the greatest predictive power in academic performance, underscoring its relevance; strong performance in this area is not only crucial for the selection exam but also reflects the analytical and problem-solving skills essential for medical training. Our findings support this assertion, as we observed a medium-high correlation between the EXANI-II results and students’ academic performance. However, the relatively low predictive power (p = 0.001) suggests that while the EXANI-II test is a valuable tool, it should not be the sole criterion for assessing a student’s potential for success in medical education.

Conclusion

The present study demonstrates that while selection tests such as the CENEVAL EXANI-II exhibit moderate to high correlations with early academic performance of medical students in their first semesters, their ability to predict long-term success is limited. This indicates that these tests, although useful, are not sufficient on their own to determine a student’s full academic potential throughout their medical education. It is recommended that future research explore the integration of additional metrics and assessment methods to improve the prediction of academic performance and assist students in their transition and success within medical education programs.

Finally, medical programs should consider implementing ongoing support and development programs for selected students. This proactive approach can help students develop the critical skills necessary to overcome academic and personal challenges throughout their medical education. Programs focused on mentorship, clinical and emotional skills development workshops, and stress management and resiliency resources can be invaluable in supporting students’ continued success and wellbeing.

Data availability statement

The original contributions presented in this study are included in this article/supplementary material, further inquiries can be directed to the corresponding author.

Ethics statement

This research proposal is conducted in a wholly ethical manner, strictly adhering to the highest standards of academic integrity and human rights. The study uses already recorded academic data from students, which is a safe and non-invasive approach that minimizes risks to participants. In addition, no written informed consent was required, as it is considered safe research according to national and institutional regulations. The research was approved by the Research Ethics Committee of the Alberto Romo Caballero MD of the School of Medicine of the Autonomous University of Tamaulipas. All data collected have been coded to guarantee confidentiality, and only the investigators involved in the study have access to the database. These measures ensure the protection of the privacy of the participants and the integrity of the research process.

Author contributions

MC: Conceptualization, Methodology, Project administration, Writing – original draft, Writing – review & editing. RD: Data curation, Resources, Supervision, Writing – original draft, Writing – review & editing. RS: Formal analysis, Investigation, Methodology, Writing – original draft, Writing – review & editing. BG: Formal analysis, Investigation, Validation, Writing – original draft, Writing – review & editing. EH: Formal analysis, Validation, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Conflict of interest

The authors declare that the research was conducted without any commercial or financial relationships that could potentially create a conflict of interest.

Generative AI statement

The authors declare that no Generative AI was used in the creation of this manuscript.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Althewini, A., and Al Baz, N. (2022). Prediction of admission tests for medical students’ academic performance. Adv. Med. Educ. Pract. 13, 1287–1292. doi: 10.2147/AMEP.S355474

PubMed Abstract | Crossref Full Text | Google Scholar

Association of American Medical Colleges and Howard Hughes Medical Institute. (2009). Scientific Foundations for Future Physicians Report of the AAMC-HHMI Committee. [Online]. Available online at: https://www.aamc.org/system/files?file=2020-02/scientificfoundationsforfuturephysicians.pdf (accessed August 28, 2024).

Google Scholar

Carrillo-Avalos, B., Leenen, I., Trejo-Mejía, J., and Sánchez Mendiola, M. (2024). Evidencias de validez del proceso de admisión a una escuela de medicina en México. Rev. Invest. Educ. Méd. 13, 37–55. doi: 10.22201/fm.20075057e.2024.50.23546

Crossref Full Text | Google Scholar

Centro Nacional de Evaluación para la Educación Superior (2020). Ceneval: Exani II. [Online]. Available online at: https://ceneval.edu.mx/examenes-ingreso-exani_ii/ (accessed July 07, 2023).

Google Scholar

EL ANOVA (2018). Máxima Consultoria. [Online]. Available online at: https://www.maximaconsultoria.es/blog/el-anova-de-una-via/ (accessed 30, 2023)

Google Scholar

Eveland, A., Wilhelm, S., Wong, S., Prado, L., and Barsky, S. H. A. (2022). preliminary study pf the probitive value of personality assessment in medical school admissions with the Unites States. BMC Med. Educ. 22:890. doi: 10.1186/s12909-022-03901-x

PubMed Abstract | Crossref Full Text | Google Scholar

Finnerty, P. (2010). The role and value of the basic sciences in medical education: An Examination of flexner’s legacy. Med. Sci. Educ. 85, 349–355.

Google Scholar

Flores-Hernández, F., Sánchez-Mendiola, M., and Martínez-González, A. (2016). Modelo de predicción del rendimiento académico de los estudiantes del ciclo básico de la carrera de Medicina a partir de la evaluación del desempeño docente. Rev. Mexicana Invest. Educ. 21, 975–991.

Google Scholar

García-Domínguez, L. A. (2016). Pruebas de selección como predictores de rendimiento académico de estudiantes de medicina. Rev. Invest. Méd. 5, 88–92. doi: 10.1016/j.riem.2016.01.018

Crossref Full Text | Google Scholar

Gatica-Lara, F., Méndez-Ramírez, I., Sánchez-Mendiola, M., and Martínez-González, A. (2010). Variables asociadas al éxito académico en estudiantes de la Licenciatura en Medicina de la UNAM. Rev. Facultad Med. 53, 9–18.

Google Scholar

Hefny, A., Almansoori, T., El-Zubeir, M., AlBawardi, A., Shaban, S., Magzoub, M., et al. (2024). Relationship between admission selection tools and student attrition in the early years of medical school. J. Taibah Univer. Med. Sci. 19, 447–452. doi: 10.1016/j.jtumed.2024.02.004

PubMed Abstract | Crossref Full Text | Google Scholar

Hernández-Madrigal, M., Rivera-García, A., Ramírez-Flores, E., Hernández-García, P., and Gamboa-Cerda, S. (2021). Evaluación de los exámenes estandarizados como predictores del desempeño académico en una Universidad pública mexicana. Revista Electrónica Educ. Pedagogía. 5, 30–43. doi: 10.15658/rev.electron.educ.pedagog21.04050803

Crossref Full Text | Google Scholar

Li, J., Thompson, R., and Shulruf, B. (2019). Struggling with strugglers: Using data from selection tools for early identification of medical students at risk of failure. BMC Med. Educ. 19:415. doi: 10.1186/s12909-019-1860-z

PubMed Abstract | Crossref Full Text | Google Scholar

Martínez-González, A., Soto-Estrada, G., García-Minjares, M., and Sánchez-Mendiola, M. (2021). Perfil de éxito académico en estudiantes de medicina de acuerdo a su trayectoria académica del bachillerato. Invest. Educ. Méd. 10, 23–31. doi: 10.22201/fm.20075057e.2021.38.20330

Crossref Full Text | Google Scholar

Miranda-Saucedo, C., Arceo-Guzmán, M., and Pimentel-Ramírez, M. (2023). Propuesta de mejora en el índice de titulación en la licenciatura de médico cirujano de la Facultad de Medicina. UAEMéx. J. RedCA 6, 71–91. doi: 10.36677/redca.v6i16.21427

Crossref Full Text | Google Scholar

Muñoz-Comonfort, A., Leenen, I., and Fortoul-van der Goes, T. I. (2014). Correlation between diagnostic assessment and academic performance of medical students. Invest. Educ. Méd. 3, 85–91. doi: 10.1016/S2007-5057(14)72731-0

Crossref Full Text | Google Scholar

Norman, G., Van Der Vleuten, C., Newble, D., Dolmas, D., Mann, K., Rothman, A., et al. (2002). International Handbook of Reserch in Medical Education, 1st Edn, eds G. Norman, C. Van Der Vleuten, and D. Newble (Netherlands: Springer - Science).

Google Scholar

Ortiz-Castillo, J. (2016). Relación de la Ansiedad y depresión sobre el rendimiento académico en estudiantes de medicina. Psiquiatría y Salud Mental. 33, 123–131.

Google Scholar

Patterson, F., Roberts, C., Hanson, M., Hampe, W., Eva, K., Ponnamperuma, G., et al. (2018). Ottawa consensus statement: Selection and recruitment to the healthcare professions. Med. Teach. 40, 1091–1101. doi: 10.1080/0142159X.2018.1498589

PubMed Abstract | Crossref Full Text | Google Scholar

Tamimi, A., Hassuneh, M., Tamimi, I., Juweid, M., Shibli, D., AlMasri, B., et al. (2023). Admission criteria and academic performance in medical school. BMC Med. Educ. 23:273. doi: 10.1186/s12909-023-04251-y

PubMed Abstract | Crossref Full Text | Google Scholar