AUTHOR=French Brian F. , Finch W. Holmes , Immekus Jason C. TITLE=Multilevel Generalized Mantel-Haenszel for Differential Item Functioning Detection JOURNAL=Frontiers in Education VOLUME=4 YEAR=2019 URL=https://www.frontiersin.org/journals/education/articles/10.3389/feduc.2019.00047 DOI=10.3389/feduc.2019.00047 ISSN=2504-284X ABSTRACT=

Research has demonstrated that when data are collected in a multilevel framework, standard single level differential item functioning (DIF) analyses can yield incorrect results, particularly inflated Type I error rates. Prior research in this area has focused almost exclusively on dichotomous items. Thus, the purpose of this simulation study was to examine the performance of the Generalized Mantel-Haenszel (GMH) procedure and a Multilevel GMH (MGMH) procedure for the detection of uniform differential item functioning (DIF) in the presence of multilevel data with polytomous items. Multilevel data were generated with manipulated factors (e.g., intraclass correction, subjects per cluster) to examine Type I error rates and statistical power to detect DIF. Results highlight the differences in DIF detection when the analytic strategy matches the data structure. Specifically, the GMH had an inflated Type I error rate across conditions, and thus an artificially high power rate. Alternatively, the MGMH had good power rates while maintaining control of the Type I error rate. Directions for future research are provided.