An Application of the Odds Ratios Method in Differential Item Functioning:An Illustration with an English Test
Liu Chutong,Jin Ruyi,He Ying,Zhang Minqiang,Gao Fangxin
DOI: https://doi.org/10.16719/j.cnki.1671-6981.20230226
2023-01-01
Abstract:The detection of differential item functioning（DIF） is an essential step for increasing the validity of a test among groups. A common feature of the conventional DIF approaches is that different groups of examinees are placed on the same metric based on a matching variable. However,contaminated matching variable would make the DIF detection biased. Some investigators developed the odds ratios（OR） method, which does not require matching variables. The OR method is easy to understand and simple for practitioners. However, all previous studies about the OR method are simulation studies, and the application of OR method in the DIF detecting of the academic achievement test needs further investigation. Previous studies indicated that the IRT-based approaches performed well in the DIF detecting of the academic achievement test. The Wald test and likelihood ratio test（LRT） are widely used in DIF detection as IRT-based approaches. It is helpful to compare the performance and the operating procedure of OR method with the Wald test and the LRT method.The current study aimed to introduce the operating procedure of the OR methods in the DIF detecting of the academic achievement test, and demonstrated that the OR method was a better method in DIF detecting by comparing the performance and the operating procedure among the OR method, the Wald test, and the LRT method. An empirical study was conducted, and the data was selected from an English test. Valid data from 3241senior high school students, including 1427 males（44.03%） were analyzed. The DIF detection was conducted on 35 dichotomous items from the English examination. The difficulty of the 35 dichotomous items were acceptable（from.20 to.80）. Besides, the Cronbach’s alphas of the 35 items were.86. The DIF detection results of the OR method, the Wald test, and the LRT method on English test would be testified and compared.The empirical study found that 16 items were detected as DIF items by the OR method, including both 8 items favoring females and males respectively. 14 items were detected as DIF items by the Wald χ2 test, including 10 items favoring females and 4 items favoring males. In addition,there were 14 items detected as DIF items by the LRT method, including 8 items favoring females and 6 items favoring males. The DIF detection results of the OR method and the LRT method were consistent for 31 items, and inconsistent for 4 items. The DIF detection results of the Wald χ2 test and the LRT method were consistent for 19 items, and inconsistent for 16 items.Previous studies have provided evidence for the application of the LRT method in the DIF detecting of the academic achievement test. This study further indicated that performance of the OR method was similar to the LRT method, but different to the Wald χ2 test. Thus, the present study also provided evidence for the application of OR method in the DIF detection of the academic achievement test, which was of great empirical value.Besides, concerning the operating procedure, the OR method is easy and simple for practitioners, and the computation task in the OR method is minimized. It is obvious that the OR method is more efficient. Generally speaking, the OR method is practicable and efficient in DIF detection.