Pitt Logo LinkContact Us

COMPARING DIF DETECTION FOR MULTIDIMENSIONAL POLYTOMOUS MODELS USING MULTI GROUP CONFIRMATORY FACTOR ANALYSIS AND THE DIFFERENTIAL FUNCTIONING OF ITEMS AND TESTS

Kannan, Priya (2011) COMPARING DIF DETECTION FOR MULTIDIMENSIONAL POLYTOMOUS MODELS USING MULTI GROUP CONFIRMATORY FACTOR ANALYSIS AND THE DIFFERENTIAL FUNCTIONING OF ITEMS AND TESTS. Doctoral Dissertation, University of Pittsburgh.

[img] PDF - Primary Text
Restricted to University of Pittsburgh users only until 13 May 2016.

Download (2097Kb) | Request a copy

    Abstract

    This study evaluated the robustness of DIF detection for multidimensional polytomous items using two different estimation methods, MG-CFA and MGRM-DFIT. A simulation study across 960 study conditions was performed. The purpose of this study was to establish the Type-I error rate and Power of DIF detection for the MG-CFA and MGRM-DFIT estimation methods across the study conditions. The MGRM-DFIT method consistently controlled Type-I error rate under alpha across all study conditions. Though the MGRM-DFIT method demonstrated high power in detecting DIF for the combined items, it had lower power in detecting DIF for each item individually. The MGRM-DFIT method had higher power of DIF detection when impact (true distributional differences) is in the opposite direction of manipulated DIF. Overall, compared to the non-DIF items, NCDIF values are larger, and CDIF values are smaller for the 4 DIF items. Across the replications and the study conditions, CDIF was not as consistent as NCDIF. The MG-CFA method demonstrated slightly inflated Type-I error rate in a couple of study conditions (particularly in the presence of impact). However, the MG-CFA method demonstrated lower power across all study conditions. This could partly be explained by the low magnitude of DIF that was manipulated in the 'α/λ' parameter in this study. Parameter estimation for the MGRM, and the MGRM-DFIT method should be incorporated as part of commonly used software packages. In general, the MG-CFA method is recommended for DIF detection with multidimensional polytomous types of items, since it performs more consistently as a univariate test and as a multivariate test, and is easily available as part of several commonly used software packages.


    Share

    Citation/Export:
    Social Networking:

    Details

    Item Type: University of Pittsburgh ETD
    Creators/Authors:
    CreatorsEmailORCID
    Kannan, Priyaprk16@pitt.edu, pria.kannan@gmail.com
    ETD Committee:
    ETD Committee TypeCommittee MemberEmailORCID
    Committee ChairKim, Kevin Hkhkim@education.pitt.edu
    Committee MemberStone, Clementcas@pitt.edu
    Committee MemberKirisci, Leventlevent@pitt.edu
    Committee MemberLane, Suzannesl@pitt.edu
    Title: COMPARING DIF DETECTION FOR MULTIDIMENSIONAL POLYTOMOUS MODELS USING MULTI GROUP CONFIRMATORY FACTOR ANALYSIS AND THE DIFFERENTIAL FUNCTIONING OF ITEMS AND TESTS
    Status: Unpublished
    Abstract: This study evaluated the robustness of DIF detection for multidimensional polytomous items using two different estimation methods, MG-CFA and MGRM-DFIT. A simulation study across 960 study conditions was performed. The purpose of this study was to establish the Type-I error rate and Power of DIF detection for the MG-CFA and MGRM-DFIT estimation methods across the study conditions. The MGRM-DFIT method consistently controlled Type-I error rate under alpha across all study conditions. Though the MGRM-DFIT method demonstrated high power in detecting DIF for the combined items, it had lower power in detecting DIF for each item individually. The MGRM-DFIT method had higher power of DIF detection when impact (true distributional differences) is in the opposite direction of manipulated DIF. Overall, compared to the non-DIF items, NCDIF values are larger, and CDIF values are smaller for the 4 DIF items. Across the replications and the study conditions, CDIF was not as consistent as NCDIF. The MG-CFA method demonstrated slightly inflated Type-I error rate in a couple of study conditions (particularly in the presence of impact). However, the MG-CFA method demonstrated lower power across all study conditions. This could partly be explained by the low magnitude of DIF that was manipulated in the 'α/λ' parameter in this study. Parameter estimation for the MGRM, and the MGRM-DFIT method should be incorporated as part of commonly used software packages. In general, the MG-CFA method is recommended for DIF detection with multidimensional polytomous types of items, since it performs more consistently as a univariate test and as a multivariate test, and is easily available as part of several commonly used software packages.
    Date: 13 May 2011
    Date Type: Completion
    Defense Date: 04 April 2011
    Approval Date: 13 May 2011
    Submission Date: 18 April 2011
    Access Restriction: 5 year -- Restrict access to University of Pittsburgh for a period of 5 years.
    Patent pending: No
    Institution: University of Pittsburgh
    Thesis Type: Doctoral Dissertation
    Refereed: Yes
    Degree: PhD - Doctor of Philosophy
    URN: etd-04182011-000321
    Uncontrolled Keywords: ; Differential Item Functioning; Item Response Theory; Multi group Confirmatory Factor Analysis; Multidimensional Graded Response Models; Multidimensional models; Polytomous models; Structural Equation Modeling
    Schools and Programs: School of Education > Psychology in Education
    Date Deposited: 10 Nov 2011 14:38
    Last Modified: 22 May 2012 15:52
    Other ID: http://etd.library.pitt.edu/ETD/available/etd-04182011-000321/, etd-04182011-000321

    Actions (login required)

    View Item

    Document Downloads