Gaussian Variational Estimation for Multidimensional Item Response Theory

Abstract

Multidimensional Item Response Theory (MIRT) is widely used in assessment and evaluation of educational and psychological tests. It models the individual response patterns by specifying functional relationship between individuals' multiple latent traits and their responses to test items. One major challenge in parameter estimation in MIRT is that the likelihood involves intractable multidimensional integrals due to latent variable structure. Various methods have been proposed that either involve direct numerical approximations to the integrals or Monte Carlo simulations. However, these methods have some limitations in that they are computationally demanding in high dimensions and rely on sampling from a posterior distribution.

In the second chapter of the thesis, we propose a new Gaussian Variational EM (GVEM) algorithm which adopts a variational inference to approximate the intractable marginal likelihood by a computationally feasible lower bound. The optimal choice of variational lower bound allows us to derive closed-form updates in EM procedure, which makes the algorithm efficient and easily scale to high dimensions. We illustrate that the proposed algorithm can also be applied to assess the dimensionality of the latent traits in an exploratory analysis. Simulation studies and real data analysis are presented to demonstrate the computational efficiency and estimation precision of the GVEM algorithm in comparison to the popular alternative Metropolis-Hastings Robbins-Monro algorithm. In addition, theoretical guarantees are derived to establish the consistency of the estimator from the proposed GVEM algorithm.

One of the key elements in MIRT is the relationship between the items and the latent traits, so-called a test structure. The correct specification of this relationship is crucial for accurate assessment of individuals. Hence, it is of interest to study how to accurately estimate the test structure from data. In the third chapter, we propose to apply GVEM to solve a latent variable selection problem for MIRT and empirically estimate the test structure. The main idea is to impose L1-type penalty to the variational lower bound of the likelihood to recover a simple test structure in iterative procedures. Simulation studies show that the proposed method accurately estimates the test structure and is computationally efficient. A real data analysis on the large-scale assessment test called National Education Longitudinal Study of 1988 is presented.

In the last chapter, we discuss some of the interesting extensions of our proposed method. The first extension is to develop the estimation method via GVEM procedures for the Multidimensional 4-Parameter Logistic model, which is known to be more challenging than previously discussed MIRT models. The second extension is to study Differential Item Functioning (DIF) analysis in MIRT. In brief, DIF occurs when groups (such as defined by gender, ethnicity, or education) have different probabilities of responses for a given test item even though people have the same latent abilities. Our goal is to identify test items that have DIF. We formulate the DIF analysis in MIRT as the regularization problem and solve it via our proposed GVEM approach. Simulation studies are presented to show the performance of our proposed method on these topics.