An investigation of a criterion- referenced test using G-theory, and factor and cluster analyses

Abstract

There has been relatively little research on analytical procedures for examin ing the dependability and validity of criterion-referenced tests especially when compared to similar investigations for norm-referenced ESL or EFL tests. This study used three analytical procedures, namely, G-theory, factor and cluster analyses, to investigate the dependability and validity of a criterion-referenced test developed at the University of California, Los Angeles in 1989. Dependability estimates showed that test scores are not equally depend able for all placement groups and are rather undependable for two out of the four placement groups. Factor analysis of test scores for the placement groups showed that though two-factor solutions were the best solutions for the different groups, there were differences in the way the subtests loaded in the different groups, with progressively fewer subtests loading on the second factor as ability increased. This finding led to the extension study with cluster analysis which showed that a number of students might have been differently placed if subtest scores were used to place them.