Animated videos in assessment: comparing
validity evidence from and test-takers’ reactions
to an animated and a text-based situational
judgment test
Karakolidis, AnastasiosORCID: 0000-0002-7460-7759, O'Leary, MichaelORCID: 0000-0002-6771-904X and Scully, DarinaORCID: 0000-0002-6076-717X
(2021)
Animated videos in assessment: comparing
validity evidence from and test-takers’ reactions
to an animated and a text-based situational
judgment test.
International Journal of Testing, 21
(2).
pp. 57-79.
ISSN 1530-5058
The linguistic complexity of many text-based tests can
be a source of construct-irrelevant variance, as test-takers’ performance may be affected by factors that are
beyond the focus of the assessment itself, such as
reading comprehension skills. This experimental study
examined the extent to which the use of animated
videos, as opposed to written text, could (i) reduce
construct-irrelevant variance attributed to language
and reading skills and (ii) impact test-takers’ reactions
to a situational judgment test. The results indicated
that the variance attributed to construct-irrelevant factors was lower by 9.5% in the animated version of the
test. In addition, those who took the animated test
perceived it to be more valid, fair, and enjoyable, than
those who took the text-based test. They also rated
the language used as less difficult to understand. The
implications of these findings are discussed.