Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Animated videos in assessment: a comparison study of validity evidence from and test-takers’ Reactions to an animated and a text-Based Version of a situational judgment test

Karakolidis, Anastasios orcid logoORCID: 0000-0002-7460-7759 (2019) Animated videos in assessment: a comparison study of validity evidence from and test-takers’ Reactions to an animated and a text-Based Version of a situational judgment test. PhD thesis, Dublin City University.

Abstract
The majority of tests in use today rely on static text to communicate information, ideas, and concepts and to pose questions. However, the overuse of text may have consequences for the validity of the inferences drawn from test-takers’ scores. This may be true especially in the case of assessments taken by test-takers with poor reading comprehension skills or with low levels of proficiency in the language of the test. More specifically, linguistic complexity can be a source of construct-irrelevant variance as test-takers’ performance can be negatively affected by factors that are beyond the focus of the assessment itself. This study examined the extent to which the use of animated videos, as opposed to static text, can (i) reduce construct-irrelevant variance in test scores and (ii) have a positive impact on test-takers’ reactions to the test. A true experiment was conducted with 129 native and non-native English speakers, using an animated-video and a text-based version of the same situational judgment test of practical knowledge. The results indicated that, overall, the variance attributed to construct-irrelevant factors (i.e., native language, English proficiency, and reading comprehension in English) was lower by 9.4% in the animated-video versus the text-based version of the test. In addition, the animated-video test was perceived by participants to be more valid, fair and enjoyable. Study participants also found the language used in the animated-video test less difficult to process, but no significant differences between the two formats were found with respect to the perceived difficulty of the content. Finally, the use of animated videos did not result in participants investing a significantly greater effort in the test. The implications of these and other findings, as well as recommendations for policy, practice, and future research are discussed in the final chapter of the thesis.
Metadata
Item Type:Thesis (PhD)
Date of Award:November 2019
Refereed:No
Supervisor(s):O'Leary, Michael and Scully, Darina
Uncontrolled Keywords:Assessment; Testing; Animated Videos; Practical Knowledge; Validity
Subjects:UNSPECIFIED
DCU Faculties and Centres:DCU Faculties and Schools > Institute of Education > School of Policy & Practice
Research Institutes and Centres > Centre for Assessment Research, Policy and Practice in Education (CARPE)
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 3.0 License. View License
Funders:Prometric Inc
ID Code:23741
Deposited On:22 Nov 2019 12:47 by Michael O'leary . Last Modified 23 Sep 2022 10:13
Documents

Full text available as:

[thumbnail of PhD Thesis_Anastasios Karakolidis]
Preview
PDF (PhD Thesis_Anastasios Karakolidis) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
3MB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record