This paper presents results of the efect of diferent translation modalities on users when working with the Microsoft Word user interface. An experimental study was set up with 84 Japanese, German, Spanish, and English native speakers working with Microsoft Word in three modalities: the published translated version, a machine translated (MT) version (with unedited MT strings incorporated into the MS Word interface) and the published English version. An eye-tracker measured the cognitive load and usability according to the ISO/TR 16982 guidelines: i.e., efectiveness, efciency, and satisfaction followed by retrospective think-aloud protocol. The results show that the users’ efectiveness (number of tasks completed) does not signifcantly difer due to the translation modality. However, their efciency (time for task completion) and self-reported satisfaction are signifcantly higher when working with the released product as opposed to the unedited MT version, especially when participants are less experienced. The eye-tracking results show that users experience a higher cognitive load when working with MT and with the human-translated versions as opposed to the English original. The results suggest that language and translation modality play a signifcant role in the usability of software products whether users complete the given tasks or not and even if they are unaware that MT was used to translate the interface.