Kats, Daniel, Ray, Mondira, Moorkens, Joss
ORCID: 0000-0003-0766-0071, Rai, Dinesh, Brewster, Ryan C. L., Khan, Alisa, Rader, Benjamin, Brownstein, John S. and Hron, Jonathan
(2026)
GPT-4o Translation of Pediatric Discharge Instructions Across Languages.
Hospital Pediatrics
.
ISSN 2154-1671
Abstract
BACKGROUND AND OBJECTIVES
Patients and caregivers who use languages other than English in health care settings rarely receive language-concordant discharge instructions. Large language models show promise for improving access to translations but must be validated to ensure their output is safe for clinical use. This study measured the quality of GPT-4o medical translations to Portuguese, Haitian Creole, and Arabic.
METHODS
Previously translated personalized discharge instructions (performed by professional human translators) were extracted from inpatient admissions at a large US pediatric academic medical center. The discharge instructions were translated to Portuguese, Haitian Creole, and Arabic by GPT-4o. Medical translators then evaluated the translations using both the Multidimensional Quality Metrics (MQM) framework and a preference scale.
RESULTS
Translations of 14 to 20 discharge instructions for each of the 3 languages were evaluated. Noninferiority testing showed no significant difference in translation quality between GPT-4o and human translations for Portuguese and Arabic, but worse performance by GPT-4o than human translations for Haitian Creole (mean MQM score 90.5 ± SD 5.5 vs 97.0 ± SD 2.1). Linguists preferred the human translation over GPT-4o in 69% (SE = 7%) of the Haitian Creole cases.
CONCLUSIONS
In this cross-sectional study, GPT-4o translation performance of pediatric discharge instructions was noninferior for Portuguese and Arabic but was worse in Haitian Creole. Human validation of medical translations remains necessary for all languages, but large language models can be used to draft high-quality translations in languages with proven high performance, while resources are shifted toward more intensive post-editing of translations in languages with lower performance.
Metadata
| Item Type: | Article (Published) |
|---|---|
| Refereed: | Yes |
| Subjects: | Social Sciences > Education Social Sciences > Linguistics and Language |
| DCU Faculties and Centres: | DCU Faculties and Schools > Faculty of Humanities and Social Science DCU Faculties and Schools > Faculty of Humanities and Social Science > School of Applied Language and Intercultural Studies Research Institutes and Centres > ADAPT |
| School, Faculty or Unit: | Faculty of Humanities & Social Science Faculty of Humanities & Social Science > SALIS |
| Publisher: | American Academy of Pediatrics |
| Official URL: | https://publications.aap.org/hospitalpediatrics/ar... |
| Copyright Information: | Authors |
| Funders: | Biomedical Informatics and Data Science Research Training Program, Research Ireland at the ADAPT Centre, Dublin City University [13/RC/2106_P2] |
| ID Code: | 33567 |
| Deposited On: | 16 Nov 2026 09:48 by Joss Moorkens . Last Modified 16 Nov 2026 09:48 |
Documents
Full text available as:
|
PDF
- Archive staff only. This file is embargoed until 8 October 2027
- Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Creative Commons: Attribution-Noncommercial-No Derivative Works 4.0 1MB |
Metrics
Altmetric Badge
Dimensions Badge
Downloads
Downloads
Downloads per month over past year
Archive Staff Only: edit this record