Children with below average cognitive ability represent a substantial yet under-researched population for whom cognitive and social demands, which increase in complexity year by year, may pose significant challenges. This observational study examines the longitudinal relationship between early cognitive ability and emotional-behavioral difficulties (EBDs) between the age of three and nine. Participants include 7134 children from the population-based cohort study growing up in Ireland. Cognitive ability was measured at age three using the Picture Similarities Scale. A t-score one to two standard deviations below the mean was defined as below average cognitive ability (n = 767). EBDs were measured using the Strengths and Difficulties Questionnaire (SDQ) at three, five, and nine years of age. Generalized linear mixed models and logistic regression were used to examine the relationship. Below average cognitive ability was an independent predictor of higher longitudinal SDQ scores. After adjustment, children with below average cognitive ability were 1.39 times more likely (AOR 1.39, 95% CI 1.17–1.66, p < 0.001) to experience a clinically significant EBD between the ages of three to nine years. This study demonstrates the increased risk of EBDs for children with below average cognitive ability. A scalable method of early identification of at-risk children should be a research priority for public health, enabling early intervention for cognitive and adaptive outcomes.