Generating semantically consistent text remains a crucial challenge in Natural Language Processing (NLP), especially in input-controlled settings such as data-to-text generation. Despite advances in neural language generation, current models often produce outputs that contain semantic inconsistencies with the input, including omissions, hallucinations, and distortions. These issues stem from the black-box, non-deterministic nature of neural models, which limits their interpretability and controllability. This thesis investigates how semantic consistency, the property that all information conveyed by the output is entailed by the input, can be more reliably evaluated and ultimately improved in generated text. Focusing on data-to-text generation, the thesis proposes that advancing semantic controllability requires a combination of principled formalisation of semantic errors and robust evaluation methods. First, it synthesises prior work on error annotation to develop a task-agnostic consensus taxonomy for semantic errors, distinguishing omission, addition, and substitution errors. Second, it applies this taxonomy in human evaluation studies, including span-based semantic error annotation to better understand the nature and reliability of human judgements. Third, it assesses the capacity of automatic evaluation methods, including lexical, embedding-based, and prompt-based approaches, to reflect human ratings of semantic consistency. In this context, the thesis explores LLM-as-judge setups for scoring system outputs and uses LLMs to sanity-check repeated human evaluation experiments, investigating their generalisability and reliability. Together, these contributions provide new tools insights, approaches and results for the fine-grained assessment of semantic consistency and lay the groundwork for more interpretable, reliable, and scalable evaluation frameworks in neural language generation.