Neural networks have demonstrated remarkable success in various domains but they often struggle with generalization beyond their training data. To address these limitations and enhance the robustness of machine learning models, this thesis explores the integration of domain knowledge into neural networks through two approaches, network analysis and ordinary differential equations (ODEs). We begin by investigating neural network performance in diverse tasks, such as hyperglycemia/hypoglycemia diagnosis using exosome profiles and oxygen uptake estimation from sensor measurements. The study then progresses to more structured data with complex networks. Subsequently, we incorporate network structure into machine learning using graph neural networks, applying this method to an air quality forecasting task where locations and their correlations form a network. An alternative approach is then investigated by integrating ODE systems describing dynamical systems into a data-driven machine learning framework. This comprises the development of advanced techniques to enable neural networks to learn underlying physics, including ODE Normalization, Gradient Balancing, Causal Training, and Domain Decomposition. These methods address challenges in training with stiff systems across large domains. The frameworks in this research are then validated using simulated data for the Lorenz system and a system of ODEs modelling mosquito populations. This work is further developed to accommodate real-life observations, by making adjustments to model inputs, neural network architecture, and activation functions. This extended framework is then evaluated against real-world mosquito counts in an inverse problem setting, learning relationships between meteorological conditions and mosquito development. Our results demonstrate that incorporating domain knowledge into neural networks enhances model generalizability, improving both accuracy and extrapolation capabilities. Moreover, this approach maintains the explainability of the added knowledge while leveraging the flexibility of machine learning models.