Deep learning is a more recent form of machine learning based on a set of algorithms that attempt to learn using a deep graph with multiple processing layers, where layers are composed of multiple linear and non-linear transformational nodes. While research in this area has shown to improve the predictive accuracy in a number of domains, deep learning systems are highly complex and experiments can be hard to manage. In this dissertation, we present a deep learning system, built from scratch, which enables fully configurable deep learning experiments. By configurable, we mean selecting the overall learning algorithm, the number of layers within the deep network, the nodes within network layers and the propagation functions deployed at each node. We use a range of deep network configurations together with different datasets to illustrate the potential of this system but also to highlight the difficulties in tuning the model and hyper-parameters to maximise accuracy. Our research also provides a conceptual data model to capture all aspects of deep learning experiments. By specifying a conceptual model, it provides a platform for the storage and management of experimental snapshots, a key support for experiment and parameter optimisation and analysis. In addition, we developed a toolkit which supports the management and analysis of deep learning experiments and provides a new method for pausing and calibrating experiments. It also offers possibilities for interchanging experiment setup and results between deep learning researchers. Our validation takes the form of a series of case studies built from the requirements of end users and demonstrates the effectiveness of our toolkit in building deep learning algorithms.