Towards explaining deep neural networks through graph analysis
Horta, Vitor A.C. and Mileo, AlessandraORCID: 0000-0002-6614-6462
(2019)
Towards explaining deep neural networks through graph analysis.
In: International Conference on Database and Expert Systems Applications DEXA 2019, 26-20 Aug 2019, Linz, Austria.
ISBN 978-3-030-27683-6
Due to its potential to solve complex tasks, deep learning is
being used across many different areas. The complexity of neural networks however makes it difficult to explain the whole decision process
used by the model, which makes understanding deep learning models
an active research topic. In this work we address this issue by extracting the knowledge acquired by trained Deep Neural Networks (DNNs)
and representing this knowledge in a graph. The proposed graph encodes
statistical correlations between neurons’ activation values in order to expose the relationship between neurons in the hidden layers with both the
input layer and output classes. Two initial experiments in image classification were conducted to evaluate whether the proposed graph can help
understanding and explaining DNNs. We first show how it is possible to
explore the proposed graph to find what neurons are the most important
for predicting each class. Then, we use graph analysis to detect groups
of classes that are more similar to each other and how these similarities
affect the DNN. Finally, we use heatmaps to visualize what parts of the
input layer are responsible for activating each neuron in hidden layers.
The results show that by building and analysing the proposed graph it
is possible to gain relevant insights of the DNN’s inner workings.