Neural architecture search using particle swarm and ant colony optimization
Lankford, Séamus and Grimes, DiarmuidORCID: 0000-0001-5551-6504
(2020)
Neural architecture search using particle swarm and ant colony optimization.
In: AICS 2020 Artificial Intelligence and Cognitive Science, 7-8 Dec 2020, Dublin, Ireland.
Neural network models have a number of hyperparameters
that must be chosen along with their architecture. This can be a heavy
burden on a novice user, choosing which architecture and what values
to assign to parameters. In most cases, default hyperparameters and architectures are used. Significant improvements to model accuracy can
be achieved through the evaluation of multiple architectures. A process
known as Neural Architecture Search (NAS) may be applied to automatically evaluate a large number of such architectures. A system integrating open source tools for Neural Architecture Search (OpenNAS), in the classification of images, has been developed as part of this research. OpenNAS takes any dataset of grayscale, or RBG images, and generates Convolutional Neural Network (CNN) architectures based on a range of metaheuristics using either an AutoKeras, a transfer learning or a Swarm Intelligence (SI) approach. Particle Swarm Optimization (PSO) and Ant Colony Optimization (ACO) are used as the SI algorithms. Furthermore, models developed through such metaheuristics may be combined using stacking ensembles.
In the context of this paper, we focus on training and optimizing CNNs
using the Swarm Intelligence (SI) components of OpenNAS. Two major
types of SI algorithms, namely PSO and ACO, are compared to see which
is more effective in generating higher model accuracies. It is shown, with
our experimental design, that the PSO algorithm performs better than
ACO. The performance improvement of PSO is most notable with a more
complex dataset. As a baseline, the performance of fine-tuned pre-trained
models is also evaluated.