Generative multimodal models based on diffusion models
have seen tremendous growth and advances in recent years.
Models such as DALL-E and Stable Diffusion have become
increasingly popular and successful at creating images from
texts, often combining abstract ideas. However, like other
deep learning models, they also reflect social biases they inherit from their training data, which is often crawled from
the internet. Manually auditing models for biases can be very
time and resource consuming and is further complicated by
the unbounded and unconstrained nature of inputs these models can take. Research into bias measurement and quantification has generally focused on small single-stage models
working on a single modality. Thus the emergence of multistage multimodal models requires a different approach. In
this paper, we propose Multimodal Composite Association
Score (MCAS) as a new method of measuring gender bias
in multimodal generative models. Evaluating both DALL-E 2
and Stable Diffusion using this approach uncovered the presence of gendered associations of concepts embedded within
the models. We propose MCAS as an accessible and scalable
method of quantifying potential bias for models with different
modalities and a range of potential biases.
Advances in Bias and Fairness in Information Retrieval. Communications in Computer and Information Science
1840.
Springer Nature Switzerland. ISBN 978-3-031-37248-3