Sentiment Analysis techniques enable the automatic extraction of sentiment in social media data, including popular platforms as Twitter. For retailers and marketing analysts, such methods can support the understanding of customers' attitudes towards brands, especially to handle crises that cause behavioural changes in customers, including the COVID-19 pandemic. However, with the increasing adoption of black-box machine learning-based techniques, transparency becomes a need for those stakeholders to understand why a given sentiment is predicted, which is rarely explored for retailers facing social media crises. This study develops an Explainable Sentiment Analysis (XSA) application for Twitter data, and proposes research propositions focused on evaluating such application in a hypothetical crisis management scenario. Particularly, we evaluate, through discussions and a simulated user experiment, the XSA support for understanding customer's needs, as well as if marketing analysts would trust such an application for their decision-making processes. Results illustrate the XSA application can be effective in providing the most important words addressing customers sentiment out of individual tweets, as well as the potential to foster analysts' confidence in such support.