Kaya Arkin (2105361)
Exploratory Research on Explainable LLMs (Airbus AI Research)

Project Abstract
Motivated by the need for increased trust and transparency in generative AI systems, this project seeks to develop a set of recommendations and insights on explainability for LLMs, intended for future use by Airbus AI Research. Focusing on the aviation sector, the research leverages a database of airline incidents (Federal Aviation Administration Service Difficulty Report) to fine-tune a DistilBERT model that predicts component failures from incident reports. To explain the model’s prediction, a counterfactual analysis approach was employed, where input words were iteratively replaced with their synonyms or antonyms, and the resulting changes in predictions were observed. An interactive application was developed to allow users to explore these counterfactuals directly, offering a hands-on way to investigate the model’s behaviour. The analysis provided meaningful insights into the LLM’s inner workings and helped uncover gaps in the training data. However, the results also indicated that a more intelligent, domain-aware strategy for word replacement would significantly improve the quality and relevance of the explanations.
Keywords: Artificial Intelligence (AI), XAI, Aerospace
Conference Details
Session: A
Location: Sir Stanley Clarke Auditorium at 11:00 13:00
Markers: Bertie Muller, Eike Neumann
Course: BSc Computer Science with a Year in Industry 4yr FI
Future Plans: I’m looking for work