Alessio Miaschi is a Post Doctoral Researcher at the ItaliaNLP Lab from the Institute for Computational Linguistics “A. Zampolli” (ILC-CNR, Pisa).
He received his PhD in Computer Science from the University of Pisa in 2022 with a thesis focused on the definition of techniques for interpreting and understanding the linguistic knowledge implicitly encoded in recent state-of-the-art Neural Language Models.
His current research interest mainly focuses on the development and analysis of neural network models for language processing, as well as on the definition of NLP tools for educational applications. Since 2020, he has been working as a teaching assistant for the Computational Linguistics courses at the University of Pisa.
The field of Natural Language Processing (NLP) has seen unprecedented progress in the last few years. Much of this progress is due to the replacement of traditional systems with newer and more powerful algorithms based on neural networks and deep learning.
This improvement, however, comes at the cost of interpretability since deep neural models offer little transparency about their inner workings and their abilities.
Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of these models.
This talk is divided into two parts. In the first part, we briefly introduce Neural Language Models (NLMs) and the main techniques developed for interpreting their decisions and inner linguistic knowledge.
In the second part, we l look at how to fine-tune one of the most popular NLM and then analyze its decisions according to two different interpretability methods: integrated gradients and analysis of attention matrices.