From Interpretability Methods to Interpretable Models

Abstract

Deep learning models are increasingly deployed for high-stakes settings, yet their decision-making remains largely opaque to the people who rely on them. This thesis argues that, alongside developing better tools to explain those opaque models, explainable AI (XAI) should seek to understand the principles that make models less opaque in the first place. To this end, it develops a framework grounded in large-scale human experiments to measure, compare, and shape the interpretability of vision models. First, the thesis investigates the appropriate representational basis for interpretability, showing that dictionary-learned features are more interpretable than individual neurons. Next, it introduces two behavioral protocols to measure human understanding of visual features and systematically compare interpretability across models. These reveal that vision foundation models are less interpretable than their supervised baseline, a gap best predicted by feature locality and alignment with human semantic judgments. Finally, the thesis investigates whether interpretability can be improved by aligning models with human semantic judgments, finding that this approach benefits some models but not others. Together, these results provide the first behaviorally grounded account of how interpretable vision foundation models are, why they differ, and how far their interpretability can be deliberately improved.

Bio

Julien Colin is an ELLIS PhD student at ELLIS Alicante, working under the supervision of Nuria Oliver (ELLIS Alicante) and Thomas Serre (brown University). He holds a Master's degree in Artificial and Natural Cognition (2021, Grenoble INP - Phelma). His research focuses on what makes AI models understandable to humans: how to measure it, how to shape it, and the mechanisms behind it.

Watch the thesis defense live here

Details about the committee can be found here

Main publications discussed in the thesis

Colin, J., Oliver, N., & Serre, T (2026). From Interpretability Methods to Interpretable Models. ECCV Workshop on Explainable Computer Vision. ECCV

Colin, J., Goetschalckx, L., Fel, T., Boutin, V., Serre, T., & Oliver, N. (2026). Choosing the Right Basis for Interpretability: Psychophysical Comparison Between Neuron-based and Dictionary-based Representation. Transactions on Machine Learning Research. TMLR

Colin, J., Oliver, N., & Serre, T (2026). Can Human Perceptual Similarity Alignment Improve Interpretability in Visual Representations?. Conference on Cognitive Computational Neuroscience (CCN)[https://2026.ccneuro.org/poster/?id=7cDVJL4Kgo]

Colin, J., Goetschalckx, L., Oliver, N., & Serre. T. (2026). Capability $\neq$ Interpretability: Human Interpretability of Vision Foundation Models. arXiv

Colin, J., Oliver, N., & Serre, T (2026). Does Human-Alignment Benefit Interpretability?. ICLR Workshop on Representational Alignment ($Re^4$-Align) ICLR