Events, programs, and opportunities from MAIA. Join mailing list

Interpretability

← All weeks

This week, we will explore interpretability, the subfield of machine learning that searches for explanations of models grounded in their weights and activations, rather than their inputs and outputs. We begin with an essay that argues that interpretability is important for AI safety, reliability, and robustness. Next, we will read three research blog posts by the Anthropic interpretability team. The first is about finding “features” in model neurons that correspond to interpretable concepts. The second is about attribution graphs, which help explain how AI models arrive at their answers. The third is about the J-space, which is a technique for understanding what an AI model is “thinking about”. The main goal for this week is to understand what interpretability is, why it is important, and what our best methods and results have been so far.

Week 6 readings

Mapping the Mind of a Large Language Model

Anthropic Interpretability team, May 2024

Tracing the Thoughts of a Large Language Model

Anthropic Interpretability team, March 2025

A Global Workspace in Language Models

Anthropic Interpretability team, July 2026

Additional readings