Kola Ayonrinde

Kola Ayonrinde, Anthropic Fellows, koayon@gmail.com; Jack Lindsey, Anthropic TL;DR: The J-Lens was proposed to read verbalisable representations from language model activations. We introduce the J++ Lens: an improvement to the J-Lens that enables more faithful readouts across 9–284B models. The J++ Lens filters noisy gradients before they enter the averaged Jacobian.

TL;DR: We recently released two papers about the Philosophy of (Mechanistic) Interpretability [here and here] and a reading list [here]. We believe that building a foundation for interpretability which leverages lessons from other disciplines (philosophy, neuroscience, social science) can help us understand AI models. We also believe this is a useful area for philosophers, neuroscientists and human-computer interaction (HCI) researchers to contribute to AI Safety.

This work was produced as part of the ML Alignment & Theory Scholars Program - Summer 24 Cohort. TL;DR The current approach to explaining model internals is to (1) disentangle the language model representations into feature activations using a Sparse Autoencoder (SAE) and then (2) explain the corresponding feature activations using an AutoInterp process.