Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Introduction Anthropic recently released Stage-Wise Model Diffing, which presents a novel way of tracking how transformer features change during fine-tuning. We've replicated this work on a TinyStories-33M language model to study feature changes in a more accessible research context. Instead of SAEs we worked with single-model all-layer crosscoders, and found that the technique is also effective with cross-layer features. This post documents our methodology.
We establish the Hatsugai-Kohmoto model as a stable quartic fixed point (distinct from Wilson-Fisher) by computing the β function in the presence of perturbing local interactions. In vicinity of the half-filled doped Mott state, the β function vanishes for all local interactions regardless of their sign. The only flow away from the HK model is through the superconducting channel which lifts the spin degeneracy as does any ordering tendency.
Recently, models with long-range interactions—known as Hatsugai-Kohmoto (HK) models—have emerged as a promising tool to study the emergence of superconductivity and topology in strongly correlated systems. Two obstacles, however, have made it difficult to understand the applicability of these models, especially to topological features: they have thermodynamically large ground state degeneracies and they tacitly assume spin conservation.
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
