James Whittington

Keywords prefrontal cortex hippocampus cognitive maps sequence memory working memory episodic memory recurrent neural networks neural algorithms neural representations Introduction Predicting what will happen next in novel environments is a fundamental feature of intelligent cognition. However, due to the one-dimensional (1D) and irreversible nature of time itself, we can only learn about structured environments through sequential experience.

Abstract Hippocampus is critical for memory, imagination and constructive reasoning. However, recent models have suggested that its neuronal responses can be well explained by graphs, or state-spaces, that model the transitions between experiences. Here we use simulations and hippocampal recordings to reconcile these views.

Papers Published on May 26 · Featured in Daily Papers on May 28 Abstract In this technical report, we present Zamba, a novel 7B SSM-transformer hybrid model which achieves competitive performance against leading open-weight models at a comparable scale. Zamba is trained on 1T tokens from openly available datasets and is the best non-transformer model at this scale.