1. Incentive landscapes that can’t feasibly be induced by a reward function You’re a deity, tasked with designing a bird brain. You want the bird to get good at singing, as judged by a black-box hardcoded song-assessing algorithm that you already built into the brain last week. The bird chooses actions based in part on within-lifetime reinforcement learning involving dopamine. What reward signal do you use? Well, we want to train the bird to sing the song correctly.
