Publications

Survival requires balancing reward seeking and threat avoidance, yet how distinct dopamine systems coordinate to support this remains unclear. Using a naturalistic foraging paradigm in which mice pursue water reward under threat from a monster object, we...

Postural control circuitry performs the essential function of maintaining balance and body position in response to perturbations that are either self-generated (e.g., reaching to pick up an object) or externally delivered (e.g., being pushed by another...

Animals, including humans, use coordinated facial movements to sample the environment, ingest nutrients, and communicate. Rodents, in particular, produce rhythmic facial movements during spontaneous behavior and cognitive tasks. Measuring these movements...

Animals exposed to pairings of a neutral stimulus with reward acquire a conditioned response to the neutral stimulus. A prominent hypothesis, formalized in the Temporal Difference (TD) learning algorithm, is that animals learn to predict the future reward...

The ability to estimate the value associated with a specific stimulus or action is essential for adaptive behavior. Value can be updated either incrementally through experience or rapidly by inference based on latent environmental structure. Yet, how the...

Caloric depletion induces behavioral changes that help an animal find food and restore its homeostatic balance. Hunger increases exploration and risk-taking behavior, allowing an animal to forage for food despite risks; however, it is unknown which neural...

The capacity to adapt behavior across different contexts is a fundamental feature of intelligence and is crucial for animal life, yet the mechanisms by which neural circuits are contextually reconfigured to alter their function are poorly understood. As a...

Patch foraging is a ubiquitous decision-making process in which animals decide when to abandon a resource patch of diminishing value to pursue an alternative. We developed a virtual foraging task in which mouse behavior varied systematically with patch...

The neurotransmitter dopamine plays a major role in learning by acting as a teaching signal to update the brain’s predictions about rewards. A leading theory proposes that this process is analogous to a reinforcement learning algorithm called temporal...

A hallmark of various psychiatric disorders is biased future predictions. Here we examined the mechanisms for biased value learning using reinforcement learning models incorporating recent findings on synaptic plasticity and opponent circuit mechanisms in...

To thrive in complex environments, animals and artificial agents must learn to act adaptively to maximize fitness and rewards. Such adaptive behaviour can be learned through reinforcement learning, a class of algorithms that has been successful at...

The widespread adoption of deep learning to model neural activity often relies on “black-box” approaches that lack an interpretable connection between neural activity and network parameters. Here, we propose using algorithm unrolling, a method for...

Associative learning depends on contingency, the degree to which a stimulus predicts an outcome. Despite its importance, the neural mechanisms linking contingency to behavior remain elusive. In the present study, we examined the dopamine activity in the...

Responding appropriately to potential threats before they materializeis critical to avoiding disastrous outcomes. Here we examine howthreat-coping behavior is regulated by the tail of the striatum (TS) and its dopamine input. Mice were presented with a...

Social grouping increases survival in many species, including humans1,2. By contrast, social isolation generates an aversive state (‘loneliness’) that motivates social seeking and heightens social interaction upon reunion3,4,5. The observed rebound in...

Machine learning research has achieved large performance gains on a wide range of tasks by expanding the learning target from mean rewards to entire probability distributions of rewards — an approach known as distributional reinforcement learning (RL)¹...

Dopamine neurons in the ventral midbrain play important roles in various behavioral functions such as learning, motivation, and movement. The exact mechanisms and algorithms by which dopamine regulates these functions, however, remain to be clarified...

This chapter discusses diversity in the signaling properties of dopamine neurons, with a particular focus on their activity and function in response to reward and threat. While dopamine neurons were traditionally reported to be excited by reward-related...

In this chapter, we discuss anatomical and functional inputs and outputs of dopamine neurons. Classical tracing studies show that the midbrain areas where dopamine neurons are located receive projections from various brain areas, and dopamine neurons, in...

The most influential account of phasic dopamine holds that it reports reward prediction errors (RPEs). The RPE-based interpretation of dopamine signaling is, in its original form, probably too simple and fails to explain all the properties of phasic...

To behave adaptively, animals must learn to predict future reward, or value. To do this, animals are thought to learn reward predictions using reinforcement learning. However, in contrast to classical models, animals must learn to estimate value using...

Spontaneous animal behaviour is built from action modules that are concatenated by the brain into sequences^1,2. However, the neural mechanisms that guide the composition of naturalistic, self-motivated behaviour remain unknown. Here we show that dopamine...

A large body of evidence has indicated that the phasic responses of midbrain dopamine neurons show a remarkable similarity to a type of teaching signal (temporal difference (TD) error) used in machine learning. However, previous studies failed to observe...

Reinforcement learning models of the basal ganglia map the phasic dopamine signal to reward prediction errors (RPEs). Conventional models assert that, when a stimulus predicts a reward with fixed delay, dopamine activity during the delay should converge...

In the brain, dopamine is thought to drive reward-based learning by signaling temporal difference reward prediction errors (TD errors), a ‘teaching signal’ used to train computers. Recent studies using optogenetic manipulations have provided multiple...

Learning from successes and failures often improves the quality of subsequent decisions. Past outcomes, however, should not influence purely perceptual decisions after task acquisition is complete since these are designed so that only sensory evidence...

Learning about rewards and punishments is critical for survival. Classical studies have demonstrated an impressive correspondence between the firing of dopamine neurons in the mammalian midbrain and the reward prediction errors of reinforcement learning...

Different regions of the striatum regulate different types of behavior. However, how dopamine signals differ across striatal regions and how dopamine regulates different behaviors remain unclear. Here, we compared dopamine axon activity in the ventral...

Reinforcement-learning theories provide a normative perspective on learning and decision-making. In the 1990s, neurophysiology experiments revealed an exceptional correspondence between the activity of midbrain dopamine neurons and the reward prediction...

Rapid phasic activity of midbrain dopamine neurons is thought to signal reward prediction errors (RPEs), resembling temporal difference errors used in machine learning. However, recent studies describing slowly increasing dopamine signals have instead...

Since its introduction, the reward prediction error theory of dopamine has explained a wealth of empirical phenomena, providing a unifying framework for understanding the representation of reward and value in the brain^1,2,3. According to the now canonical...

Midbrain dopamine signals are widely thought to report reward prediction errors that drive learning in the basal ganglia. However, dopamine has also been implicated in various probabilistic computations, such as encoding uncertainty and controlling...

Midbrain dopamine neurons are well known for their role in reward-based reinforcement learning. We found that the activity of dopamine axons in the posterior tail of the striatum (TS) scaled with the novelty and intensity of external stimuli, but did not...

Learning to predict future outcomes is critical for driving appropriate behaviors. Reinforcement learning (RL) models have successfully accounted for such learning, relying on reward prediction errors (RPEs) signaled by midbrain dopamine neurons. It has...