Johns Hopkins School of Medicine

Dopamine and serotonin signals for reward across time scales

Jeremiah Y. Cohen
Solomon H. Snyder Department of Neuroscience, Brain Science Institute, Johns Hopkins University School of Medicine, Baltimore, MD 21205, USA. 
*E-mail: jeremiah.cohen@jhmi.edu

Science 2 October 2015:

 Vol. 350  no. 6256  pp. 47-48 
 DOI: 10.1126/science.aad3003 

•Essays on Science and Society
It is one of the peculiar features of most modern neurophysiology that the experimentalist…seldom knows which type of neuron he or she is listening to…. This problem deserves immediate and serious attention.

Francis Crick, 1999 (1)

Neurobiology has thrived on the consilience of multiple fields. For most of the 20th century, this meant combining chemistry and neuroanatomy [“The gains in brain are mainly in the stain” (2)], as well as electronics and neurophysiology (the gains in brain are mainly in the amplifier's gain, if you like). Recent years have seen the addition of molecular biology to our bag of tricks. We wished to take advantage of tools from different fields to understand the relationship between neurons that release dopamine or serotonin and reward (3, 4).

Beginning with an accidental discovery in 1953, Olds and colleagues found that electrical stimulation of particular brain areas was powerfully rewarding: Rats would compulsively press a lever to obtain stimulation, ignoring naturally rewarding stimuli such as food, water, or mates (5, 6). This discovery inspired researchers to try to determine the neural basis of such reinforcement.

Dopamine is a key neurotransmitter for reward in the brain. In the 1990s, Schultz and colleagues discovered that dopaminergic neurons in the ventral tegmental area (VTA) were excited by unexpected reward, whereas their response to reward was diminished when a sensory cue predicted the reward (7). That is, dopaminergic neurons are excited when the outcome is better than expected, whereas they are inhibited when the outcome is worse than expected. This suggests that the brain's reward system is activated only when the brain fails to predict outcomes correctly. Such a signal is called “reward prediction error” (RPE).

These findings changed the way we think about the reward system, but how such calculations are performed in the brain remains unknown. A number of related reinforcement-learning models have suggested that a value-dependent inhibitory signal might counteract the excitatory reward signal to suppress dopaminergic output for expected, but not unexpected, rewards (8). To determine whether this is how the system works in vivo, we aimed to record the signals of different types of neurons in the VTA of mice during a reward-prediction task.

As Crick pointed out in 1999, a challenge from the outset is how to determine what types of neuron are being recorded during such experiments. In classical neurophysiological studies, the experimenter places a microelectrode in an animal's brain and compares neuronal activity with the animal's behavior. We “tagged” dopaminergic neurons (and, in separate experiments, inhibitory GABAergic neurons) with a light-activated protein, channelrhodopsin-2 (9, 10). Using transgenic mice that express the enzyme Cre recombinase in either dopaminergic or GABAergic neurons, we introduced Cre-dependent viruses carrying channelrhodopsin-2 (9, 10), which created light-sensitive dopaminergic or GABAergic neurons. We then implanted several small electrodes and an optical fiber into the VTA. While recording the activity of randomly sampled neurons, we noted whether each neuron responded to a pulse of light from a laser. If it did, then we knew it was a dopaminergic (or GABAergic) neuron.

~~We recorded from VTA neurons while mice performed a classical conditioning task, in which odors predicted different rewards or punishments. Our first surprise came when we discovered that every neuron we observed showed one of three distinct types of activity. Type I neurons showed RPE-like activity, firing brief bursts of action potentials when the mouse smelled a reward-predicting odor (or the reward itself). These neurons showed decreases in activity relative to baseline when the reward was unexpectedly omitted. Type II neurons showed reward-expectation–like activity, increasing their firing in a sustained way when the mouse smelled a reward-predicting odor. These neurons fired regardless of whether the reward was delivered as expected or unexpectedly omitted. Type III neurons showed activity that mirrored type II neurons, except that firing rates decreased in anticipation of a reward rather than increased. Our second surprise came when we compared the activity of all neurons to the neurotransmitter they released (dopamine or GABA). We found that dopaminergic neurons were all type I and that GABAergic neurons were all type II. The identity of the type III neurons has yet to be determined.

~~We next asked whether dopaminergic neurons signaled only the immediate properties of the environment (for example, “I'm about to get a reward”), or whether slower dopaminergic signaling could underlie longer-term emotional states. We varied the amount of reward slowly over time, and were surprised to find no evidence of longer-term reward signaling in dopaminergic neurons. How, then, do our enduring mood states arise?

To answer this question, we turned our attention to a different neuromodulator: serotonin. Serotonin is thought to be involved in many behaviors, but its function has been elusive (11). This is due, in part, to the challenges in recording from identified serotonergic neurons during behavior. We took an approach similar to the one described above, tagging serotonergic neurons with channelrhodopsin-2 and establishing criteria for identifying these cells (3). We recorded the activity of these light-identified serotonin neurons as mice participated in a task in which the amount of reward or punishment varied predictably over time. We found that 40% of dorsal raphe serotonergic neurons showed slow variations in the activity that correlated with the amount of reward in the environment. This was remarkable, particularly given that the VTA dopaminergic neurons did not respond to this information. In addition, many serotonergic neurons signaled short-term information about upcoming rewards and punishments with brief changes in firing rates. This suggests that serotonergic neurons have the ability to signal reward and punishment on both slow and fast time scales and may be involved in generating emotional states like mood.