Serotonin and Dopamine: Unifying Affective, Activational ...

1 Serotonin and Dopamine: Unifying Affective, Activational , and Decision FunctionsRoshan Cools*,1, Kae Nakamura2,3and Nathaniel D Daw41 Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Centre for Cognitive Neuroimaging,Nijmegen, The Netherlands;2 Department of Physiology, School of Medicine, Kansai Medical University, Moriguchi City,Japan;3 PRESTO, Honcho Kawaguchi, Saitama, Japan;4 Center for Neural Science & Department of Psychology, New YorkUniversity, New York, NY, USAS erotonin, like dopamine (DA), has long been implicated in adaptive behavior, including decision making and reinforcementlearning. However, although the two neuromodulators are tightly related and have a similar degree of functional importance,compared with DA, we have a much less specific understanding about the mechanisms by which Serotonin affects , we draw on recent work on computational models of dopaminergic function to suggest a framework by which many ofthe seemingly diverse functions associated with both DA and serotoninFcomprising both affective and Activational ones, aswell as a number of other functions not overtly related to eitherFcan be seen as consequences of a single root Reviews(2011)36,98 113; ; published online 25 August 2010 Keywords:aversion; reward; inhibition; impulsivity; activation.

2 Punishment INTRODUCTIONThe ascending monoamine neuromodulatory systems areimplicated in healthy and disordered functions so wideranging and so apparently heterogeneous that characteriz-ing their function more crisply is an important scientificpuzzle. In the case of dopamine (DA)Fwhich is involved incognition, motivation, and movementFnotable progresshas been made in the last decade using an interdisciplinaryand interspecies approach. In particular, computationalmodels of reinforcement learning (RL: trial-and-errorlearning to obtain rewards) have been used as a frameworkformally to interpret and connect observations fromneurophysiological, brain imaging, and behavioral/pharma-cological studies in humans and contrast, although the neuromodulator Serotonin (5-HT) has functional and clinical importance at least equalto that of DA (eg, it is implicated in impulsivity, depression,and pain), there is no similarly formal and well-developedframework for understanding any of its roles.

3 Here, we takeearly steps toward such a theoretical framework byreviewing aspects of function that have been prominentlyassociated with 5-HT, namely, aversive processing andbehavioral inhibition, and leveraging the example of DA tosuggest how the data supporting these ideas might beinterpreted, together with other functions, as manifestationsof a common, underlying computational particular, we consider the implications of a recentcomputational theory of DA (Nivetal, 2007) for offering acommon explanation for a number of seemingly distinctfunctional associations of both DA and 5-HT. We discussthe theory informally (omitting equations) and use it as aframework to discuss studies using psychopharmacologicalmanipulations of 5-HT in humans and experimentalrodents, as well as single-neuron recording studies in non-human primates.

4 In the first half of the review, we discusshow Nivetal s concept of an opportunity cost of time offersa common explanation for both affective (reward andpunishment) and Activational (behavioral vigor and with-holding) aspects of the neuromodulators functions. Afterthis, we develop this framework to discuss how a numberof additional, seemingly disparate, aspects of decisionmaking that have been associated with these systems, suchas time discounting and risk sensitivity, can also be seenas consequences of the same mechanism. Throughout, westress many caveats, interpretation difficulties, and experi-mental concerns; our goal here is to articulate a set ofimportant behaviors, computations, and quantities thatmight guide more definitive experiments. In addition,similar to Boureau and Dayan (2010; this issue) (see alsoDayan and Huys, 2008 and Daw, Kakade and Dayan, 2002),Received 12 April 2010; revised 16 July 2010.

5 Accepted 16 July 2010*Correspondence: Dr R Cools, Centre for Cognitive Neuroimaging,Donders Institute for Brain, Cognition and Behaviour, Radboud UniversityNijmegen, Kapittelweg 29, Nijmegen 6500HB, The Netherlands, Tel: +31243 610 656, Fax: +31 243 610 989, E-mail: 36,98 113&2011 Nature Publishing Group All rights reserved 0893-133X/11$ overall strategy is to push outward from our relativelysecure understanding of DA, through what is known aboutthe similarity and differences in DA and 5-HT functions andabout how the two neuromodulators interact, to extrapolatea tentative extended understanding encompassing DAand 5-HT collectively in a common framework. Boureauand Dayan take a complementary approach, offering,in particular, a more detailed discussion of the nature ofinteractions between DA and 5-HT, and between rewardand punishment in the context of different components , REINFORCEMENT, AND BEHAVIORALACTIVATIONThe puzzles and controversies of DA have long centeredaround the question of how to understand its seeminglydual function in both reward and movement (Ungerstedt,1971; Lyon and Robbins, 1975; Milner, 1977; Evenden andRobbins, 1984; Berridge and Robinson, 1998; Ikemotoand Panksepp, 1999; Schultz, 2007).

6 On the one hand, DAis implicated in motivation and reinforcement, for instance,it is a focus of drugs of abuse and self-stimulation. On theother, it is a facilitator of vigorous action: consider thepoverty of movement that accompanies dopaminergicdegeneration in Parkinson s disease (PD) or the hyperactivityand stereotypy engendered by psychostimulant drugs thatenhance DA, such as methamphetamine (Lyon and Robbins,1975; Robbins and Sahakian, 1979). In principle, these twoaxes of behavior might be independent, but they appearinstead to be closely coupled through the action of , one early hypothesis (Mogensonetal, 1980)characterized the nucleus accumbens (a key dopaminergictarget) as the limbic-motor gateway in which motivationalconsiderations gained access to the control of this idea, more recent RL theories link theseaspects by claiming that DA is involved in learning whichbehaviors are associated with reward.

7 Variants of thereward/action duality also underlie longstanding contro-versies about what psychological aspects of reward DAmight subserveFfor instance, hedonics, reinforcement, ormotivational and Activational (Ikemoto and Panksepp, 1999;Berridge, 2007; Robbins and Everitt, 2007)Fand thequestion whether DA impacts behavior via learning versusperformance (Gallisteletal, 1974; Berridge, 2007; Nivetal,2007). We focus on this last question , given DA s dual nature, theories of itsfunction have grown largely separately on two tracks,rooted in different experimental methodologies and theore-tical approaches. The predominant view in computationaland systems neuroscience holds that DA serves to promoteRL, that is, trial-and-error instrumental learning, to chooserewarding actions (Houketal, 1995; Montagueetal, 1996;Schultzetal, 1997; Samejimaetal, 2005; Morrisetal, 2006).

8 This idea is derived from electrophysiological recordingsfrom neurons in the midbrain dopaminergic nuclei ofprimates performing simple tasks for reward (Ljungbergetal, 1991; Hollerman and Schultz, 1998; Waeltietal, 2001),together with the insight that the phasic firing of theseneurons quantitatively resembles a reward prediction error signal used in computational algorithms for RL to improveaction choice so as to obtain more rewards (Sutton andBarto, 1990; Montagueetal, 1996; Sutton and Barto, 1998;Montagueetal, 2004; Bayer and Glimcher, 2005;Frank, 2005). More recently, studies employing temporallyprecise methods in freely behaving animals, such aselectrochemical voltammetric approaches, which enablethe measurement of phasic DA release directly (Dayetal,2007; Roitmanetal, 2008), as well as optogeneticapproaches, which enable the transient activation of specificDA neurons (Tsaietal, 2009), have substantiated theseideas.

9 Furthermore, functional neuroimaging has revealedthat similar prediction error signals in humans (McClureetal, 2003; O Dohertyetal, 2003) might be modulated byDA (Pessiglioneetal, 2006), whereas microelectroderecordings during deep brain stimulation surgery havedemonstrated that such prediction error signals are alsoencoded by the human midbrain (Zaghlouletal, 2009) (seealso D Ardenneetal, 2008).At the same time, more psychological approaches, largelygrounded in causal manipulations (eg, drug or lesion) ofdopaminergic function, tend to envision DA as beinginvolved less in acquisition and more in the performanceof motivated behavior. Indeed, the most pronounced effectsof causal DA manipulations tend to be on performancerather than learning, with DA promoting behavioral vigoror activation more generally (Lyon and Robbins, 1975;Ikemoto and Panksepp, 1999; Berridge, 2007; Robbins andEveritt, 2007; Salamoneetal, 2007).

10 Two current inter-pretations characterize these effects as arising via dopami-nergic mediaton of incentive motivation (Berridge, 2007) orcost/benefit tradeoffs (Salamoneetal, 2007). Other authorswriting from a similar tradition have provided a moregeneral Activational account, with parallel roles for DA inthe dorsal and ventral striatum (Robbins and Everitt, 1982,1992; Robbins and Everitt, 2007), stressing both a perfor-mance-based energetic component to DA and reinforce-ment-related functions more akin to those posited in thecomputational RL models, for example, conditioned re-inforcement and stamping-in of stimulus response habits(Wise, 2004). Indeed, early experimental work by Gallisteletal(1974) argued for both reinforcing and activationaleffects of (putatively dopaminergic) brain stimulationreward, distinguished as progressive and immediate effectsof contingent versus noncontingent THE DUAL FUNCTION OF DAOne attempt to reconcile these two streams of thought(Nivetal, 2007) extended RL accounts, which hadtraditionally focused on learning which action is mostrewarding, into an additional formal analysis of howMultiple functions of Serotonin and dopamineR Coolset these actions should be performed.