Why not try harder? Computational approach to motivation deficits in neuro-psychiatric diseases

Why not try harder? Computational approach to motivation deficits in neuro-psychiatric diseases Abstract Motivation deficits, such as apathy, are pervasive in both neurological and psychiatric diseases. Even when they are not the core symptom, they reduce quality of life, compromise functional outcome and increase the burden for caregivers. They are currently assessed with clinical scales that do not give any mechanistic insight susceptible to guide therapeutic intervention. Here, we present another approach that consists of phenotyping the behaviour of patients in motivation tests, using computational models. These formal models impose a precise and operational definition of motivation that is embedded in decision theory. Motivation can be defined as the function that orients and activates the behaviour according to two attributes: a content (the goal) and a quantity (the goal value). Decision theory offers a way to quantify motivation, as the cost that patients would accept to endure in order to get the benefit of achieving their goal. We then review basic and clinical studies that have investigated the trade-off between the expected cost entailed by potential actions and the expected benefit associated with potential rewards. These studies have shown that the trade-off between effort and reward involves specific cortical, subcortical and neuromodulatory systems, such that it may be shifted in particular clinical conditions, and reinstated by appropriate treatments. Finally, we emphasize the promises of computational phenotyping for clinical purposes. Ideally, there would be a one-to-one mapping between specific neural components and distinct computational variables and processes of the decision model. Thus, fitting computational models to patients’ behaviour would allow inferring of the dysfunctional mechanism in both cognitive terms (e.g. hyposensitivity to reward) and neural terms (e.g. lack of dopamine). This computational approach may therefore not only give insight into the motivation deficit but also help personalize treatment. apathy, computational psychiatry, goal-directed behaviour, behavioural neurology, decision-making Introduction In this review, we present testing and analytic tools that may provide better quantification and discrimination of motivation deficits. These tools are embedded in the conceptual framework of decision theory. We thus start by sketching briefly the conceptual framework, then we describe the principles of behavioural tests and computational models that can be used to assess motivation, and finally we expose the potential neural bases of motivational processes and the typical manifestations of motivation deficits in the clinics. Note that our purpose is not to advocate a particular model but to present the interests of the computational approach in general. Conceptual framework The etymology suggests that the term motivation originally refers to a force that sets the behaviour in motion. Yet when we say that someone is strongly motivated, we seem to imply that motivation is something that we can quantify in order to predict the behaviour. And when we ask about real motivations behind observed behaviours, we look for reasons expressed in terms of implicit goals. Thus, motivation can be construed as a concept with two attributes, content and quantity, that somehow determine the behaviour. The effects of motivation are the direction of behaviour, which is determined by the content (i.e. the goal), and the intensity of behaviour, which is determined by the quantity (i.e. the goal value). The issue with clinical scales While neuroscientists investigate the processes through which the brain selects and implements goals such that they can drive the behaviour, clinicians are mostly concerned with assessing the intensity of motivation in their patients. This means estimating the values that the patient assigns to standard goals that people may have in general, such as getting a good job position. It is usually done using questionnaires with multiple answers that can be scored to quantify the motivation deficit, i.e. apathy (Levy and Dubois, 2006; Drijgers et al., 2010). Different apathy rating scales have been proposed that vary in the richness of clinical details and consequently in the duration of assessment (Marin, 1990; Starkstein et al., 1992; Robert et al., 2002; Sockeel et al., 2006; Radakovic and Abrahams, 2014). These scales obviously help the patients to express their trouble with motivation. For this purpose, questions use words from common language, such as ‘Do you have motivation?’ (Question 7 in Starkstein’s apathy scale) (Table 1). Table 1 Starkstein’s apathy scale 1.  Are you interested in learning new things?  Not at all  Slightly  Some  A lot  2.  Does anything interest you?  Not at all  Slightly  Some  A lot  3.  Are you concerned about your condition?  Not at all  Slightly  Some  A lot  4.  Do you put much effort into things?  Not at all  Slightly  Some  A lot  5.  Are you always looking for something to do?  Not at all  Slightly  Some  A lot  6.  Do you have plans and goals for the future?  Not at all  Slightly  Some  A lot  7.  Do you have motivation?  Not at all  Slightly  Some  A lot  8.  Do you have the energy for daily activities?  Not at all  Slightly  Some  A lot  9.  Does someone have to tell you what to do each day?  Not at all  Slightly  Some  A lot  10.  Are you indifferent to things?  Not at all  Slightly  Some  A lot  11.  Are you unconcerned with many things?  Not at all  Slightly  Some  A lot  12.  Do you need a push to get started on things?  Not at all  Slightly  Some  A lot  13.  Are you neither happy nor sad, just in between?  Not at all  Slightly  Some  A lot  14.  Would you consider yourself apathetic?  Not at all  Slightly  Some  A lot  1.  Are you interested in learning new things?  Not at all  Slightly  Some  A lot  2.  Does anything interest you?  Not at all  Slightly  Some  A lot  3.  Are you concerned about your condition?  Not at all  Slightly  Some  A lot  4.  Do you put much effort into things?  Not at all  Slightly  Some  A lot  5.  Are you always looking for something to do?  Not at all  Slightly  Some  A lot  6.  Do you have plans and goals for the future?  Not at all  Slightly  Some  A lot  7.  Do you have motivation?  Not at all  Slightly  Some  A lot  8.  Do you have the energy for daily activities?  Not at all  Slightly  Some  A lot  9.  Does someone have to tell you what to do each day?  Not at all  Slightly  Some  A lot  10.  Are you indifferent to things?  Not at all  Slightly  Some  A lot  11.  Are you unconcerned with many things?  Not at all  Slightly  Some  A lot  12.  Do you need a push to get started on things?  Not at all  Slightly  Some  A lot  13.  Are you neither happy nor sad, just in between?  Not at all  Slightly  Some  A lot  14.  Would you consider yourself apathetic?  Not at all  Slightly  Some  A lot  Each answer is scored between 0 and 3, with the highest score corresponding to lowest motivation (leftmost answer for Questions 1–8, rightmost answer for Questions 9-14). Based on the bimodal distribution observed in patients with Parkinson’s disease, a score superior to 14 has been suggested as a marker of apathy. Although they help clinicians to get a rough idea of the motivation deficit, these scales suffer from some limitations. First, they depend on the patient’s insight, which may not be fine-grained in the case of diseases that affect cognitive functions, or on the insight of relatives and caregivers, who can be absent or too distant from the patient to give valuable information. Second, they assess entities (either motivation as a whole or subcategories such as ‘interests’ or ‘concerns’) that are not likely to have direct counterparts at the neural level. To overcome these limitations, we suggest the following 2-fold alternative strategy: (i) adding behavioural tests to questionnaires; and (ii) characterizing behavioural performance using a normative framework—namely, decision theory (Fig. 1). Figure 1 View largeDownload slide A schematic view of motivation. The box-and-arrow schema illustrates goal-directed behaviour. The brain adjusts the direction and intensity of behaviour so as to reduce the delay or increase the probability of goal attainment. Decision theoretic principles posit that agents should maximize the net value, obtained by subtracting costs (effort and time involved by the behaviour) from benefits (how much rewarding overcomes punishing aspects of the goal). To perform this optimization the brain needs an approximate anticipation of both costs and benefits. In this framework, motivation can have three different meanings: motivation-as-content refers to the goal, motivation-as-quantity refers to the goal value, motivation-as-process refers to behavioural adjustments toward the goal. Figure 1 View largeDownload slide A schematic view of motivation. The box-and-arrow schema illustrates goal-directed behaviour. The brain adjusts the direction and intensity of behaviour so as to reduce the delay or increase the probability of goal attainment. Decision theoretic principles posit that agents should maximize the net value, obtained by subtracting costs (effort and time involved by the behaviour) from benefits (how much rewarding overcomes punishing aspects of the goal). To perform this optimization the brain needs an approximate anticipation of both costs and benefits. In this framework, motivation can have three different meanings: motivation-as-content refers to the goal, motivation-as-quantity refers to the goal value, motivation-as-process refers to behavioural adjustments toward the goal. The promises of decision theory Elementary principles of decision theory assume that when considering whether or not to take a course of action, agents contrast costs and benefits to get a net value. If the action is compared to doing nothing, then it is engaged only when its net value is positive. If it is compared to a set of alternative actions, then it is engaged only when its net value is above the others.   NetValue(Actioni)= Benefit[Goal(Actioni)]-Cost(Actioni) (1) The benefit term corresponds to the value of the action goal, i.e. how good it is for the agent to engage this action. Importantly, values may be attached to actions in two different ways. On the one hand, the action may be inherently valuable, like when reading a book because it is fun. In this case the goal is the action itself, and the motivation is said to be intrinsic. On the other hand, the action may be good because it leads to a valuable outcome, like when reading a book is required to pass an examination. In this case the goal is the exam success, the motivation is extrinsic, and the behaviour qualified as instrumental. There is therefore no major conceptual difference between extrinsic and intrinsic motivation from the perspective of decision theory. Note that the goal can be multidimensional and include both positive and negative elements, which may be called gains and losses or rewards and punishments. For instance, being the captain of a team can be set as a goal because it has high positive value on the dimensions of power and self-esteem, which may overcome the negative values related to duties and responsibilities. Naturally, a given anticipated state of the world can only be set as a goal if the positive values (of rewards) exceed the negative values (of punishments). The action does not necessarily lead to the goal, i.e. the valuable state anticipated by the agent, in a direct and deterministic way. For the benefit to be positive, actions only need to bring the agent closer to the goal, by decreasing either the uncertainty or the delay attached to goal reaching. In fact, subjective estimates show that people indeed discount the goal value by both delay and uncertainty. There is a huge literature on delay and uncertainty discounting, the latter being also linked to the notion of risk, defined as the variance of possible outcomes (Frederick et al., 2002; Green and Myerson, 2004). Critically, however, delay and uncertainty are not action costs: they are only modulators of goal value. Action costs are induced by the allocation of resources required for performing the action. They can be of two sorts: time and effort. Time is a cost because if time is spent on a given course of action it is no longer available for other actions that might be profitable. Of course, this opportunity cost of time only matters for actions that cannot be executed simultaneously. Effort is a cost because effortful actions consume resources that might be needed later and will have to be restored through some other costly actions. This is obvious in the case of physical effort, which consumes energetic resources and fatigues the muscles such that they may not operate efficiently for later purposes. It is not that obvious in the case of mental effort, for which the idea of a biological resource being consumed is still debated (Inzlicht et al., 2014; Botvinick and Braver, 2015). Indeed most of, if not all, the energy consumed by the brain is used for maintaining spontaneous activity, i.e. activity that is not related to a particular cognitive task (Raichle and Gusnard, 2005). This has led some authors to argue that people avoid mental effort because of the opportunity cost induced by cognitive resource allocation. The idea is that when we engage central cognitive modules in a given task, they are no longer available for other possible beneficial tasks (Kurzban et al., 2013; Boureau et al., 2015). In any case, people tend to avoid mental effort (Kool et al., 2010; Westbrook et al., 2013), so we must assume that it does entail a cost, which still needs to be specified at the biological and/or functional level. Action costs can be taken as measures of motivation intensity. This is because the action is only engaged if the net value is positive. Therefore, the highest cost that a person is willing to accept, in order to reach a particular goal, is equal to the goal value. In principle, motivation could be assessed by measuring the time that the person would be willing to invest to reach the goal. Yet in practice, researchers have focused on effort when developing behavioural paradigms to assess motivation. Behavioural tests By definition, it is impossible to vary intrinsic motivation and assess the behavioural consequences without changing the task to be performed, which may introduce a problematic confound. It is much simpler to manipulate motivation extrinsically, by varying the outcome associated with a given task or action, which can require more or less effort. The possibility that effortful actions trigger intrinsic motivation might be an issue in theory, but in practice empirical studies have shown that animals and persons tend to follow the law of least effort (Hull, 1943; Zipf, 1949). Experimental paradigms have been originally developed to test animals, mostly rodents, and were later adapted to humans. Two categories may be distinguished, depending on whether a choice is explicitly implemented or not. Interestingly, the processes targeted by the two sorts of tasks have been named differently, with an emphasis either on effort or reward (e.g. ‘effort-based decision’ and ‘incentive motivation’), although paradigms invariably involve the manipulation of both reward and effort levels (Table 2). Table 2 A synthetic view of motivational tests used in humans Type of …  Main options  … task  Selection within continuous range (Schmidt et al., 2008) Binary choice (Treadway et al., 2012) Willingness to accept (Chong et al., 2015)    … effort  Physical effort:  power grip peak (Pessiglione et al., 2007) or duration (Meyniel et al., 2013) number of button presses (Treadway et al., 2009) Mental effort:  N-back (Kool et al., 2010) numerical Stroop (Schmidt et al., 2012)    … reward  Real reward such as money (Le Bouc and Pessiglione, 2013) Virtual reward such as apples (Bonnelle et al., 2015) Subliminal reward (real or virtual money) (Schmidt et al., 2010)    … model  Hyperbolic or exponential discounting (Prevost et al., 2010) Parabolic (Hartmann et al., 2013) or sigmoidal discounting (Klein-Flugge et al., 2015) Subtraction of supralinear cost function (Le Bouc et al., 2016)  Type of …  Main options  … task  Selection within continuous range (Schmidt et al., 2008) Binary choice (Treadway et al., 2012) Willingness to accept (Chong et al., 2015)    … effort  Physical effort:  power grip peak (Pessiglione et al., 2007) or duration (Meyniel et al., 2013) number of button presses (Treadway et al., 2009) Mental effort:  N-back (Kool et al., 2010) numerical Stroop (Schmidt et al., 2012)    … reward  Real reward such as money (Le Bouc and Pessiglione, 2013) Virtual reward such as apples (Bonnelle et al., 2015) Subliminal reward (real or virtual money) (Schmidt et al., 2010)    … model  Hyperbolic or exponential discounting (Prevost et al., 2010) Parabolic (Hartmann et al., 2013) or sigmoidal discounting (Klein-Flugge et al., 2015) Subtraction of supralinear cost function (Le Bouc et al., 2016)  The table lists the main options (for the type of task, effort, reward, and model) that have been implemented in behavioural paradigms designed to assess motivation in human participants. References suggest one paper in which the corresponding option has been implemented. The advantages and drawbacks of the different options are explained in the text. Note that the list is non-exhaustive and that references are somewhat arbitrary. Binary choice In effort-based decision tasks, subjects have a choice between two options: producing little effort for small reward, or producing a higher effort for a bigger reward. This has been operationalized in rodents, for instance with a T-maze (Fig. 2) where a big reward (more food pellets) is placed in one arm behind a barrier, whereas the barrier-free arm only leads to a small reward (Salamone et al., 2007). Another example would be an operant box with two levers, one requiring more presses but providing more food pellets than the other (Walton et al., 2006). The number of food pellets (benefit) and the height of the barrier or the number of lever presses (cost) can be varied so as to determine the cost that the animal is willing to accept in order to get one food pellet, which gives a measure for the subjective reward value. Figure 2 View largeDownload slide Behavioural tasks assessing motivation as a reward/effort trade-off. Motivation intensity can be quantified as the amount of effort that the agent is willing to expend in order to obtain a potential reward. This trade-off between effort and reward has been operationalized in binary choice tasks (left) or free operant tasks (right). In animals (top), binary choice is typically implemented in a T-maze, with one branch representing the small reward/low effort option and the other branch the bigger reward (more food)/higher effort (due to the barrier) option. Free operant behaviour is classically assessed in a Skinner box where pressing the lever triggers food delivery. When the ratio between the number of lever presses and the number of food pellets is fixed, the reward obtained is simply proportional to the effort produced, such that the reward/effort levels are freely selected within a continuum. Note that the same sort of apparatus can be used to implement binary choice, with two levers associated with different ratios between effort and reward levels (not shown). Thus, binary choice can be envisaged as a special case of free operant behaviour, for which a large range of options (and not just two) are available. This can also be seen in the behavioural tests adapted to human participants (bottom), where effort is produced on a power grip and reward is given in monetary units. The figure shows one key screenshot of the visual animation presented to participants in a task trial. The vertical orange bar provides a visual feedback on the force produced, within a grid that is scaled to the participant’s maximal force. In the free operant version (sometimes called incentive motivation task), all grip forces between zero and maximal force may be selected, for a proportional reward that ranges from zero to full incentive (€1 in the illustrated example). In the binary choice version (sometimes called effort discounting task), only two reward-effort combinations are available (force levels are indicated by orange horizontal targets and reward levels by coin images). This behavioural paradigm can also be adapted to measure directly the willingness-to-accept a given effort for a given reward. In this case (not shown), a single option is displayed on the screen, as a target/coin association that the participant chooses to accept or decline. Figure 2 View largeDownload slide Behavioural tasks assessing motivation as a reward/effort trade-off. Motivation intensity can be quantified as the amount of effort that the agent is willing to expend in order to obtain a potential reward. This trade-off between effort and reward has been operationalized in binary choice tasks (left) or free operant tasks (right). In animals (top), binary choice is typically implemented in a T-maze, with one branch representing the small reward/low effort option and the other branch the bigger reward (more food)/higher effort (due to the barrier) option. Free operant behaviour is classically assessed in a Skinner box where pressing the lever triggers food delivery. When the ratio between the number of lever presses and the number of food pellets is fixed, the reward obtained is simply proportional to the effort produced, such that the reward/effort levels are freely selected within a continuum. Note that the same sort of apparatus can be used to implement binary choice, with two levers associated with different ratios between effort and reward levels (not shown). Thus, binary choice can be envisaged as a special case of free operant behaviour, for which a large range of options (and not just two) are available. This can also be seen in the behavioural tests adapted to human participants (bottom), where effort is produced on a power grip and reward is given in monetary units. The figure shows one key screenshot of the visual animation presented to participants in a task trial. The vertical orange bar provides a visual feedback on the force produced, within a grid that is scaled to the participant’s maximal force. In the free operant version (sometimes called incentive motivation task), all grip forces between zero and maximal force may be selected, for a proportional reward that ranges from zero to full incentive (€1 in the illustrated example). In the binary choice version (sometimes called effort discounting task), only two reward-effort combinations are available (force levels are indicated by orange horizontal targets and reward levels by coin images). This behavioural paradigm can also be adapted to measure directly the willingness-to-accept a given effort for a given reward. In this case (not shown), a single option is displayed on the screen, as a target/coin association that the participant chooses to accept or decline. Equivalent paradigms have been developed for human subjects, with effort being manipulated in terms of, for example, the force to be exerted on a handgrip (Fig. 2) or the number of clicks on a mouse (Treadway et al., 2009; Prevost et al., 2010). Compared to all the other measures, the grip task has the advantage of isolating effort from delay. This is because in standard designs, participants are instructed to produce short pulses, such that the effort cost (linked to peak force) can vary while keeping the duration roughly constant. This prevents situations in which enhancing effort postpones the reward, and thus increases temporal discounting of reward. Another potential confound is risk, if participants are uncertain about their ability to produce the required effort. This is generally avoided by restricting the range of forces to what participants can achieve with a 100% chance of success. The rewards used in human studies are generally more abstract than in animals: typically money or even points (tokens). They are generally thought to involve the same brain system as primary rewards, as implied by the notion of ‘common neural currency’ (Levy and Glimcher, 2012). Their advantage is 2-fold: first they are easy to quantify and second they prevent the issue of satiety, which may change the reward value during the course of the experiment. As famously argued by economists, marginal utility might be a decreasing function of monetary amount, meaning that a same objective gain would appears as less valuable (subjectively), once significant money has been accumulated. However, this effect likely remains limited given the small incentive ranges spanned in standard experiments. Willingness to accept In incentive motivation (or instrumental behaviour activation) tasks, there is no explicit choice in the sense that the alternative options are not explicitly laid down. For example, in paradigms using progressive ratio schedules, the number of lever presses required for a given number of food pellets is increased from one trial block to the next, until the animal ceases responding. As seen before, the number of lever presses at break point can be taken as a direct measure of the motivation induced by the food reward (i.e. the incentive). This sort of task can also be interpreted in the framework of decision theory, as there is an implicit decision to make, between pressing the lever or not. In other words, there is a hidden option that is just doing nothing. In fact this option is always present in animal experiments, which may stop performing the task at any time, for instance by running away or by breaking fixation, such that these ‘errors’ can be interpreted as motivated choices (Minamimoto et al., 2012). This sort of choices, with a single action that may be performed or not, can be made more explicit in humans, in the form of ‘accept’ versus ‘decline’ alternatives. Such tasks measure the willingness to accept performing a unique type of action for various combinations of effort and reward levels (Bonnelle et al., 2015). It could be argued that such yes/no decisions are more natural for the brain, because in the ancestral environment in which it has evolved (before the emergence of stores and restaurants), options do not appear simultaneously (Stephens and Krebs, 1986). When a new option arises, the agent is likely to be already pursuing a goal, such that the decision is whether to keep with the current course of action or to switch to the new alternative. Selection within a continuous range Another paradigm where choices are implicit are free operant tasks using fixed ratio schedules, meaning that the number of food pellets is proportional to the number of lever presses (Fig. 2). From a decision-theoretic perspective, animals are viewed as making decisions between pressing and not pressing at every time step, or choosing the time interval between presses, which may give a continuous set of numerous hidden options, depending on the resolution of time estimates and the maximal interval allowed (Niyogi et al., 2014). In a human version of this free operant paradigm (Fig. 2), participants are first shown a given incentive level (amount of money) and then asked to squeeze a handgrip, knowing that payoff will be proportional to peak force (Schmidt et al., 2008). More precisely, the payoff is calculated as the fraction of the incentive corresponding to the proportion of maximal force that subjects produce, such that they get half the incentive if they produce half their maximal force. Here again, the task (sometimes called incentive motivation task) can be viewed as a continuous version of the binary choice task (sometimes called effort discounting task), where the set of alternative options includes all possible forces between zero and maximal force. An interesting variant of this paradigm can be obtained by indexing the payoff not on peak force but on effort duration (Meyniel et al., 2013). In this case, a target force level is imposed and payoff accumulates, at a speed proportional to incentive level, as long as the force being produced by the participant is above the target. The duration of the trial (typically 30 s) is too long for the force to be maintained throughout the end, so subjects have to release the grip at some point, take a break to recover from fatigue and then start squeezing again to get more money. Effort production in this task can be interpreted as resulting from decisions about the durations of effort and rest periods, which in principle can take any value from zero to trial length. Specificities of testing humans Testing humans instead of animals present a number of advantages. One is that verbal instructions, associated with some familiarization with the task, may permit to investigate steady performance (i.e. with limited learning effects), without overtraining the subjects, which might make performance more like a habit. This is important because producing habitual behaviour has been shown to involve brain circuits different from those underpinning goal-directed behaviour (O'Doherty, 2016). A second advantage is the possibility to use virtual reward and effort. Indeed, consequential choices, where subjects have to perform the effort and truly receive the reward, can be reduced to a random fraction or even suppressed. This opens the possibility of presenting on every trial new items that resemble the sort of effort and reward we encounter in everyday life (an offer could be, for instance: would you clean up your room if I get you an ice cream?). Previous studies have shown that using virtual compared to real reward make little difference on effort production or intertemporal choice (Bickel et al., 2009; Schmidt et al., 2010). Yet it might not be neutral for decisions to propose effort items that are never actually implemented during the task. Some patients might have issues with anticipated costs, on which a priori decisions are based, and others with experienced costs, which would influence decisions posterior to effort production. Manipulating anticipated versus experienced costs has indeed been shown to result in distinct behavioural patterns (Meyniel et al., 2014). A third advantage is the accessibility of mental effort, which may be much more amenable to experimental investigation. Mental effort generally means attentional effort and has been manipulated by varying the difficulty of executive tasks such as Stroop, N-back, or task-switching. Just as physical effort, mental effort can be implemented in decision-making or incentive motivation tasks, i.e. with explicit or implicit choices. In decision tasks, it has been well-established that healthy subjects prefer to avoid difficult executive tasks (e.g. 3-back compared to 1-back), and only accept to perform them if the expected reward is significantly bigger than the one associated with an easy version of the same task (Kool et al., 2010; Westbrook et al., 2013; Apps et al., 2015). However, incentive effects on performance in tasks involving attentional effort appeared to be weaker than those observed with physical effort (Schmidt et al., 2012). This might mean that mental effort is more difficult to adjust, or less costly compared to physical effort. Finally, testing humans enables the assessment of possible dissociations between conscious and subconscious effects of incentive levels. This can be done by masking the cues indicating incentive levels to subjects, using standard subliminal presentation procedures. It has been demonstrated that subjects produce more effort in the grip task for higher incentives, even if they cannot report which incentive was presented (Pessiglione et al., 2007). This effect was nonetheless tiny compared to that observed in supraliminal conditions, where strategic adaptations (saving resources for when they matter) are likely to have an impact. In both cases (subliminal and supraliminal), the incentive motivation effect was associated with an increase in skin conductance response, a measure of autonomic activity. It is not entirely clear whether subliminal motivation effects correspond to a (subconscious) instrumental adjustment of behaviour, or to a more basic appetitive reflex, or even to emotional arousal. Indeed, when presenting emotional pictures incidentally and orthogonally to incentive levels (Schmidt et al., 2009), emotional arousal was found to increase effort production in a manner that was independent from motivational effects (no interaction). Note however that, although appetitive and emotional reactions to incentives may affect effort production and related decisions in many paradigms, their effect size seems limited in comparison to instrumental effects. Computational modelling Even in simple behavioural readouts of motivation, several factors might play a role. For instance, a shift in the propensity to favour high reward–high effort options could be due to an increased sensitivity to reward or a decreased sensitivity to effort. Computational models may be helpful for disentangling between such alternative explanations of behavioural changes. Modelling the cost/benefit trade-off Computational models are algorithms that perform the task imposed to subjects, meaning that they generate behavioural outputs through a limited series of mathematical operations applied to the experimental factors. Decision theory provides a generic and normative framework for deriving the computational models that can be used to simulate the behaviour in specific motivation tests. If in these experimental tests, goals can be reduced to the rewards that actions provide, and if actions can be reduced to the amount of effort that they involve, then Equation (1) can be simplified to:   V(Ei)=R(Ei)−C(Ei) (2) with V(Ei) being the net value of producing effort Ei, R(Ei) the reward associated to effort Ei and C(Ei) the cost of producing effort Ei. The contingencies between reward and effort are directly specified by the task, for example €10 can be associated with reaching 80% of maximal grip force. With money, reward level is an objective amount (such as €10) that can be directly entered in the equation, if we ignore distortions such as decreasing marginal utility. The main difficulty in establishing such models is to specify the cost of the required effort (80% of maximal force in the example). Pioneering investigations of how rewards are subjectively devalued by effort were inspired by delay discounting models, which account for how rewards are subjectively devalued by delay of delivery (Green and Myerson, 2004; Prevost et al., 2010). Non-linear functions such as exponential or hyperbolic decay with time are the mathematical forms most commonly used for delay discounting. Note that these functions can only give positive values. This is problematic in the case of effort because it would mean that agents always prefer producing the effort (even climbing a mountain for a peanut) than doing nothing. As this would be absurd, we focused in this review on a subtractive form of the value function, following on recent formalizations (Manohar et al., 2015; Le Bouc et al., 2016). This is not anecdotal, since it opens the possibility of attaching negative values to the production of efforts that are not rewarded enough. Discounting can nevertheless be non-linear with this subtractive form, depending on the shape of the cost function. Cost functions have been a matter of debate in recent years. In the case of physical effort, a consensus seems to emerge around the notion that discounting should be concave, as in parabolic functions (Hartmann et al., 2013), or at least initially concave, as in sigmoidal functions (Klein-Flugge et al., 2015). Interestingly, classical formalization in motor control theory, where action cost is typically defined as the integral of squared motor command, also predicts that cost should be a supralinear function of the objective muscle contraction (Rigoux and Guigon, 2012). These studies therefore converge to the conclusion that physical effort cost increases faster than a motor output such as grip force. The cost function might be different in the case of mental effort (Westbrook et al., 2013; Bonnelle et al., 2015), for which the essence of effort cost remains controversial. As an illustration, we describe below a net value function that could be integrated in a computational model to account for the behaviour in the incentive motivation test, where the payoff is proportional to the reward at stake and to the force produced:   V(Fi)=(1+kr.R).Fi−kc.(1+kf.T).Fi/(1−Fi) (3) with Fi being the considered peak force (in proportion to maximal force), R the potential reward (incentive level) and T the trial index. Note that Fi is the controlled variable that the agent must set (in order to maximize the net value), while R and T are experimental factors imposed by the task design. Simulations of costs and benefits generated by this value function, and the associated behavioural patterns, are shown in Fig. 3. Figure 3 View largeDownload slide Simulation of a motivation model. Simulation results were obtained with a model that can be approximated by Equation (3). It was applied to the incentive motivation task that is illustrated in Fig. 2 (bottom right) and used in a recent publication (Le Bouc et al., 2016), with six reward levels randomly distributed over 60 trials. (A) Simulated hidden variables. In a given trial, the model anticipates for each possible force peak (a proxy for effort level), the associated benefit (left) and the associated cost (right). The expected benefit is proportional to the force peak, with a slope that depends on the reward level offered in the present trial, i.e. the payoff corresponding to maximal force production (only four levels are illustrated). The expected cost is a supralinear function of force peak, with a slope that depends on fatigue level, i.e. the number of trials completed so far (two fatigue levels are illustrated, with the red line and red dots showing the expected cost at the beginning and at the end of the task, respectively). The net value is obtained by subtracting costs from benefits. The predicted behaviour is the optimal force peak (for which the net value is maximal), as illustrated by the green arrows. Note that effort expenditure (predicted force peak) increases with reward level, reproducing the incentive motivation effect. (B) Simulated behavioural patterns. Top and bottom panels show how the predicted behaviour varies with the two task factors: reward level and trial index. The different columns show how the predicted behaviour varies when changing one single free parameter (grey versus black lines). While changing the sensitivity to effort cost (Kc) globally shifts effort production up or down, changing the sensitivity to reward (Kr) affects the impact of reward level, and changing the sensitivity to fatigue (Kf) affects the impact of trial index. Figure 3 View largeDownload slide Simulation of a motivation model. Simulation results were obtained with a model that can be approximated by Equation (3). It was applied to the incentive motivation task that is illustrated in Fig. 2 (bottom right) and used in a recent publication (Le Bouc et al., 2016), with six reward levels randomly distributed over 60 trials. (A) Simulated hidden variables. In a given trial, the model anticipates for each possible force peak (a proxy for effort level), the associated benefit (left) and the associated cost (right). The expected benefit is proportional to the force peak, with a slope that depends on the reward level offered in the present trial, i.e. the payoff corresponding to maximal force production (only four levels are illustrated). The expected cost is a supralinear function of force peak, with a slope that depends on fatigue level, i.e. the number of trials completed so far (two fatigue levels are illustrated, with the red line and red dots showing the expected cost at the beginning and at the end of the task, respectively). The net value is obtained by subtracting costs from benefits. The predicted behaviour is the optimal force peak (for which the net value is maximal), as illustrated by the green arrows. Note that effort expenditure (predicted force peak) increases with reward level, reproducing the incentive motivation effect. (B) Simulated behavioural patterns. Top and bottom panels show how the predicted behaviour varies with the two task factors: reward level and trial index. The different columns show how the predicted behaviour varies when changing one single free parameter (grey versus black lines). While changing the sensitivity to effort cost (Kc) globally shifts effort production up or down, changing the sensitivity to reward (Kr) affects the impact of reward level, and changing the sensitivity to fatigue (Kf) affects the impact of trial index. The benefit term, (1+kr.R).Fi, accounts for the contingency imposed by experimental design, according to which exerting more force linearly increases the monetary payoff. Such contingency might be present in many real-life situations where exerting more effort would enhance the probability or reduce the delay of reward delivery. Other contingencies could be envisaged, for instance an all-or-none payoff depending on whether a target such as a force window is hit or missed. In this case, the benefit term could be formalized as krR.P(Fi), with P(Fi) being the probability of hitting the target given the considered peak force and some motor noise. Similar formulations can be applied to situations where the outcome is not a potential reward but a potential punishment such as monetary loss. The benefit term also includes a constant, which may account for the fact that producing more force has positive outcomes other than money, for instance it could make an impression on the experimenter. This is consistent with the repeated observation that subjects squeeze the grip in this task even for negligible monetary incentives (Schmidt et al., 2008; Le Bouc et al., 2016). The cost term includes an explosive cost function, kc.Fi/(1-Fi), modulated by a fatigue function, (1+kf.T). The cost function is a simple approximation of the equation derived from motor control theory (Rigoux and Guigon, 2012; Lebouc et al., 2016). Note that if F is expressed as a proportion of maximal force, then cost is null when no force is produced, and infinite when force approaches the theoretical maximum. This maximum may correspond to what can be expected at best from the musculature of the arm. In the case of handgrip, previous studies (Forbes et al., 1988; Hsu et al., 1993; Neu et al., 2002) have shown that maximal force can be approximated from simple measures of the forearm length, circumference and skin width (a proxy for non-muscular tissues). Fatigue is modelled as a linear increase with the number of trials achieved so far, for the sake of simplicity. This fatigue function captures the phenomenon that a same force is more and more costly to produce as fatigue progressively kicks in. More complex functions of trial index could be envisaged, and cumulative effort over trials could be a better proxy than mere trial index. We also note that a symmetrical function could be included in the benefit term to account for the phenomenon of satiety, according to which incentive effects would diminish with trial index or cumulative reward. We do not imply that the simple formulation adopted in Equation (3) is the unique possibility or that it provides the best account of motivational processes. We chose this illustration because it respects the fundamental principles of decision theory and because it is sufficient to understand the model fitting approach and hence the interests of computational phenotyping. Fitting model free parameters The net value function suggested in Equation (3) follows on a normative principle, in the sense that it specifies what subjects should do in order to maximize benefits and minimize costs. However, it does not discard the possibility that individuals may vary in their subjective attitude towards various dimensions such as reward, effort, fatigue etc. This subjectivity is enabled by the constants (the k parameters), which account for variations in how agents weight the various factors that impact cost and benefit terms. Constants in computational models are viewed as free parameters because they can be adjusted to fit (as closely as possible) the behaviour exhibited by a given subject in a given test. Value functions can be fitted directly to behavioural measures in choice paradigms that allow inferring indifference points, i.e. reward–effort combinations that are selected in 50% of trials when confronted two-by-two. The fitting procedure in this case consists of searching the set of free parameters that yield a theoretical value function whose distance from indifference points is minimal, as classically implemented in least squares methods. Indifference points can be determined through an adaptive design, such as a stair-case procedure, that increments effort and/or rewards levels based on observed choices (Westbrook et al., 2013; Klein-Flugge et al., 2015; Le Bouc et al., 2016). Another approach is to integrate in the model a function that generates the behaviour, in accordance to a value maximization principle. In the incentive motivation test, where any force can be chosen between zero and maximum, the predicted behaviour is simply the peak force that gives the maximal net value (for which the value derivative is null): F* = argmax V(F). If noise is incorporated in the model, then a probability (or likelihood) can be assigned to every observable peak force. Here, model fitting consists of searching for the set of free parameters that maximize the log likelihood of the peak force series produced across trials. This is implemented in a variety of optimization algorithms, from basic grid search methods to Bayesian model inversion techniques. Similar computational modelling and fitting approaches can be applied to binary choice, which can be seen as a special case of the incentive motivation problem, where the option set is reduced to just one pair. In this case, the two option values, which can be calculated through Equation (3), are generally entered into a softmax function, which gives the likelihood of the observed choice: P(Fc) = 1/(1+exp(β.(V(Fu)-V(Fc)))), with Fc and Fu being the chosen and unchosen forces, respectively and β a free parameter termed ‘inverse temperature’ that captures the noise in the choice process. The shape of the softmax function is a sigmoid that gives a probability of 0.5 when the two option values are equal, and converges to 1 or 0 when one option gets much better than the other. Fitting the parameters of the net value function thus reduces to (non-linear) logistic regression, which is classically employed to account for binary data. Eventually, adjusted parameters, or parameter estimates, obtained by fitting the model to the data, characterize the subject’s behaviour in a testing session. They provide a computational phenotype that quantifies individual attitudes such as sensitivity to reward attraction (kr), effort cost (kc), fatigue effect (kf), etc. These models are therefore not purely normative, in the sense that they do not determine what should be done once and for all: they instead take into account interindividual differences. This does not imply that they represent trait measures, as a same subject in different states might produce different behaviours, which would be best explained by different parameters. Changes in parameter estimates can therefore be used to evaluate the impact of a disease, or the effect of a treatment, or even normal fluctuations (in mood for instance). Disease and treatment effects can also be assessed in a subtly different way, through Bayesian model comparison, which provides a metric to elect the most plausible model given the behavioural data. The idea here is to provide qualitative, rather than quantitative, differences. For instance, one may want to compare models in which the disease affects sensitivity to reward versus sensitivity to effort, to discriminate between different forms of apathy (see application to Parkinson’s disease in Fig. 4). Figure 4 View largeDownload slide Computational characterization of motivation deficit. The figure illustrates the computational approach of motivation deficit, taking the example of dopamine depletion in Parkinson’s disease (adapted from Le Bouc et al., 2016). Behavioural data were acquired in a group of Parkinson’s disease patients tested ON and OFF dopaminergic medication, as well as in matched healthy controls, using the binary choice and free operant tasks displayed in Fig. 2. The model fitted to behavioural data can be approximated by Equation (3), and includes the same free parameters as in the simulations (Fig. 3). (A) Model-free results. Graphs show inter-subject means and standard errors for the force selected in the choice task, or freely generated in the motivation task, as a function of incentive level (from 0.1 to 5€). The slopes appeared to differ between groups, an effect that was captured by computational modelling. (B) Comparison of parameter estimates. Graphs show inter-subject means and standard errors for posterior estimates of free parameters, after model fitting through Bayesian inversion. Parameter estimates were then compared between groups using t-tests. Only parameter Kr differed significantly between Parkinson’s disease patients and controls, and between ON and OFF patients. (C) Clinico-computational correlation. The effects of dopaminergic medication on the computational parameter Kr and on the Starkstein apathy score were correlated across patients. This supports the idea that dopaminergic medication alleviates the motivation deficit by increasing reward sensitivity (as opposed to decreasing effort cost or susceptibility to fatigue). (D) Bayesian model selection. If we only consider the three parameters Kr, Kc and Kf, there are 23 = 8 possible models (each parameter can be affected or not by medication). Then for each parameter we compare the four models in which medication has an effect to the four models in which there is no effect. Exceedance probability suggests that a medication effect on Kr is highly plausible (compared to no effect), whereas an absence of effect was much more plausible for Kc and Kf. Thus, model selection leads to the same conclusion as comparing parameters estimates: the effect of dopaminergic medication is a selective modulation of Kr. PD = Parkinson’s disease. Figure 4 View largeDownload slide Computational characterization of motivation deficit. The figure illustrates the computational approach of motivation deficit, taking the example of dopamine depletion in Parkinson’s disease (adapted from Le Bouc et al., 2016). Behavioural data were acquired in a group of Parkinson’s disease patients tested ON and OFF dopaminergic medication, as well as in matched healthy controls, using the binary choice and free operant tasks displayed in Fig. 2. The model fitted to behavioural data can be approximated by Equation (3), and includes the same free parameters as in the simulations (Fig. 3). (A) Model-free results. Graphs show inter-subject means and standard errors for the force selected in the choice task, or freely generated in the motivation task, as a function of incentive level (from 0.1 to 5€). The slopes appeared to differ between groups, an effect that was captured by computational modelling. (B) Comparison of parameter estimates. Graphs show inter-subject means and standard errors for posterior estimates of free parameters, after model fitting through Bayesian inversion. Parameter estimates were then compared between groups using t-tests. Only parameter Kr differed significantly between Parkinson’s disease patients and controls, and between ON and OFF patients. (C) Clinico-computational correlation. The effects of dopaminergic medication on the computational parameter Kr and on the Starkstein apathy score were correlated across patients. This supports the idea that dopaminergic medication alleviates the motivation deficit by increasing reward sensitivity (as opposed to decreasing effort cost or susceptibility to fatigue). (D) Bayesian model selection. If we only consider the three parameters Kr, Kc and Kf, there are 23 = 8 possible models (each parameter can be affected or not by medication). Then for each parameter we compare the four models in which medication has an effect to the four models in which there is no effect. Exceedance probability suggests that a medication effect on Kr is highly plausible (compared to no effect), whereas an absence of effect was much more plausible for Kc and Kf. Thus, model selection leads to the same conclusion as comparing parameters estimates: the effect of dopaminergic medication is a selective modulation of Kr. PD = Parkinson’s disease. Although the sort of computational models exposed here provide reasonable accounts for optimization processes, they are silent about how values are generated. When the anticipated outcomes are expressed in objective metrics such as a number of pellets or a sum in euros, a slight subjective distortion might provide a good enough approximation. But when they come to episodes embedded in social contexts, such as family vacation projects, current models fall short of suggesting how values are constructed. Nevertheless, decision-theoretic models can be used to optimize action selection if proxies for subjective values are obtained from participants through likeability or desirability ratings on analogue scales. Note, however, that decision theory is agnostic about whether values have an affective component or not. The expected value (at the time of choice) and the experienced value (at the time of consumption) do not necessarily match what could be subjectively felt as desire (or dread) and pleasure (or pain). What is important for natural selection is that behavioural policies maximize reproductive success. Desire and pleasure might be just proxies for how good a given state is, i.e. how much it favours eventual reproduction. They might partially depart from the values that actually guide the behaviour, through the maximization mechanisms that the brain has evolved and which may be modelled by decision theory. Similar issues may arise when autonomic data, such as pupil diameter or skin conductance, are used to fit computational models, as was done in passive paradigms, i.e. in the absence of behavioural data (Petrovic et al., 2008). The question is to what extent autonomic responses represent valid proxies for the values that guide the behaviour. It has been shown that skin conductance responses were correlated with the magnitude or probability of both reward and punishment (LaBar et al., 1998; Critchley et al., 2001; Delgado et al., 2006; Schmidt et al., 2009; Gergelyfi et al., 2015). Thus, skin conductance could be used as a proxy for the absolute value of reward or punishment separately, but would be off for actions that combine positive and negative outcomes. Besides, it remains unknown whether skin conductance, which reflects arousal levels, strictly matches the value estimates that guide decisions. Along the same lines, pupil dilation is generally considered as a marker of effort, both physical and mental (Kahneman and Wright, 1971; Beatty and Wagoner, 1978; Piquado et al., 2010; Alnaes et al., 2014; Zenon et al., 2014). Thus, pupil size could be taken as a proxy for effort level, although it remains unclear how closely it would follow the effort cost estimate integrated in the net value equation that guides decisions. The alternative is that pupil size may reflect the amount of effort eventually invested in the action when it is performed, and not the amount of effort that is anticipated during decision-making. It should also be stressed that the dissociation between skin conductance and pupil size as reflecting outcome value and effort cost is not clear-cut, as both measures may reflect a common form of autonomic arousal. Finally, we have restricted our computational presentation to the basics of decision theory, leaving aside learning processes. It is nonetheless expected that learning should occur at many levels in the tests used to assess motivation, not only with respect to outcome value but also with regards to sensorimotor contingencies. Learning effects can be partially controlled through training sessions on the task, or captured by relevant learning models. A presentation of these models would, however, go far beyond the scope of this review. Neural implementation In this section, we briefly review studies that investigated motivation as a trade-off between cost and benefit—effort and reward, typically. We focus on work in primates that might have a direct translation to clinics, even if this work was largely inspired by earlier studies in rodents, for which we refer readers to a recent review (Salamone et al., 2016). We first explore activation studies that link brain activity to reward/effort optimization, in both humans and monkeys. We then review the behavioural consequences of lesions and pharmacological manipulations in animals. What can be learned from lesions and pharmacological treatments in human clinical conditions is discussed in the next section. Meta-analysis of functional MRI studies (Fig. 5) helps with delineating a set of brain regions involved in reward and effort processing. We deliberately set a conservative statistical threshold to focus on key nodes, so we do not pretend to provide an exhaustive description. The reward network includes the orbitofrontal cortex (mostly the medial part), the ventral striatum bilaterally and the midbrain (around dopaminergic nuclei). The effort network includes principally the anterior cingulate cortex and bilateral anterior insula. In the following we briefly consider the cortical, subcortical and neuromodulatory components of these networks. Figure 5 View largeDownload slide Meta-analysis of reward/effort neural representation. Statistical maps show with colour-coded z-score the results of functional MRI meta-analysis performed on the Neurosynth platform. As the number of functional MRI studies employing the keywords ‘reward’ and ‘effort’ was very different, we used reverse and forward inference, respectively. The slices were selected so as to display the most noticeable clusters, and superimposed on canonical anatomical template. The [x, y, z] coordinates refer to the Montreal Neurological Institute (MNI) space. aI = anterior insula; vS = ventral striatum. Note that less significant clusters were observed in the posterior and anterior cingulate cortex for reward, and in the inferior parietal and inferior frontal cortex for effort. Figure 5 View largeDownload slide Meta-analysis of reward/effort neural representation. Statistical maps show with colour-coded z-score the results of functional MRI meta-analysis performed on the Neurosynth platform. As the number of functional MRI studies employing the keywords ‘reward’ and ‘effort’ was very different, we used reverse and forward inference, respectively. The slices were selected so as to display the most noticeable clusters, and superimposed on canonical anatomical template. The [x, y, z] coordinates refer to the Montreal Neurological Institute (MNI) space. aI = anterior insula; vS = ventral striatum. Note that less significant clusters were observed in the posterior and anterior cingulate cortex for reward, and in the inferior parietal and inferior frontal cortex for effort. Cortical structures The medial part of the orbitofrontal cortex (or ventromedial prefrontal cortex, vmPFC) has been repeatedly shown to represent reward values in humans (Peters and Buchel, 2010; Bartra et al., 2013; Clithero and Rangel, 2014). More precisely, vmPFC haemodynamic response was positively correlated with the stimulus value, whether it was an objective value such as monetary amount, or a subjective value such as likeability rating. Accordingly, single-unit activity in the monkey vmPFC was strongly associated with stimulus value, integrating subjective aspects such as factors related to the internal state (satiety) or representations stored in memory (Bouret and Richmond, 2010; Strait et al., 2014; Abitbol et al., 2015). Single-unit recordings in monkeys have also established correlations (both positive and negative) with stimulus value in more lateral parts of the orbitofrontal cortex (lOFC) (Padoa-Schioppa and Cai, 2011; Wallis, 2011). In contrast, physical effort levels associated to actions were positively correlated with the dorsal anterior cingulate cortex (dACC) haemodynamic response, which also correlated negatively with the reward levels associated to action outcomes (Burke et al., 2013; Kurniawan et al., 2013; Skvortsova et al., 2014; Bonnelle et al., 2015; Scholl et al., 2015). These results are consistent with a role for the dACC in integrating costs and benefits and computing the net value of actions, whereas the role of the vmPFC might be confined to the outcome space, ignoring action costs. Physical effort levels were also found to be reflected in the anterior insula, together with outcomes bearing strong aversive valence such as pain or punishment (Seymour et al., 2005; Pessiglione et al., 2006; Samanez-Larkin et al., 2008; Prevost et al., 2010; Skvortsova et al., 2014). Interestingly, cognitive effort appeared to recruit the same brain areas, notably the dACC (Schouppe et al., 2014; Massar et al., 2015). The differential involvement of the ACC and vmPFC/lOFC in effort representations was also observed in primate studies. Indeed single cell activities were more sensitive to the amount of effort needed to obtain a reward in the ACC compared to the lOFC/vmPFC (Shidara and Richmond, 2002; Kennerley et al., 2009; San-Galli et al., 2016). Similar conclusions were reached from lesion studies in rats: lesions of the ACC decreased the willingness to climb a barrier to get the reward (Walton et al., 2003). Furthermore, a double dissociation was found between ACC and OFC lesions, which impaired effort-based and delay-based decisions, respectively (Rudebeck et al., 2006). The same dissociation has been observed in humans, with the delay of reward delivery modulating value representation in the vmPFC, and the effort associated to the action requested to obtain the reward modulating value representation in the dACC (Prevost et al., 2010). Subcortical structures In functional MRI studies using incentive motivation tasks, the ventral striatum and pallidum were found to activate with the amount of reward at stake and to drive motor performance (Knutson et al., 2001; Schmidt et al., 2012; Kroemer et al., 2014). This activation of the ventral striato-pallidum complex was observed in subliminal conditions, where subjects were unaware of incentive level (Pessiglione et al., 2007), and was dissociated from emotional arousal (Schmidt et al., 2009). It was also shown to drive both physical and mental effort, depending on the task demand (Schmidt et al., 2012). The ventral striatum and pallidum are part of the limbic circuit of the basal ganglia, which receives projections from the orbitofrontal cortex (Brown and Pluck, 2000; Haber, 2003). Nevertheless, their role in incentive motivation seems different from that of the orbitofrontal cortex, as they increase their activity when subjects produce more effort (to get more reward). Similar conclusions have been drawn from electrophysiological studies in monkeys (Tachibana and Hikosaka, 2012). However, when reward and effort levels have to be integrated in order to make a choice (and not to drive performance), activity in the striatum appeared to signal the net value, i.e. reward minus effort (Botvinick et al., 2009; Croxson et al., 2009; Kurniawan et al., 2010). Another line of research examined the role of dopamine in the nucleus accumbens, a structure homologous to the ventral striatum in rodents (see Salamone et al., 2007 for a review). Increasing and decreasing dopamine levels shifted the choices toward high effort–large reward and low effort–small reward options, respectively. This result could be explained either by dopamine enhancing reward attractiveness, or by dopamine diminishing effort cost. Voltammetry studies in rodents suggested that dopamine release in the nucleus accumbens scales with reward but not with effort level (Gan et al., 2010; Hollon et al., 2014; Hamid et al., 2016). This is in-line with single cell electrophysiological recordings in monkeys that showed a relatively limited sensitivity of dopamine neurons to information about upcoming physical effort, compared to expected reward (Pasquereau and Turner, 2013; Varazzani et al., 2015). This relative lack of sensitivity to effort by dopamine neurons contrasts with their strong sensitivity to other forms of cost that relate to reward delivery, such as risk or delay (Kobayashi and Schultz, 2008; Lak et al., 2014). These results suggest that dopamine helps to cope with effortful actions by signalling reward availability, without contributing to the estimation of effort cost. This is compatible with the energizing effects of dopamine enhancers that have been observed in humans (Wardle et al., 2011) (see also next section), or the reduced motivation following acute phenylalanine/tyrosine depletion, which reduces dopamine level (Venugopalan et al., 2011). If not dopamine, other neuromodulators might be involved in signalling effort cost, such as noradrenaline and serotonin. Electrophysiological recordings showed that when monkeys initiate an action, the firing of neurons in the locus coeruleus scales positively with the amount of force required (Bouret and Richmond, 2015; Varazzani et al., 2015). This is compatible with the idea that noradrenaline release signals the amount of effort required by upcoming actions. However, locus coeruleus neurons displayed virtually no response to information about effort provided by a visual cue, in contrast to what has been reported for ACC neurons in similar tasks (Kennerley et al., 2009). One interpretation is that noradrenaline is important at the time of action, when the effort needs to be produced, but not in the earlier decision to make the effort or not. Evidence for a role of serotonin in the processing of effort cost comes from pharmacological manipulation in healthy humans. Increasing serotonin level with antidepressants (SSRIs) prolonged effort duration in an incentive motivation task, an effect that was computationally captured by a reduction of effort cost (Meyniel et al., 2016). This is in agreement with a more general role for serotonin in overcoming costs, as was previously shown for the delay subjects have to wait before obtaining the reward (Miyazaki et al., 2012; Fonseca et al., 2015). Finally, opioids might also play a role in moderating the impact of experienced or anticipated effort, which could be seen as a prolongation of their role in attenuating pain experience. A tentative neuro-computational scenario It is tempting to establish links between the net value equation that adjusts the reward/effort trade-off and the findings reviewed in this section. A speculative but reasonable summary might be that (i) some regions, such as the vmPFC, compute the benefit term; (ii) other regions, such as the dACC, integrate the effort cost and signal the net value; and (iii) neuromodulators adjust the weights of reward and effort in the net value estimation. In Fig. 6, we go further and suggest a possible implementation of the computational mechanism that finds the optimal behaviour by maximizing the net value equation. For illustration, we take the case of the incentive force task (Le Bouc et al., 2016), a subcase of free operant paradigms (Fig. 2), to which Equation (3) can be applied. The problem for the brain is to find the force F* that maximizes the net value function V(F), without explicitly computing the value of all admissible forces. One solution is adjusting force dynamically so as to operate a gradient ascent on the value function, as follows:   ∂F∂t=λ∂V∂F (4) where the temporal derivative of force is obtained by multiplying the derivative of value with respect to force by a parameter λ that controls the rate of convergence of the gradient ascent. By construction, the steady-state limt→∞F(t) of this ordinary differential equation is a local maximum F* of the value function V(F). Computationally speaking, Equation (4) requires the moment-to-moment calculation of the gradient ∂V/∂F=kr∂B/∂F−kc∂C/∂F of the value function V with respect to force F. This gradient is decomposed into two terms: the gradient of benefits and the gradient of costs with respect to force. The former measures the efficiency of effort investment, in terms of the net benefit accrued by a unitary change in force. The latter measures the susceptibility of costs, in terms of the net energy expenditure incurred when force is increased by a unitary amount. In turn, this scheme highlights the computational nodes that are critical for performing effort allocation in the brain. In brief, effort allocation requires brain systems specialized for (i) evaluating the benefit and cost gradients; (ii) aggregating those to form the net gradient; and (iii) transforming the latter into the appropriate dynamical adjustment of instantaneous force production. In addition, the scheme may involve a feedback loop of proprioceptive and/or afferent copies of sensorimotor signals that enable the evaluation of the cost gradient. Note that in this particular design, the benefit gradient is constant and equal to the potential reward R (i.e. incentive level). Figure 6 View largeDownload slide A speculative implementation of motivation process in the human brain. The figure illustrates the mechanisms that the brain might implement to generate behaviour in a typical incentive motivation task, where one behavioural output must be selected within a continuous range. Under decision theory, the behavioural output must maximize the benefit B and minimize the cost C. In this example (see incentive force task in Le Bouc et al., 2016), the behavioural output is a force F, the benefit is a fraction of the reward R corresponding to F (relative to maximal force Fmax), and the cost is a supralinear function of F. The problem for the brain is therefore to find the force F that maximizes the net value (V = B – C) (see explanation in the text). (A) Plausible anatomical locations for the representation of computational variables involved in the model. The pivotal region would be the dACC, which would integrate the goal value conveyed by ventral fronto-striatal circuits and the effort cost transmitted by the anterior insula. The dACC would then send the net value to premotor and motor regions, which would elaborate a motor command for the muscles. Muscle contraction would on the one hand have an added value, and on the other entail an effort cost, which would be integrated in the corresponding brain regions. Within a particular trial, the behaviour would be generated through several loops during which all representations are updated. Note that the loop can be internally simulated (with no overt movement) if the motor regions inform directly (by an efferent copy of the motor command) the perceptual regions, from which effort cost can be estimated. There is no consensus about the role of neuromodulators in this machinery—here we illustrate a tentative functional repartition where dopamine modulates the goal value, 5HT the effort cost and noradrenaline the net value. (B) Mechanistic derivation of optimal behaviour in a putative brain-scale network. The key mechanism is a gradient ascent: the temporal derivative of force (generated in motor regions) would scale with the derivative of net value with respect to force (aggregated in the dACC by subtracting cost derivative from benefit derivative). In the incentive motivation task, the derivative of the benefit is proportional to reward R (i.e. incentive level). The effective connectivity in the network would correspond to the weighting of the different factors influencing the net value, in particular reward (kr) and effort (kc). The dynamics at longer time scales (across trials), which could give rise to fatigue or satiety effects, is not represented. 5HT = serotonin; B = expected benefit; C = expected cost; DA = dopamine; dACC = dorsal anterior cingulate cortex; F = produced force; Fp = perceived force; M1 = primary motor cortex; NA = noradrenaline; PM = premotor cortex; R = reward level; SI-II = primary and secondary somatosensory cortex; V = net value; vmPFC = ventromedial prefrontal cortex; VS = ventral striatum. Figure 6 View largeDownload slide A speculative implementation of motivation process in the human brain. The figure illustrates the mechanisms that the brain might implement to generate behaviour in a typical incentive motivation task, where one behavioural output must be selected within a continuous range. Under decision theory, the behavioural output must maximize the benefit B and minimize the cost C. In this example (see incentive force task in Le Bouc et al., 2016), the behavioural output is a force F, the benefit is a fraction of the reward R corresponding to F (relative to maximal force Fmax), and the cost is a supralinear function of F. The problem for the brain is therefore to find the force F that maximizes the net value (V = B – C) (see explanation in the text). (A) Plausible anatomical locations for the representation of computational variables involved in the model. The pivotal region would be the dACC, which would integrate the goal value conveyed by ventral fronto-striatal circuits and the effort cost transmitted by the anterior insula. The dACC would then send the net value to premotor and motor regions, which would elaborate a motor command for the muscles. Muscle contraction would on the one hand have an added value, and on the other entail an effort cost, which would be integrated in the corresponding brain regions. Within a particular trial, the behaviour would be generated through several loops during which all representations are updated. Note that the loop can be internally simulated (with no overt movement) if the motor regions inform directly (by an efferent copy of the motor command) the perceptual regions, from which effort cost can be estimated. There is no consensus about the role of neuromodulators in this machinery—here we illustrate a tentative functional repartition where dopamine modulates the goal value, 5HT the effort cost and noradrenaline the net value. (B) Mechanistic derivation of optimal behaviour in a putative brain-scale network. The key mechanism is a gradient ascent: the temporal derivative of force (generated in motor regions) would scale with the derivative of net value with respect to force (aggregated in the dACC by subtracting cost derivative from benefit derivative). In the incentive motivation task, the derivative of the benefit is proportional to reward R (i.e. incentive level). The effective connectivity in the network would correspond to the weighting of the different factors influencing the net value, in particular reward (kr) and effort (kc). The dynamics at longer time scales (across trials), which could give rise to fatigue or satiety effects, is not represented. 5HT = serotonin; B = expected benefit; C = expected cost; DA = dopamine; dACC = dorsal anterior cingulate cortex; F = produced force; Fp = perceived force; M1 = primary motor cortex; NA = noradrenaline; PM = premotor cortex; R = reward level; SI-II = primary and secondary somatosensory cortex; V = net value; vmPFC = ventromedial prefrontal cortex; VS = ventral striatum. These theoretical brain systems can be mapped onto possible anatomical locations based on the neuroscientific literature on the reward/effort trade-off (Fig. 6). This neural implementation is of course highly speculative but may serve to derive predictions about the sort of behavioural deficits that should be induced by specific brain damage. Lastly, this computational perspective suggests that the coupling strength between the aggregation system and the benefit and cost systems corresponds to the control parameters kr and kc. In other terms, induced changes in effective connectivity between these systems should induce concomitant variations in the way individuals trade costs against benefits. This mechanistic interpretation is interesting because it ties together the known biological impact of neuromodulators onto synaptic plasticity and the behavioural effect of related pharmacological drugs onto effort allocation. Clinical manifestation The different neural structures involved in the reward/effort trade-off can be dysfunctional in a number of pathological conditions, and/or targeted by therapeutic interventions. Some specific motivational dysfunctions have been identified, in both neurology and psychiatry, and distinguished from motor or cognitive disabilities that might also affect the behaviour. We review this work, with a special emphasis on studies that rely upon computational modelling. Neurology Motivation disorders have been observed following focal lesions (stroke or tumour) and degenerative diseases. The most frequent disorder is apathy, which is generally defined as a reduction of goal-directed behaviour, and present in 50% of cases after traumatic brain injury (Arnould et al., 2013), 40% in Parkinson’s disease (Starkstein et al., 1992), 50% in pre-dementia Alzheimer’s disease (Landes et al., 2005), and 35% in patients after vascular lesions (Caeiro et al., 2013; van Dalen et al., 2013). Apathy alters functional recovery during rehabilitation after vascular lesions, and represents an independent predictor of poor functional outcome (Hama et al., 2007; Mayo et al., 2009; Caeiro et al., 2013), as well as everyday life dependency (van Almenkerk et al., 2015). Apathy has a negative impact on quality of life and can also lead to significant burden for caregivers (Aarsland et al., 1999; Samus et al., 2005). The condition in which reward and effort processing have been the most investigated is Parkinson’s disease, which is characterized by a progressive degeneration of dopaminergic neurons, prominent motor symptoms such as akinesia, and frequent non-motor symptoms such as apathy. In Parkinson’s disease, apathy might result from dopamine depletion in the mesocorticolimbic pathway, which connects the ventral tegmental area to ventral striatum, orbitofrontal and anterior cingulate cortex (Remy et al., 2005; Thobois et al., 2010; Brown et al., 2012; Martinez-Horta et al., 2014). Comparing Parkinson’s disease patients with and without dopaminergic medication has brought some insight into the role of dopamine in the reward/effort trade-off. Dopaminergic medication has been shown to increase the amount of effort that Parkinson’s disease patients were willing to produce for a given reward in a progressive ratio schedule task that involved clicking on a keyboard to earn money (Porat et al., 2014), and in an accept versus decline choice task that involved squeezing a handgrip to earn fictive rewards (Chong et al., 2016). Dopaminergic medication has also been shown to bias choices toward high effort–big reward option in a binary choice task that traded handgrip force against monetary payoff, and to enhance effort production in a free operant (or incentive motivation) task that involved squeezing a handgrip to win monetary rewards, knowing that payoff would be proportional to force at peak (Le Bouc et al., 2016). Thus, dopaminergic enhancers have yielded consistent effects on effort expenditure for rewards across a variety of experimental paradigms, in line with what was observed in rodents (Salamone et al., 2007). Similar effects have also been obtained with high-frequency stimulation of the subthalamic nucleus in Parkinson’s disease patients (Palminteri et al., 2013; Zenon et al., 2016b). Computational modelling showed that the energizing effect of dopamine enhancers was best captured by an increased sensitivity to reward magnitude, and not a decreased sensitivity to effort cost (Le Bouc et al., 2016). This computational effect of dopaminergic medication could account for both the enhanced propensity to select high effort–big reward option in the choice task and the enhanced force production in the motivation task. Thus, the same computational mechanism could explain the orientation (action selection) and the intensity (action vigour) of the behaviour. Dopaminergic medication also had an effect on the motor activation rate, a computational parameter used in optimal control theory that serves to control the speed with which force is produced, irrespective of reward level. Crucially, this motor effect was independent (uncorrelated across patients) from the motivational effect on reward magnitude. Moreover, medication effects on motor and motivational parameters were correlated with improvement of motor symptoms and apathy, respectively. Such dissociation shows that computational analysis can help disentangle motor and motivational disorders in conditions where it is not obvious to tell whether the patient cannot or simply does not want to produce a behaviour. This result is compatible with other model-based studies of Parkinson’s disease patients’ behaviour, which also pointed to decreased reward sensitivity, irrespective of costs (Manohar et al., 2015; Zenon et al., 2016a). We note that dopamine receptor agonists may not only improve apathy but also induce impulse control disorders (ICDs) in a significant proportion of Parkinson’s disease patients (∼17% according to Voon et al., 2017). ICDs can include gambling disorder, binge eating, compulsive shopping, compulsive sexual behaviours and compulsive medication use (dopamine dysregulation syndrome). In decision-making tasks, ICDs have been associated with enhanced sensitivity to delay, resulting in more impulsive choices (Housden et al., 2010; Voon et al., 2010b; Joutsa et al., 2015), or with higher propensity to make risky choices (Djamshidian et al., 2010; Voon et al., 2011). However, ICDs could also be explained by dysfunction of learning mechanisms, such as an imbalance between positive and negative reinforcement (Voon et al., 2010a; Piray et al., 2014). As learning models go beyond our scope, we refer readers to a recent review of putative mechanisms underlying ICDs (Voon et al., 2017). Another case where apathy has been explored using reward/effort paradigms is the so-called autoactivation deficit. This syndrome is characterized by a severe form of apathy following bilateral lesions in the striato-pallidal complex (Laplane et al., 1989). Patients present a complete lack of self-initiated behaviour that contrasts with the preserved motor and cognitive abilities that they can exhibit when solicited by another person (Laplane and Dubois, 2001). These patients have been explored in the first study that assessed reward/effort trade-off in neurological conditions (Schmidt et al., 2008). In the incentive motivation task, patients were free to produce any handgrip force knowing that (fictive) payoff would be in proportion, whereas in the instructed control task, patients were explicitly told how much to squeeze the handgrip. Patients showed a preserved ability to modulate their force according to the instructions, a preserved autonomic (skin conductance) response to incentive cues (coin images), but an inability to modulate their force according to monetary incentives (Schmidt et al., 2008). These results suggested that striato-pallidal lesions do not impair motor control, nor do they impair the affective evaluation of monetary incentive, but rather the process that translates potential incentives into motor activation (Schmidt et al., 2008). Interestingly, patients still produced a force in the incentive motivation task, suggesting that only the financial part of the benefit term was affected in the net value function. Further computational analyses might help specify the motivational dysfunction that characterizes the autoactivation deficit. It has been shown that dopaminergic medication could help restore motivational function in these patients (Adam et al., 2013), which might indicate that apathy in autoactivation deficit is qualitatively similar, but quantitatively more severe, than apathy in Parkinson’s disease. Finally, apathy has been described following lesions of the medial wall, which can include the ACC and vmPFC. Unfortunately, few studies have investigated the reward/effort trade-off in these patients. It has been reported that ACC lesions attenuate the conscious feeling of mental effort (Naccache et al., 2005), without impairing cognitive control performance. This is consistent with the report that electrical stimulation of ACC induces the subjective impression of an imminent challenge and the determined intention to cope with it (Parvizi et al., 2013). Examination of a large cohort of patients showed that frontal medial damage was associated with a reduced effect of reward on invigorating saccade velocity (Manohar and Husain, 2016). Somewhat surprisingly, this study reported an increased sensitivity to reward following specific (subgenual) vmPFC lesions, and a trend in the opposite direction following ventral striatum lesions. Further studies using proper physical or mental effort would be needed to precise the effect of such lesions on the net value function. Psychiatry Disorders of motivation are frequently observed in psychiatric diseases. Even when they are not the most apparent symptoms, they may interfere with cognitive abilities and impede functional outcomes. Indeed, two of the most frequent and devastating psychiatric diseases, schizophrenia and depression, include lack of motivation in their definition. According to the DSM-5, one of the two main negative symptoms in schizophrenia is avolition—a decrease in motivated self-initiated purposeful activities (American Psychiatric Association, 2013). Similarly, a diagnosis of major depressive disorder requires either depressed mood or markedly diminished interest or pleasure in all, or almost all, activities most of the day, nearly every day. Conversely, one of the criteria of manic/hypomanic episodes is an increase in goal-directed activity (even if psychomotor agitation, i.e. purposeless non-goal-directed activity, is most often observed as severity increases). Beyond motivation deficit and excessive motivation, several psychiatric diseases are partly defined by atypical interest (e.g. autism spectrum disorders) or deviant motivation (e.g. addictive disorders). In this section, we will mainly focus on schizophrenia and depression, where reward/effort-based decision-making was particularly investigated in the recent years. From an historical point of view, the very first descriptions of schizophrenia by Kraepelin and Bleuler already emphasized the motivation deficit, which has been considered a cardinal and crucial component of negative symptoms (Kirkpatrick et al., 2006; Foussias and Remington, 2010). However, negative symptoms, which are by definition less spectacular and de facto less responsive to antipsychotics than delusions or hallucinations, were largely understudied until quite recently. Nevertheless, recent research put forward the critical importance of avolition, which has been shown to be one of the best predictors of functional outcome (Bowie et al., 2006; Foussias et al., 2011; Konstantakopoulos et al., 2011; Evensen et al., 2012; Fervaha et al., 2013a, 2014a, b), not only at the chronic stage of the disease but also during first episodes (Faerden et al., 2009, 2010; Chang et al., 2016) or even in ultra-high risk patients (Lam et al., 2015). Moreover, the lack of motivation could partly account for other dimensions of the disease, such as cognitive dysfunction, which are usually evaluated through neuropsychological tests that require participants to be motivated for a correct performance (Fervaha et al., 2014c). Regarding behavioural tests of motivation, many studies revealed a reward processing deficit using desirability/pleasantness ratings, intertemporal choices or reinforcement learning tasks (see Gold et al., 2008 for a review). Effort was introduced lately in a seminal study showing that patients with schizophrenia are less likely to favour high effort, especially when paired with largest and most certain monetary rewards (Gold et al., 2013). This shift in the reward/effort trade-off has been replicated by several studies (Fervaha et al., 2013c; Barch et al., 2014; Hartmann et al., 2015; Treadway et al., 2015; Wang et al., 2015; McCarthy et al., 2016; Strauss et al., 2016), across various types of tasks (binary choice, willingness to accept) and efforts (key pressing, handgrip squeezing), with only one negative result (Docx et al., 2015). It has also been extended to cognitive (rather than motor) effort (Wolf et al., 2014; Culbreth et al., 2016), again with one negative finding (Gold et al., 2015a). Furthermore, most studies found a correlation between the willingness to produce efforts and the severity of negative symptoms assessed with standard clinical scales (Gold et al., 2015b). Interestingly, a recent study compared five different reward/effort trade-off tasks in a large sample of 94 patients with schizophrenia, with a 4-week retest (Reddy et al., 2015). All tasks, with the notable exception of the perceptual categorization task, discriminated between patients and controls, with good enough reliability, but relatively modest correlation with negative symptoms (Horan et al., 2015). Surprisingly, no correlation was found with treatment, although most medications used in schizophrenia target dopaminergic transmission. Indeed, other authors found a strong effect of drug type (particularly conventional antipsychotics) on the willingness to expend effort, but argued that different drugs generally mean different patients, and claimed for randomized trials (Gold et al., 2015b). Critically, and despite the growing amount of evidence pointing toward an altered reward/effort trade-off in schizophrenia (Fervaha et al., 2013b; Strauss et al., 2014; Green and Horan, 2015; Green et al., 2015), no existing study attempted to discriminate between decreased sensitivity to reward and increased sensitivity to effort, using computational modelling tools. The only step in this direction was taken in a study reporting that patients with schizophrenia produced the same amount of effort as controls on average, but that effort expenditure was less driven by reward, such that choices were suboptimal in terms of maximizing outcomes (Treadway et al., 2015). More quantitative analyses would be required to disclose the potential distortions of the net value function in schizophrenia, as well as the impact of treatments such as antidopaminergic medication. As in schizophrenia, motivation deficit is a frequent and burdensome symptom of depression, which is highly predictive of functional impairment and subjective wellbeing (Calabrese et al., 2014; Fervaha et al., 2016). Moreover, as for avolition in schizophrenia, motivation deficit is less responsive to conventional treatment, such as serotoninergic antidepressant in unipolar depression or mood stabilizer in bipolar disorder, than other dimensions of the disease (Calabrese et al., 2014). Over the past decade, most behavioural investigations focused on reward processing versus punishment processing (Eshel and Roiser, 2010; Huys et al., 2013; Admon and Pizzagalli, 2015; Chen et al., 2015). Typical results in depressed patients are excessive processing of negative stimuli, hyposensitivity to positive outcomes with decreased pleasurable experience, and blunted response to reward in the ventral striatum. Effort was first introduced with an incentive motivation test using a handgrip device (Cléry-Melin et al., 2011), which established that contrary to healthy controls, depressed patients would not produce more effort when more money is at stake. A follow-up study showed that a normal pattern of effort production in response to monetary incentives was restored after remission from depressive episode (Mauras et al., 2016). The diminished sensitivity of effort production to incentives has been replicated many times, using different tasks (binary choice, willingness to accept) and efforts (key pressing, handgrip squeezing, cognitive effort) (Sherdell et al., 2012; Treadway et al., 2012; Yang et al., 2014; Hershenberg et al., 2016). While the shift in the reward/effort trade-off was correlated with Beck Depression Inventory (BDI) score in healthy subjects (Treadway et al., 2009), and was found more pronounced in actual depression than in subsyndromal depression, two studies failed to find a correlation with global BDI score in depressed patients (Yang et al., 2014; Hershenberg et al., 2016). Thus, abnormal effort allocation might represent a specific dimension of depression rather than a consequence of depression severity. Indeed, the willingness to produce effort for a given reward was correlated with lack of motivation, as measured by apathy scales, in both healthy participants and depressed patients (Bonnelle et al., 2015; Mauras et al., 2016). The link between anhedonia and reward/effort-based decision-making is less clear and still a matter of debate (Treadway and Zald, 2011; Der-Avakian and Markou, 2012; Rizvi et al., 2016). Regarding diagnostic, only one study compared unipolar and bipolar depressed patients, and found no difference between these two populations, nor between the medication classes used to treat them (Hershenberg et al., 2016). A decreased reward/effort ratio was also found in drug-naïve patients (Yang et al., 2014), which is an important control since serotoninergic medication could impact reward or effort processing. Again, while consistent evidence was found for an alteration of reward/effort-based decision-making in depression, no study could dissociate between the two candidate underlying mechanisms, namely increased sensitivity to effort or decreased sensitivity to reward. Thus, the question remains to what extent these apparently similar reward/effort trade-off alterations in schizophrenia and depression are indeed supported by the same cognitive distortion and neurobiological pathophysiology. Importantly, decreased reward/effort ratio is not an unspecific feature that would be observed in any psychiatric disease. Indeed this ratio was reported to be unaltered for instance in pathological gamblers (Fauth-Buhler et al., 2014), or even increased in autism spectrum disorder (Damiano et al., 2012). Unfortunately, reward/effort-based decision-making has not been explored in patients for whom an excessive motivation would be predicted from the clinical presentation, such as during hypomanic episodes of bipolar disorder. Conclusions and perspectives Compared to psychometric scoring that assesses motivation deficit as a whole, computational phenotyping based on behavioural tests is a promising avenue for clinical neuroscience. Modelling the behaviour as generated from a value function (borrowed from decision theory) enables a normative decomposition of motivation into subcomponents that may possess anatomo-functional specificity at the neural level. For instance, the benefit term has been associated with cortical regions such as the vmPFC, whereas other regions like the dACC would integrate the costs. Also, the weights of costs and benefits might be determined by different neuromodulators such as dopamine, noradrenaline and serotonin. Thus, computational phenotyping may help discriminate between distinct mechanisms underlying motivation deficits, such as hyposensitivity to benefits and hypersensitivity to costs, or abnormal susceptibility to fatigue. It may also help dissociate motivation deficits from motor disorders, which seems crucial in a number of neuro-psychiatric diseases. An overarching objective for computational neuropsychiatry would be to build a quantitative model of motivation that integrates the neural and algorithmic levels. From virtual lesions of this grand model, one could derive predictions about specific behavioural patterns that should be observed in patients. Reciprocally, by fitting the model to observed patients’ behaviour, one could make inferences about the underlying pathophysiology. The hope here is that computational approach would (i) provide unprecedented insight into the disease; and (ii) improve on differential diagnostic and therapeutic orientation (e.g. assuming that hyposensitivity to reward should not require the same treatment as hypersensitivity to effort). There are nonetheless limitations to this approach. First, the estimation of computational parameters may depend upon the paradigms used to probe the behaviour. Instead of faithfully representing the patient’s state, parameters might be susceptible to confounding details of the tasks. A related issue is that quantification of motivation deficit may depend upon the particular goals suggested in behavioural tests. Goal values are subjective and assigning a low value to one particular goal is not necessarily pathological, as long as other goals receive high values. This may be problematic because most tasks target a reduced set of interests, typically motivation for performance or payoff (the goal is to accumulate points or money). Yet patients may not be strongly motivated by that sort of goal but still driven by other goals, such as making their relatives happy. Note that decision theory is only useful to capture disorders characterized by abnormal motivation intensity, not those defined by abnormal motivation contents (e.g. atypical interests in autism spectrum disorder). Finally, because all behavioural tasks suggest goals (as in ‘would you make an effort to get this reward?’), they might miss motivation deficits in which the generation of goals (and not goal valuation) is precisely the problem. One could envisage the case of a patient who would like to engage in some activity but the idea of doing this activity would simply not come to mind. Other types of tests would certainly be required in order to detect this type of motivation deficit. Funding No funding was received towards this work. Abbreviations Abbreviations dACC dorsal anterior cingulate cortex vmPFC ventromedial prefrontal cortex References Aarsland D, Larsen JP, Lim NG, Janvin C, Karlsen K, Tandberg E, et al.   Range of neuropsychiatric disturbances in patients with Parkinson's disease. J Neurol Neurosurg Psychiatry  1999; 67: 492– 6. Google Scholar CrossRef Search ADS PubMed  Abitbol R, Lebreton M, Hollard G, Richmond BJ, Bouret S, Pessiglione M. Neural mechanisms underlying contextual dependency of subjective values: converging evidence from monkeys and humans. J Neurosci  2015; 35: 2308– 20. Google Scholar CrossRef Search ADS PubMed  Adam R, Leff A, Sinha N, Turner C, Bays P, Draganski B, et al.   Dopamine reverses reward insensitivity in apathy following globus pallidus lesions. Cortex  2013; 49: 1292– 303. Google Scholar CrossRef Search ADS PubMed  Admon R, Pizzagalli DA. Dysfunctional reward processing in depression. Curr Opin Psychol  2015; 4: 114– 18. Google Scholar CrossRef Search ADS PubMed  Alnaes D, Sneve MH, Espeseth T, Endestad T, van de Pavert SH, Laeng B. Pupil size signals mental effort deployed during multiple object tracking and predicts brain activity in the dorsal attention network and the locus coeruleus. J Vis  2014; 14: 1– 20. Google Scholar CrossRef Search ADS PubMed  American Psychiatric Association. Diagnostic and statistical manual of mental disorders . 5th edn. Arlington, VA: American Psychiatric Publishing; 2013. Apps MA, Grima LL, Manohar S, Husain M. The role of cognitive effort in subjective reward devaluation and risky decision-making. Sci Rep  2015; 5: 16880. Google Scholar CrossRef Search ADS PubMed  Arnould A, Rochat L, Azouvi P, Van der Linden M. A multidimensional approach to apathy after traumatic brain injury. Neuropsychol Rev  2013; 23: 210– 33. Google Scholar CrossRef Search ADS PubMed  Barch DM, Treadway MT, Schoen N. Effort, anhedonia, and function in schizophrenia: reduced effort allocation predicts amotivation and functional impairment. J Abnorm Psychol  2014; 123: 387– 97. Google Scholar CrossRef Search ADS PubMed  Bartra O, McGuire JT, Kable JW. The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. Neuroimage  2013; 76: 412– 27. Google Scholar CrossRef Search ADS PubMed  Beatty J, Wagoner BL. Pupillometric signs of brain activation vary with level of cognitive processing. Science  1978; 199: 1216– 18. Google Scholar CrossRef Search ADS PubMed  Bickel WK, Pitcock JA, Yi R, Angtuaco EJ. Congruence of BOLD response across intertemporal choice conditions: fictive and real money gains and losses. J Neurosci  2009; 29: 8839– 46. Google Scholar CrossRef Search ADS PubMed  Bonnelle V, Veromann KR, Burnett Heyes S, Lo Sterzo E, Manohar S, Husain M. Characterization of reward and effort mechanisms in apathy. J Physiol Paris  2015; 109: 16– 26. Google Scholar CrossRef Search ADS PubMed  Botvinick M, Braver T. Motivation and cognitive control: from behavior to neural mechanism. Annu Rev Psychol  2015; 66: 83– 113. Google Scholar CrossRef Search ADS PubMed  Botvinick MM, Huffstetler S, McGuire JT. Effort discounting in human nucleus accumbens. Cogn Affect Behav Neurosci  2009; 9: 16– 27. Google Scholar CrossRef Search ADS PubMed  Boureau YL, Sokol-Hessner P, Daw ND. Deciding how to decide: self-control and meta-decision making. Trends Cogn Sci  2015; 19: 700– 10. Google Scholar CrossRef Search ADS PubMed  Bouret S, Richmond BJ. Ventromedial and orbital prefrontal neurons differentially encode internally and externally driven motivational values in monkeys. J Neurosci  2010; 30: 8591– 601. Google Scholar CrossRef Search ADS PubMed  Bouret S, Richmond BJ. Sensitivity of locus ceruleus neurons to reward value for goal-directed actions. J Neurosci  2015; 35: 4005– 14. Google Scholar CrossRef Search ADS PubMed  Bowie CR, Reichenberg A, Patterson TL, Heaton RK, Harvey PD. Determinants of real-world functional performance in schizophrenia subjects: correlations with cognition, functional capacity, and symptoms. Am J Psychiatry  2006; 163: 418– 25. Google Scholar CrossRef Search ADS PubMed  Brown CA, Campbell MC, Karimi M, Tabbal SD, Loftin SK, Tian LL, et al.   Dopamine pathway loss in nucleus accumbens and ventral tegmental area predicts apathetic behavior in MPTP-lesioned monkeys. Exp Neurol  2012; 236: 190– 7. Google Scholar CrossRef Search ADS PubMed  Brown RG, Pluck G. Negative symptoms: the ‘pathology' of motivation and goal-directed behaviour. Trends Neurosci  2000; 23: 412– 17. Google Scholar CrossRef Search ADS PubMed  Burke CJ, Brunger C, Kahnt T, Park SQ, Tobler PN. Neural integration of risk and effort costs by the frontal pole: only upon request. J Neurosci  2013; 33: 1706– 13a. Google Scholar CrossRef Search ADS PubMed  Caeiro L, Ferro JM, Costa J. Apathy secondary to stroke: a systematic review and meta-analysis. Cerebrovasc Dis  2013; 35: 23– 39. Google Scholar CrossRef Search ADS PubMed  Calabrese JR, Fava M, Garibaldi G, Grunze H, Krystal AD, Laughren T, et al.   Methodological approaches and magnitude of the clinical unmet need associated with amotivation in mood disorders. J Affect Disord  2014; 168: 439– 51. Google Scholar CrossRef Search ADS PubMed  Chang WC, Hui CL, Chan SK, Lee EH, Chen EY. Impact of avolition and cognitive impairment on functional outcome in first-episode schizophrenia-spectrum disorder: a prospective one-year follow-up study. Schizophr Res  2016; 170: 318– 21. Google Scholar CrossRef Search ADS PubMed  Chen C, Takahashi T, Nakagawa S, Inoue T, Kusumi I. Reinforcement learning in depression: a review of computational research. Neurosci Biobehav Rev  2015; 55: 247– 67. Google Scholar CrossRef Search ADS PubMed  Chong TT, Bonnelle V, Husain M. Quantifying motivation with effort-based decision-making paradigms in health and disease. Prog Brain Res  2016; 229: 71– 100. Google Scholar CrossRef Search ADS PubMed  Chong TT, Bonnelle V, Manohar S, Veromann KR, Muhammed K, Tofaris GK, et al.   Dopamine enhances willingness to exert effort for reward in Parkinson's disease. Cortex  2015; 69: 40– 6. Google Scholar CrossRef Search ADS PubMed  Cléry-Melin ML, Schmidt L, Lafargue G, Baup N, Fossati P, Pessiglione M. Why don't you try harder? An investigation of effort production in major depression. PLoS One  2011; 6: e23178. Google Scholar CrossRef Search ADS PubMed  Clithero JA, Rangel A. Informatic parcellation of the network involved in the computation of subjective value. Soc Cogn Affect Neurosci  2014; 9: 1289– 302. Google Scholar CrossRef Search ADS PubMed  Critchley HD, Mathias CJ, Dolan RJ. Neural activity in the human brain relating to uncertainty and arousal during anticipation. Neuron  2001; 29: 537– 45. Google Scholar CrossRef Search ADS PubMed  Croxson PL, Walton ME, O'Reilly JX, Behrens TE, Rushworth MF. Effort-based cost-benefit valuation and the human brain. J Neurosci  2009; 29: 4531– 41. Google Scholar CrossRef Search ADS PubMed  Culbreth A, Westbrook A, Barch D. Negative symptoms are associated with an increased subjective cost of cognitive effort. J Abnorm Psychol  2016; 125: 528– 36. Google Scholar CrossRef Search ADS PubMed  Damiano CR, Aloi J, Treadway M, Bodfish JW, Dichter GS. Adults with autism spectrum disorders exhibit decreased sensitivity to reward parameters when making effort-based decisions. J Neurodev Disord  2012; 4: 13. Google Scholar CrossRef Search ADS PubMed  Delgado MR, Labouliere CD, Phelps EA. Fear of losing money? Aversive conditioning with secondary reinforcers. Soc Cogn Affect Neurosci  2006; 1: 250– 9. Google Scholar CrossRef Search ADS PubMed  Der-Avakian A, Markou A. The neurobiology of anhedonia and other reward-related deficits. Trends Neurosci  2012; 35: 68– 77. Google Scholar CrossRef Search ADS PubMed  Djamshidian A, Jha A, O'Sullivan SS, Silveira-Moriyama L, Jacobson C, Brown P, et al.   Risk and learning in impulsive and nonimpulsive patients with Parkinson's disease. Mov Disord  2010; 25: 2203– 10. Google Scholar CrossRef Search ADS PubMed  Docx L, de la Asuncion J, Sabbe B, Hoste L, Baeten R, Warnaerts N, et al.   Effort discounting and its association with negative symptoms in schizophrenia. Cogn Neuropsychiatry  2015; 20: 172– 85. Google Scholar CrossRef Search ADS PubMed  Drijgers RL, Dujardin K, Reijnders JS, Defebvre L, Leentjens AF. Validation of diagnostic criteria for apathy in Parkinson's disease. Parkinsonism Relat Disord  2010; 16: 656– 60. Google Scholar CrossRef Search ADS PubMed  Eshel N, Roiser JP. Reward and punishment processing in depression. Biol Psychiatry  2010; 68: 118– 24. Google Scholar CrossRef Search ADS PubMed  Evensen J, Rossberg JI, Barder H, Haahr U, Hegelstad W, Joa I, et al.   Apathy in first episode psychosis patients: a ten year longitudinal follow-up study. Schizophr Res  2012; 136: 19– 24. Google Scholar CrossRef Search ADS PubMed  Faerden A, Finset A, Friis S, Agartz I, Barrett EA, Nesvag R, et al.   Apathy in first episode psychosis patients: one year follow up. Schizophr Res  2010; 116: 20– 6. Google Scholar CrossRef Search ADS PubMed  Faerden A, Friis S, Agartz I, Barrett EA, Nesvag R, Finset A, et al.   Apathy and functioning in first-episode psychosis. Psychiatr Serv  2009; 60: 1495– 503. Google Scholar CrossRef Search ADS PubMed  Fauth-Buhler M, Zois E, Vollstadt-Klein S, Lemenager T, Beutel M, Mann K. Insula and striatum activity in effort-related monetary reward processing in gambling disorder: the role of depressive symptomatology. Neuroimage Clin  2014; 6: 243– 51. Google Scholar CrossRef Search ADS PubMed  Fervaha G, Foussias G, Agid O, Remington G. Amotivation and functional outcomes in early schizophrenia. Psychiatry Res  2013a; 210: 665– 8. Google Scholar CrossRef Search ADS   Fervaha G, Foussias G, Agid O, Remington G. Neural substrates underlying effort computation in schizophrenia. Neurosci Biobehav Rev  2013b; 37 ( 10 Pt 2): 2649– 65. Google Scholar CrossRef Search ADS   Fervaha G, Foussias G, Agid O, Remington G. Impact of primary negative symptoms on functional outcomes in schizophrenia. Eur Psychiatry  2014a; 29: 449– 55. Google Scholar CrossRef Search ADS   Fervaha G, Foussias G, Agid O, Remington G. Motivational and neurocognitive deficits are central to the prediction of longitudinal functional outcome in schizophrenia. Acta Psychiatr Scand  2014b; 130: 290– 9. Google Scholar CrossRef Search ADS   Fervaha G, Foussias G, Takeuchi H, Agid O, Remington G. Motivational deficits in major depressive disorder: cross-sectional and longitudinal relationships with functional impairment and subjective well-being. Compr Psychiatry  2016; 66: 31– 8. Google Scholar CrossRef Search ADS PubMed  Fervaha G, Graff-Guerrero A, Zakzanis KK, Foussias G, Agid O, Remington G. Incentive motivation deficits in schizophrenia reflect effort computation impairments during cost-benefit decision-making. J Psychiatr Res  2013c; 47: 1590– 6. Google Scholar CrossRef Search ADS   Fervaha G, Zakzanis KK, Foussias G, Graff-Guerrero A, Agid O, Remington G. Motivational deficits and cognitive test performance in schizophrenia. JAMA Psychiatry  2014c; 71: 1058– 65. Google Scholar CrossRef Search ADS   Fonseca MS, Murakami M, Mainen ZF. Activation of dorsal raphe serotonergic neurons promotes waiting but is not reinforcing. Curr Biol  2015; 25: 306– 15. Google Scholar CrossRef Search ADS PubMed  Forbes GB, Brown MR, Griffiths HJ. Arm muscle plus bone area: anthropometry and CAT scan compared. Am J Clin Nutr  1988; 47: 929– 31. Google Scholar CrossRef Search ADS PubMed  Foussias G, Mann S, Zakzanis KK, van Reekum R, Agid O, Remington G. Prediction of longitudinal functional outcomes in schizophrenia: the impact of baseline motivational deficits. Schizophr Res  2011; 132: 24– 7. Google Scholar CrossRef Search ADS PubMed  Foussias G, Remington G. Negative symptoms in schizophrenia: avolition and Occam's razor. Schizophr Bull  2010; 36: 359– 69. Google Scholar CrossRef Search ADS PubMed  Frederick S, Loewenstein G, O'Donoghue T. Time discounting and time preference: a critical review. J Econ Lit  2002; 40: 351– 401. Google Scholar CrossRef Search ADS   Gan JO, Walton ME, Phillips PE. Dissociable cost and benefit encoding of future rewards by mesolimbic dopamine. Nat Neurosci  2010; 13: 25– 7. Google Scholar CrossRef Search ADS PubMed  Gergelyfi M, Jacob B, Olivier E, Zenon A. Dissociation between mental fatigue and motivational state during prolonged mental activity. Front Behav Neurosci  2015; 9: 176. Google Scholar CrossRef Search ADS PubMed  Gold JM, Kool W, Botvinick MM, Hubzin L, August S, Waltz JA. Cognitive effort avoidance and detection in people with schizophrenia. Cogn Affect Behav Neurosci  2015a; 15: 145– 54. Google Scholar CrossRef Search ADS   Gold JM, Strauss GP, Waltz JA, Robinson BM, Brown JK, Frank MJ. Negative symptoms of schizophrenia are associated with abnormal effort-cost computations. Biol Psychiatry  2013; 74: 130– 6. Google Scholar CrossRef Search ADS PubMed  Gold JM, Waltz JA, Frank MJ. Effort cost computation in schizophrenia: a commentary on the recent literature. Biol Psychiatry  2015b; 78: 747– 53. Google Scholar CrossRef Search ADS   Gold JM, Waltz JA, Prentice KJ, Morris SE, Heerey EA. Reward processing in schizophrenia: a deficit in the representation of value. Schizophr Bull  2008; 34: 835– 47. Google Scholar CrossRef Search ADS PubMed  Green L, Myerson J. A discounting framework for choice with delayed and probabilistic rewards. Psychol Bull  2004; 130: 769– 92. Google Scholar CrossRef Search ADS PubMed  Green MF, Horan WP. Effort-based decision making in schizophrenia: evaluation of paradigms to measure motivational deficits. Schizophr Bull  2015; 41: 1021– 3. Google Scholar CrossRef Search ADS PubMed  Green MF, Horan WP, Barch DM, Gold JM. Effort-based decision making: a novel approach for assessing motivation in schizophrenia. Schizophr Bull  2015; 41: 1035– 44. Google Scholar CrossRef Search ADS PubMed  Haber SN. The primate basal ganglia: parallel and integrative networks. J Chem Neuroanat  2003; 26: 317– 30. Google Scholar CrossRef Search ADS PubMed  Hama S, Yamashita H, Shigenobu M, Watanabe A, Hiramoto K, Kurisu K, et al.   Depression or apathy and functional recovery after stroke. Int J Geriatr Psychiatry  2007; 22: 1046– 51. Google Scholar CrossRef Search ADS PubMed  Hamid AA, Pettibone JR, Mabrouk OS, Hetrick VL, Schmidt R, Vander Weele CM, et al.   Mesolimbic dopamine signals the value of work. Nat Neurosci  2016; 19: 117– 26. Google Scholar CrossRef Search ADS PubMed  Hartmann MN, Hager OM, Reimann AV, Chumbley JR, Kirschner M, Seifritz E, et al.   Apathy but not diminished expression in schizophrenia is associated with discounting of monetary rewards by physical effort. Schizophr Bull  2015; 41: 503– 12. Google Scholar CrossRef Search ADS PubMed  Hartmann MN, Hager OM, Tobler PN, Kaiser S. Parabolic discounting of monetary rewards by physical effort. Behav Processes  2013; 100: 192– 6. Google Scholar CrossRef Search ADS PubMed  Hershenberg R, Satterthwaite TD, Daldal A, Katchmar N, Moore TM, Kable JW, et al.   Diminished effort on a progressive ratio task in both unipolar and bipolar depression. J Affect Disord  2016; 196: 97– 100. Google Scholar CrossRef Search ADS PubMed  Hollon NG, Arnold MM, Gan JO, Walton ME, Phillips PE. Dopamine-associated cached values are not sufficient as the basis for action selection. Proc Natl Acad Sci USA  2014; 111: 18357– 62. Google Scholar CrossRef Search ADS PubMed  Horan WP, Reddy LF, Barch DM, Buchanan RW, Dunayevich E, Gold JM, et al.   Effort-based decision-making paradigms for clinical trials in schizophrenia: part 2-external validity and correlates. Schizophr Bull  2015; 41: 1055– 65. Google Scholar CrossRef Search ADS PubMed  Housden CR, O'Sullivan SS, Joyce EM, Lees AJ, Roiser JP. Intact reward learning but elevated delay discounting in Parkinson's disease patients with impulsive-compulsive spectrum behaviors. Neuropsychopharmacology  2010; 35: 2155– 64. Google Scholar CrossRef Search ADS PubMed  Hsu ES, Patwardhan AG, Meade KP, Light TR, Martin WR. Cross-sectional geometrical properties and bone mineral contents of the human radius and ulna. J Biomech  1993; 26: 1307– 18. Google Scholar CrossRef Search ADS PubMed  Hull CL. Principles of behavior, an introduction to behavior theory . New York, NY; London: Appleton-Century Company; 1943. Huys QJ, Pizzagalli DA, Bogdan R, Dayan P. Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis. Biol Mood Anxiety Disord  2013; 3: 12. Google Scholar CrossRef Search ADS PubMed  Inzlicht M, Schmeichel BJ, Macrae CN. Why self-control seems (but may not be) limited. Trends Cogn Sci  2014; 18: 127– 33. Google Scholar CrossRef Search ADS PubMed  Joutsa J, Voon V, Johansson J, Niemela S, Bergman J, Kaasinen V. Dopaminergic function and intertemporal choice. Transl Psychiatry  2015; 5: e491. Google Scholar CrossRef Search ADS PubMed  Kahneman D, Wright P. Changes of pupil size and rehearsal strategies in a short-term memory task. Q J Exp Psychol  1971; 23: 187– 96. Google Scholar CrossRef Search ADS PubMed  Kennerley SW, Dahmubed AF, Lara AH, Wallis JD. Neurons in the frontal lobe encode the value of multiple decision variables. J Cogn Neurosci  2009; 21: 1162– 78. Google Scholar CrossRef Search ADS PubMed  Kirkpatrick B, Fenton WS, Carpenter WTJr, Marder SR. The NIMH-MATRICS consensus statement on negative symptoms. Schizophr Bull  2006; 32: 214– 19. Google Scholar CrossRef Search ADS PubMed  Klein-Flugge MC, Kennerley SW, Saraiva AC, Penny WD, Bestmann S. Behavioral modeling of human choices reveals dissociable effects of physical effort and temporal delay on reward devaluation. PLoS Comput Biol  2015; 11: e1004116. Google Scholar CrossRef Search ADS PubMed  Knutson B, Fong GW, Adams CM, Varner JL, Hommer D. Dissociation of reward anticipation and outcome with event-related fMRI. Neuroreport  2001; 12: 3683– 7. Google Scholar CrossRef Search ADS PubMed  Kobayashi S, Schultz W. Influence of reward delays on responses of dopamine neurons. J Neurosci  2008; 28: 7837– 46. Google Scholar CrossRef Search ADS PubMed  Konstantakopoulos G, Ploumpidis D, Oulis P, Patrikelis P, Soumani A, Papadimitriou GN, et al.   Apathy, cognitive deficits and functional impairment in schizophrenia. Schizophr Res  2011; 133: 193– 8. Google Scholar CrossRef Search ADS PubMed  Kool W, McGuire JT, Rosen ZB, Botvinick MM. Decision making and the avoidance of cognitive demand. J Exp Psychol Gen  2010; 139: 665– 82. Google Scholar CrossRef Search ADS PubMed  Kroemer NB, Guevara A, Ciocanea Teodorescu I, Wuttig F, Kobiella A, Smolka MN. Balancing reward and work: anticipatory brain activation in NAcc and VTA predict effort differentially. Neuroimage  2014; 102 Pt 2: 510– 19. Google Scholar CrossRef Search ADS PubMed  Kurniawan IT, Guitart-Masip M, Dayan P, Dolan RJ. Effort and valuation in the brain: the effects of anticipation and execution. J Neurosci  2013; 33: 6160– 9. Google Scholar CrossRef Search ADS PubMed  Kurniawan IT, Seymour B, Talmi D, Yoshida W, Chater N, Dolan RJ. Choosing to make an effort: the role of striatum in signaling physical effort of a chosen action. J Neurophysiol  2010; 104: 313– 21. Google Scholar CrossRef Search ADS PubMed  Kurzban R, Duckworth A, Kable JW, Myers J. An opportunity cost model of subjective effort and task performance. Behav Brain Sci  2013; 36: 661– 79. Google Scholar CrossRef Search ADS PubMed  LaBar KS, Gatenby JC, Gore JC, LeDoux JE, Phelps EA. Human amygdala activation during conditioned fear acquisition and extinction: a mixed-trial fMRI study. Neuron  1998; 20: 937– 45. Google Scholar CrossRef Search ADS PubMed  Lak A, Stauffer WR, Schultz W. Dopamine prediction error responses integrate subjective value from different reward dimensions. Proc Natl Acad Sci USA  2014; 111: 2343– 8. Google Scholar CrossRef Search ADS PubMed  Lam M, Abdul Rashid NA, Lee SA, Lim J, Foussias G, Fervaha G, et al.   Baseline social amotivation predicts 1-year functioning in UHR subjects: a validation and prospective investigation. Eur Neuropsychopharmacol  2015; 25: 2187– 96. Google Scholar CrossRef Search ADS PubMed  Landes AM, Sperry SD, Strauss ME. Prevalence of apathy, dysphoria, and depression in relation to dementia severity in Alzheimer's disease. J Neuropsychiatry Clin Neurosci  2005; 17: 342– 9. Google Scholar CrossRef Search ADS PubMed  Laplane D, Dubois B. Auto-activation deficit: a basal ganglia related syndrome. Mov Disord  2001; 16: 810– 14. Google Scholar CrossRef Search ADS PubMed  Laplane D, Levasseur M, Pillon B, Dubois B, Baulac M, Mazoyer B, et al.   Obsessive-compulsive and other behavioural changes with bilateral basal ganglia lesions. A neuropsychological, magnetic resonance imaging and positron tomography study. Brain  1989; 112 ( Pt 3): 699– 725. Google Scholar CrossRef Search ADS PubMed  Le Bouc R, Pessiglione M. Imaging social motivation: distinct brain mechanisms drive effort production during collaboration versus competition. J Neurosci  2013; 33: 15894– 902. Google Scholar CrossRef Search ADS PubMed  Le Bouc R, Rigoux L, Schmidt L, Degos B, Welter ML, Vidailhet M, et al.   Computational dissection of dopamine motor and motivational functions in humans. J Neurosci  2016; 36: 6623– 33. Google Scholar CrossRef Search ADS PubMed  Levy DJ, Glimcher PW. The root of all value: a neural common currency for choice. Curr Opin Neurobiol  2012; 22: 1027– 38. Google Scholar CrossRef Search ADS PubMed  Levy R, Dubois B. Apathy and the functional anatomy of the prefrontal cortex-basal ganglia circuits. Cereb Cortex  2006; 16: 916– 28. Google Scholar CrossRef Search ADS PubMed  Manohar SG, Chong TT, Apps MA, Batla A, Stamelou M, Jarman PR, et al.   Reward pays the cost of noise reduction in motor and cognitive control. Curr Biol  2015; 25: 1707– 16. Google Scholar CrossRef Search ADS PubMed  Manohar SG, Husain M. Human ventromedial prefrontal lesions alter incentivisation by reward. Cortex  2016; 76: 104– 20. Google Scholar CrossRef Search ADS PubMed  Marin RS. Differential diagnosis and classification of apathy. Am J Psychiatry  1990; 147: 22– 30. Google Scholar CrossRef Search ADS PubMed  Martinez-Horta S, Riba J, de Bobadilla RF, Pagonabarraga J, Pascual-Sedano B, Antonijoan RM, et al.   Apathy in Parkinson's disease: neurophysiological evidence of impaired incentive processing. J Neurosci  2014; 34: 5918– 26. Google Scholar CrossRef Search ADS PubMed  Massar SA, Libedinsky C, Weiyan C, Huettel SA, Chee MW. Separate and overlapping brain areas encode subjective value during delay and effort discounting. Neuroimage  2015; 120: 104– 13. Google Scholar CrossRef Search ADS PubMed  Mauras T, Masson M, Fossati P, Pessiglione M. Incentive sensitivity as a behavioral marker of clinical remission from major depressive episode. J Clin Psychiatry  2016; 77: e697– 703. Google Scholar CrossRef Search ADS PubMed  Mayo NE, Fellows LK, Scott SC, Cameron J, Wood-Dauphinee S. A longitudinal view of apathy and its impact after stroke. Stroke  2009; 40: 3299– 307. Google Scholar CrossRef Search ADS PubMed  McCarthy JM, Treadway MT, Bennett ME, Blanchard JJ. Inefficient effort allocation and negative symptoms in individuals with schizophrenia. Schizophr Res  2016; 170: 278– 84. Google Scholar CrossRef Search ADS PubMed  Meyniel F, Goodwin GM, Deakin JW, Klinge C, MacFadyen C, Milligan H, et al.   A specific role for serotonin in overcoming effort cost. Elife  2016; 5: e17282. Google Scholar CrossRef Search ADS PubMed  Meyniel F, Safra L, Pessiglione M. How the brain decides when to work and when to rest: dissociation of implicit-reactive from explicit-predictive computational processes. PLoS Comput Biol  2014; 10: e1003584. Google Scholar CrossRef Search ADS PubMed  Meyniel F, Sergent C, Rigoux L, Daunizeau J, Pessiglione M. Neurocomputational account of how the human brain decides when to have a break. Proc Natl Acad Sci USA  2013; 110: 2641– 6. Google Scholar CrossRef Search ADS PubMed  Minamimoto T, Hori Y, Richmond BJ. Is working more costly than waiting in monkeys? PLoS One  2012; 7: e48434. Google Scholar CrossRef Search ADS PubMed  Miyazaki K, Miyazaki KW, Doya K. The role of serotonin in the regulation of patience and impulsivity. Mol Neurobiol  2012; 45: 213– 24. Google Scholar CrossRef Search ADS PubMed  Naccache L, Dehaene S, Cohen L, Habert MO, Guichart-Gomez E, Galanaud D, et al.   Effortless control: executive attention and conscious feeling of mental effort are dissociable. Neuropsychologia  2005; 43: 1318– 28. Google Scholar CrossRef Search ADS PubMed  Neu CM, Rauch F, Rittweger J, Manz F, Schoenau E. Influence of puberty on muscle development at the forearm. Am J Physiol Endocrinol Metab  2002; 283: E103– 7. Google Scholar CrossRef Search ADS PubMed  Niyogi RK, Shizgal P, Dayan P. Some work and some play: microscopic and macroscopic approaches to labor and leisure. PLoS Comput Biol  2014; 10: e1003894. Google Scholar CrossRef Search ADS PubMed  O'Doherty JP. Multiple systems for the motivational control of behavior and associated neural substrates in humans. Curr Top Behav Neurosci  2016; 27: 291– 312. Google Scholar CrossRef Search ADS PubMed  Padoa-Schioppa C, Cai X. The orbitofrontal cortex and the computation of subjective value: consolidated concepts and new perspectives. Ann N Y Acad Sci  2011; 1239: 130– 7. Google Scholar CrossRef Search ADS PubMed  Palminteri S, Serra G, Buot A, Schmidt L, Welter ML, Pessiglione M. Hemispheric dissociation of reward processing in humans: insights from deep brain stimulation. Cortex  2013; 49: 2834– 44. Google Scholar CrossRef Search ADS PubMed  Parvizi J, Rangarajan V, Shirer WR, Desai N, Greicius MD. The will to persevere induced by electrical stimulation of the human cingulate gyrus. Neuron  2013; 80: 1359– 67. Google Scholar CrossRef Search ADS PubMed  Pasquereau B, Turner RS. Limited encoding of effort by dopamine neurons in a cost-benefit trade-off task. J Neurosci  2013; 33: 8288– 300. Google Scholar CrossRef Search ADS PubMed  Pessiglione M, Schmidt L, Draganski B, Kalisch R, Lau H, Dolan RJ, et al.   How the brain translates money into force: a neuroimaging study of subliminal motivation. Science  2007; 316: 904– 6. Google Scholar CrossRef Search ADS PubMed  Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD. Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature  2006; 442: 1042– 5. Google Scholar CrossRef Search ADS PubMed  Peters J, Buchel C. Neural representations of subjective reward value. Behav Brain Res  2010; 213: 135– 41. Google Scholar CrossRef Search ADS PubMed  Petrovic P, Kalisch R, Pessiglione M, Singer T, Dolan RJ. Learning affective values for faces is expressed in amygdala and fusiform gyrus. Soc Cogn Affect Neurosci  2008; 3: 109– 18. Google Scholar CrossRef Search ADS PubMed  Piquado T, Isaacowitz D, Wingfield A. Pupillometry as a measure of cognitive effort in younger and older adults. Psychophysiology  2010; 47: 560– 9. Google Scholar CrossRef Search ADS PubMed  Piray P, Zeighami Y, Bahrami F, Eissa AM, Hewedi DH, Moustafa AA. Impulse control disorders in Parkinson's disease are associated with dysfunction in stimulus valuation but not action valuation. J Neurosci  2014; 34: 7814– 24. Google Scholar CrossRef Search ADS PubMed  Porat O, Hassin-Baer S, Cohen OS, Markus A, Tomer R. Asymmetric dopamine loss differentially affects effort to maximize gain or minimize loss. Cortex  2014; 51: 82– 91. Google Scholar CrossRef Search ADS PubMed  Prevost C, Pessiglione M, Metereau E, Clery-Melin ML, Dreher JC. Separate valuation subsystems for delay and effort decision costs. J Neurosci  2010; 30: 14080– 90. Google Scholar CrossRef Search ADS PubMed  Radakovic R, Abrahams S. Developing a new apathy measurement scale: Dimensional Apathy Scale. Psychiatry Res  2014; 219: 658– 63. Google Scholar CrossRef Search ADS PubMed  Raichle ME, Gusnard DA. Intrinsic brain activity sets the stage for expression of motivated behavior. J Comp Neurol  2005; 493: 167– 76. Google Scholar CrossRef Search ADS PubMed  Reddy LF, Horan WP, Barch DM, Buchanan RW, Dunayevich E, Gold JM, et al.   Effort-based decision-making paradigms for clinical trials in schizophrenia: part 1—psychometric characteristics of 5 paradigms. Schizophr Bull  2015; 41: 1045– 54. Google Scholar CrossRef Search ADS PubMed  Remy P, Doder M, Lees A, Turjanski N, Brooks D. Depression in Parkinson's disease: loss of dopamine and noradrenaline innervation in the limbic system. Brain  2005; 128 (Pt 6): 1314– 22. Google Scholar CrossRef Search ADS   Rigoux L, Guigon E. A model of reward- and effort-based optimal decision making and motor control. PLoS Comput Biol  2012; 8: e1002716. Google Scholar CrossRef Search ADS PubMed  Rizvi SJ, Pizzagalli DA, Sproule BA, Kennedy SH. Assessing anhedonia in depression: potentials and pitfalls. Neurosci Biobehav Rev  2016; 65: 21– 35. Google Scholar CrossRef Search ADS PubMed  Robert PH, Clairet S, Benoit M, Koutaich J, Bertogliati C, Tible O, et al.   The apathy inventory: assessment of apathy and awareness in Alzheimer's disease, Parkinson's disease and mild cognitive impairment. Int J Geriatr Psychiatry  2002; 17: 1099– 105. Google Scholar CrossRef Search ADS PubMed  Rudebeck PH, Walton ME, Smyth AN, Bannerman DM, Rushworth MF. Separate neural pathways process different decision costs. Nat Neurosci  2006; 9: 1161– 8. Google Scholar CrossRef Search ADS PubMed  Salamone JD, Correa M, Farrar A, Mingote SM. Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology  2007; 191: 461– 82. Google Scholar CrossRef Search ADS PubMed  Salamone JD, Yohn SE, Lopez-Cruz L, San Miguel N, Correa M. Activational and effort-related aspects of motivation: neural mechanisms and implications for psychopathology. Brain  2016; 139 ( Pt 5): 1325– 47. Google Scholar CrossRef Search ADS PubMed  Samanez-Larkin GR, Hollon NG, Carstensen LL, Knutson B. Individual differences in insular sensitivity during loss anticipation predict avoidance learning. Psychol Sci  2008; 19: 320– 3. Google Scholar CrossRef Search ADS PubMed  Samus QM, Rosenblatt A, Steele C, Baker A, Harper M, Brandt J, et al.   The association of neuropsychiatric symptoms and environment with quality of life in assisted living residents with dementia. Gerontologist  2005; 45 Spec No 1: 19– 26. Google Scholar CrossRef Search ADS PubMed  San-Galli A, Varazzani C, Abitbol R, Pessiglione M, Bouret S. Primate ventromedial prefrontal cortex neurons continuously encode the willingness to engage in reward-directed behavior. Cereb Cortex  2016. doi: 10.1093/cercor/bhw351. Schmidt L, Clery-Melin ML, Lafargue G, Valabregue R, Fossati P, Dubois B, et al.   Get aroused and be stronger: emotional facilitation of physical effort in the human brain. J Neurosci  2009; 29: 9450– 7. Google Scholar CrossRef Search ADS PubMed  Schmidt L, d'Arc BF, Lafargue G, Galanaud D, Czernecki V, Grabli D, et al.   Disconnecting force from money: effects of basal ganglia damage on incentive motivation. Brain  2008; 131 ( Pt 5): 1303– 10. Google Scholar CrossRef Search ADS PubMed  Schmidt L, Lebreton M, Clery-Melin ML, Daunizeau J, Pessiglione M. Neural mechanisms underlying motivation of mental versus physical effort. PLoS Biol  2012; 10: e1001266. Google Scholar CrossRef Search ADS PubMed  Schmidt L, Palminteri S, Lafargue G, Pessiglione M. Splitting motivation: unilateral effects of subliminal incentives. Psychol Sci  2010; 21: 977– 83. Google Scholar CrossRef Search ADS PubMed  Scholl J, Kolling N, Nelissen N, Wittmann MK, Harmer CJ, Rushworth MF. The good, the bad, and the irrelevant: neural mechanisms of learning real and hypothetical rewards and effort. J Neurosci  2015; 35: 11233– 51. Google Scholar CrossRef Search ADS PubMed  Schouppe N, Demanet J, Boehler CN, Ridderinkhof KR, Notebaert W. The role of the striatum in effort-based decision-making in the absence of reward. J Neurosci  2014; 34: 2148– 54. Google Scholar CrossRef Search ADS PubMed  Seymour B, O'Doherty JP, Koltzenburg M, Wiech K, Frackowiak R, Friston K, et al.   Opponent appetitive-aversive neural processes underlie predictive learning of pain relief. Nat Neurosci  2005; 8: 1234– 40. Google Scholar CrossRef Search ADS PubMed  Sherdell L, Waugh CE, Gotlib IH. Anticipatory pleasure predicts motivation for reward in major depression. J Abnorm Psychol  2012; 121: 51– 60. Google Scholar CrossRef Search ADS PubMed  Shidara M, Richmond BJ. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science  2002; 296: 1709– 11. Google Scholar CrossRef Search ADS PubMed  Skvortsova V, Palminteri S, Pessiglione M. Learning to minimize efforts versus maximizing rewards: computational principles and neural correlates. J Neurosci  2014; 34: 15621– 30. Google Scholar CrossRef Search ADS PubMed  Sockeel P, Dujardin K, Devos D, Deneve C, Destee A, Defebvre L. The Lille apathy rating scale (LARS), a new instrument for detecting and quantifying apathy: validation in Parkinson's disease. J Neurol Neurosurg Psychiatry  2006; 77: 579– 84. Google Scholar CrossRef Search ADS PubMed  Starkstein SE, Mayberg HS, Preziosi TJ, Andrezejewski P, Leiguarda R, Robinson RG. Reliability, validity, and clinical correlates of apathy in Parkinson's disease. J Neuropsychiatry Clin Neurosci  1992; 4: 134– 9. Google Scholar CrossRef Search ADS PubMed  Stephens DW, Krebs JR. Foraging theory . Princeton, NJ; Guildford: Princeton University Press; 1986. Strait CE, Blanchard TC, Hayden BY. Reward value comparison via mutual inhibition in ventromedial prefrontal cortex. Neuron  2014; 82: 1357– 66. Google Scholar CrossRef Search ADS PubMed  Strauss GP, Waltz JA, Gold JM. A review of reward processing and motivational impairment in schizophrenia. Schizophr Bull  2014; 40 (Suppl 2): S107– 16. Google Scholar CrossRef Search ADS PubMed  Strauss GP, Whearty KM, Morra LF, Sullivan SK, Ossenfort KL, Frost KH. Avolition in schizophrenia is associated with reduced willingness to expend effort for reward on a progressive ratio task. Schizophr Res  2016; 170: 198– 204. Google Scholar CrossRef Search ADS PubMed  Tachibana Y, Hikosaka O. The primate ventral pallidum encodes expected reward value and regulates motor action. Neuron  2012; 76: 826– 37. Google Scholar CrossRef Search ADS PubMed  Thobois S, Ardouin C, Lhommee E, Klinger H, Lagrange C, Xie J, et al.   Non-motor dopamine withdrawal syndrome after surgery for Parkinson's disease: predictors and underlying mesolimbic denervation. Brain  2010; 133 ( Pt 4): 1111– 27. Google Scholar CrossRef Search ADS PubMed  Treadway MT, Bossaller NA, Shelton RC, Zald DH. Effort-based decision-making in major depressive disorder: a translational model of motivational anhedonia. J Abnorm Psychol  2012; 121: 553– 8. Google Scholar CrossRef Search ADS PubMed  Treadway MT, Buckholtz JW, Schwartzman AN, Lambert WE, Zald DH. Worth the ‘EEfRT'? The effort expenditure for rewards task as an objective measure of motivation and anhedonia. PLoS One  2009; 4: e6598. Google Scholar CrossRef Search ADS PubMed  Treadway MT, Peterman JS, Zald DH, Park S. Impaired effort allocation in patients with schizophrenia. Schizophr Res  2015; 161: 382– 5. Google Scholar CrossRef Search ADS PubMed  Treadway MT, Zald DH. Reconsidering anhedonia in depression: lessons from translational neuroscience. Neurosci Biobehav Rev  2011; 35: 537– 55. Google Scholar CrossRef Search ADS PubMed  van Almenkerk S, Smalbrugge M, Depla MF, Eefsting JA, Hertogh CM. Apathy among institutionalized stroke patients: prevalence and clinical correlates. Am J Geriatr Psychiatry  2015; 23: 180– 8. Google Scholar CrossRef Search ADS PubMed  van Dalen JW, Moll van Charante EP, Nederkoorn PJ, van Gool WA, Richard E. Poststroke apathy. Stroke  2013; 44: 851– 60. Google Scholar CrossRef Search ADS PubMed  Varazzani C, San-Galli A, Gilardeau S, Bouret S. Noradrenaline and dopamine neurons in the reward/effort trade-off: a direct electrophysiological comparison in behaving monkeys. J Neurosci  2015; 35: 7866– 77. Google Scholar CrossRef Search ADS PubMed  Venugopalan VV, Casey KF, O'Hara C, O'Loughlin J, Benkelfat C, Fellows LK, et al.   Acute phenylalanine/tyrosine depletion reduces motivation to smoke cigarettes across stages of addiction. Neuropsychopharmacology  2011; 36: 2469– 76. Google Scholar CrossRef Search ADS PubMed  Voon V, Gao J, Brezing C, Symmonds M, Ekanayake V, Fernandez H, et al.   Dopamine agonists and risk: impulse control disorders in Parkinson's disease. Brain  2011; 134 ( Pt 5): 1438– 46. Google Scholar CrossRef Search ADS PubMed  Voon V, Napier TC, Frank MJ, Sgambato-Faure V, Grace AA, Rodriguez-Oroz M, et al.   Impulse control disorders and levodopa-induced dyskinesias in Parkinson's disease: an update. Lancet Neurol  2017; 16: 238– 50. Google Scholar CrossRef Search ADS PubMed  Voon V, Pessiglione M, Brezing C, Gallea C, Fernandez HH, Dolan RJ, et al.   Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors. Neuron  2010a; 65: 135– 42. Google Scholar CrossRef Search ADS   Voon V, Reynolds B, Brezing C, Gallea C, Skaljic M, Ekanayake V, et al.   Impulsive choice and response in dopamine agonist-related impulse control behaviors. Psychopharmacology  2010b; 207: 645– 59. Google Scholar CrossRef Search ADS   Wallis JD. Cross-species studies of orbitofrontal cortex and value-based decision-making. Nat Neurosci  2011; 15: 13– 19. Google Scholar CrossRef Search ADS PubMed  Walton ME, Bannerman DM, Alterescu K, Rushworth MF. Functional specialization within medial frontal cortex of the anterior cingulate for evaluating effort-related decisions. J Neurosci  2003; 23: 6475– 9. Google Scholar PubMed  Walton ME, Kennerley SW, Bannerman DM, Phillips PE, Rushworth MF. Weighing up the benefits of work: behavioral and neural analyses of effort-related decision making. Neural Netw  2006; 19: 1302– 14. Google Scholar CrossRef Search ADS PubMed  Wang J, Huang J, Yang XH, Lui SS, Cheung EF, Chan RC. Anhedonia in schizophrenia: deficits in both motivation and hedonic capacity. Schizophr Res  2015; 168: 465– 74. Google Scholar CrossRef Search ADS PubMed  Wardle MC, Treadway MT, Mayo LM, Zald DH, de Wit H. Amping up effort: effects of d-amphetamine on human effort-based decision-making. J Neurosci  2011; 31: 16597– 602. Google Scholar CrossRef Search ADS PubMed  Westbrook A, Kester D, Braver TS. What is the subjective cost of cognitive effort? Load, trait, and aging effects revealed by economic preference. PLoS One  2013; 8: e68210. Google Scholar CrossRef Search ADS PubMed  Wolf DH, Satterthwaite TD, Kantrowitz JJ, Katchmar N, Vandekar L, Elliott MA, et al.   Amotivation in schizophrenia: integrated assessment with behavioral, clinical, and imaging measures. Schizophr Bull  2014; 40: 1328– 37. Google Scholar CrossRef Search ADS PubMed  Yang XH, Huang J, Zhu CY, Wang YF, Cheung EF, Chan RC, et al.   Motivational deficits in effort-based decision making in individuals with subsyndromal depression, first-episode and remitted depression patients. Psychiatry Res  2014; 220: 874– 82. Google Scholar CrossRef Search ADS PubMed  Zenon A, Devesse S, Olivier E. Dopamine manipulation affects response vigor independently of opportunity cost. J Neurosci  2016a; 36: 9516– 25. Google Scholar CrossRef Search ADS   Zenon A, Duclos Y, Carron R, Witjas T, Baunez C, Regis J, et al.   The human subthalamic nucleus encodes the subjective value of reward and the cost of effort during decision-making. Brain  2016b; 139 ( Pt 6): 1830– 43. Google Scholar CrossRef Search ADS   Zenon A, Sidibe M, Olivier E. Pupil size variations correlate with physical effort perception. Front Behav Neurosci  2014; 8: 286. Google Scholar PubMed  Zipf GK. Human behavior and the principle of least effort: an introduction to human ecology . Cambridge, MA: Addison-Wesley Press; 1949. © The Author (2017). Published by Oxford University Press on behalf of the Guarantors of Brain. All rights reserved. For Permissions, please email: journals.permissions@oup.com http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Brain Oxford University Press

Why not try harder? Computational approach to motivation deficits in neuro-psychiatric diseases

Loading next page...
 
/lp/ou_press/why-not-try-harder-computational-approach-to-motivation-deficits-in-iOFY9z6lVm
Publisher
Oxford University Press
Copyright
© The Author (2017). Published by Oxford University Press on behalf of the Guarantors of Brain. All rights reserved. For Permissions, please email: journals.permissions@oup.com
ISSN
0006-8950
eISSN
1460-2156
D.O.I.
10.1093/brain/awx278
Publisher site
See Article on Publisher Site

Abstract

Abstract Motivation deficits, such as apathy, are pervasive in both neurological and psychiatric diseases. Even when they are not the core symptom, they reduce quality of life, compromise functional outcome and increase the burden for caregivers. They are currently assessed with clinical scales that do not give any mechanistic insight susceptible to guide therapeutic intervention. Here, we present another approach that consists of phenotyping the behaviour of patients in motivation tests, using computational models. These formal models impose a precise and operational definition of motivation that is embedded in decision theory. Motivation can be defined as the function that orients and activates the behaviour according to two attributes: a content (the goal) and a quantity (the goal value). Decision theory offers a way to quantify motivation, as the cost that patients would accept to endure in order to get the benefit of achieving their goal. We then review basic and clinical studies that have investigated the trade-off between the expected cost entailed by potential actions and the expected benefit associated with potential rewards. These studies have shown that the trade-off between effort and reward involves specific cortical, subcortical and neuromodulatory systems, such that it may be shifted in particular clinical conditions, and reinstated by appropriate treatments. Finally, we emphasize the promises of computational phenotyping for clinical purposes. Ideally, there would be a one-to-one mapping between specific neural components and distinct computational variables and processes of the decision model. Thus, fitting computational models to patients’ behaviour would allow inferring of the dysfunctional mechanism in both cognitive terms (e.g. hyposensitivity to reward) and neural terms (e.g. lack of dopamine). This computational approach may therefore not only give insight into the motivation deficit but also help personalize treatment. apathy, computational psychiatry, goal-directed behaviour, behavioural neurology, decision-making Introduction In this review, we present testing and analytic tools that may provide better quantification and discrimination of motivation deficits. These tools are embedded in the conceptual framework of decision theory. We thus start by sketching briefly the conceptual framework, then we describe the principles of behavioural tests and computational models that can be used to assess motivation, and finally we expose the potential neural bases of motivational processes and the typical manifestations of motivation deficits in the clinics. Note that our purpose is not to advocate a particular model but to present the interests of the computational approach in general. Conceptual framework The etymology suggests that the term motivation originally refers to a force that sets the behaviour in motion. Yet when we say that someone is strongly motivated, we seem to imply that motivation is something that we can quantify in order to predict the behaviour. And when we ask about real motivations behind observed behaviours, we look for reasons expressed in terms of implicit goals. Thus, motivation can be construed as a concept with two attributes, content and quantity, that somehow determine the behaviour. The effects of motivation are the direction of behaviour, which is determined by the content (i.e. the goal), and the intensity of behaviour, which is determined by the quantity (i.e. the goal value). The issue with clinical scales While neuroscientists investigate the processes through which the brain selects and implements goals such that they can drive the behaviour, clinicians are mostly concerned with assessing the intensity of motivation in their patients. This means estimating the values that the patient assigns to standard goals that people may have in general, such as getting a good job position. It is usually done using questionnaires with multiple answers that can be scored to quantify the motivation deficit, i.e. apathy (Levy and Dubois, 2006; Drijgers et al., 2010). Different apathy rating scales have been proposed that vary in the richness of clinical details and consequently in the duration of assessment (Marin, 1990; Starkstein et al., 1992; Robert et al., 2002; Sockeel et al., 2006; Radakovic and Abrahams, 2014). These scales obviously help the patients to express their trouble with motivation. For this purpose, questions use words from common language, such as ‘Do you have motivation?’ (Question 7 in Starkstein’s apathy scale) (Table 1). Table 1 Starkstein’s apathy scale 1.  Are you interested in learning new things?  Not at all  Slightly  Some  A lot  2.  Does anything interest you?  Not at all  Slightly  Some  A lot  3.  Are you concerned about your condition?  Not at all  Slightly  Some  A lot  4.  Do you put much effort into things?  Not at all  Slightly  Some  A lot  5.  Are you always looking for something to do?  Not at all  Slightly  Some  A lot  6.  Do you have plans and goals for the future?  Not at all  Slightly  Some  A lot  7.  Do you have motivation?  Not at all  Slightly  Some  A lot  8.  Do you have the energy for daily activities?  Not at all  Slightly  Some  A lot  9.  Does someone have to tell you what to do each day?  Not at all  Slightly  Some  A lot  10.  Are you indifferent to things?  Not at all  Slightly  Some  A lot  11.  Are you unconcerned with many things?  Not at all  Slightly  Some  A lot  12.  Do you need a push to get started on things?  Not at all  Slightly  Some  A lot  13.  Are you neither happy nor sad, just in between?  Not at all  Slightly  Some  A lot  14.  Would you consider yourself apathetic?  Not at all  Slightly  Some  A lot  1.  Are you interested in learning new things?  Not at all  Slightly  Some  A lot  2.  Does anything interest you?  Not at all  Slightly  Some  A lot  3.  Are you concerned about your condition?  Not at all  Slightly  Some  A lot  4.  Do you put much effort into things?  Not at all  Slightly  Some  A lot  5.  Are you always looking for something to do?  Not at all  Slightly  Some  A lot  6.  Do you have plans and goals for the future?  Not at all  Slightly  Some  A lot  7.  Do you have motivation?  Not at all  Slightly  Some  A lot  8.  Do you have the energy for daily activities?  Not at all  Slightly  Some  A lot  9.  Does someone have to tell you what to do each day?  Not at all  Slightly  Some  A lot  10.  Are you indifferent to things?  Not at all  Slightly  Some  A lot  11.  Are you unconcerned with many things?  Not at all  Slightly  Some  A lot  12.  Do you need a push to get started on things?  Not at all  Slightly  Some  A lot  13.  Are you neither happy nor sad, just in between?  Not at all  Slightly  Some  A lot  14.  Would you consider yourself apathetic?  Not at all  Slightly  Some  A lot  Each answer is scored between 0 and 3, with the highest score corresponding to lowest motivation (leftmost answer for Questions 1–8, rightmost answer for Questions 9-14). Based on the bimodal distribution observed in patients with Parkinson’s disease, a score superior to 14 has been suggested as a marker of apathy. Although they help clinicians to get a rough idea of the motivation deficit, these scales suffer from some limitations. First, they depend on the patient’s insight, which may not be fine-grained in the case of diseases that affect cognitive functions, or on the insight of relatives and caregivers, who can be absent or too distant from the patient to give valuable information. Second, they assess entities (either motivation as a whole or subcategories such as ‘interests’ or ‘concerns’) that are not likely to have direct counterparts at the neural level. To overcome these limitations, we suggest the following 2-fold alternative strategy: (i) adding behavioural tests to questionnaires; and (ii) characterizing behavioural performance using a normative framework—namely, decision theory (Fig. 1). Figure 1 View largeDownload slide A schematic view of motivation. The box-and-arrow schema illustrates goal-directed behaviour. The brain adjusts the direction and intensity of behaviour so as to reduce the delay or increase the probability of goal attainment. Decision theoretic principles posit that agents should maximize the net value, obtained by subtracting costs (effort and time involved by the behaviour) from benefits (how much rewarding overcomes punishing aspects of the goal). To perform this optimization the brain needs an approximate anticipation of both costs and benefits. In this framework, motivation can have three different meanings: motivation-as-content refers to the goal, motivation-as-quantity refers to the goal value, motivation-as-process refers to behavioural adjustments toward the goal. Figure 1 View largeDownload slide A schematic view of motivation. The box-and-arrow schema illustrates goal-directed behaviour. The brain adjusts the direction and intensity of behaviour so as to reduce the delay or increase the probability of goal attainment. Decision theoretic principles posit that agents should maximize the net value, obtained by subtracting costs (effort and time involved by the behaviour) from benefits (how much rewarding overcomes punishing aspects of the goal). To perform this optimization the brain needs an approximate anticipation of both costs and benefits. In this framework, motivation can have three different meanings: motivation-as-content refers to the goal, motivation-as-quantity refers to the goal value, motivation-as-process refers to behavioural adjustments toward the goal. The promises of decision theory Elementary principles of decision theory assume that when considering whether or not to take a course of action, agents contrast costs and benefits to get a net value. If the action is compared to doing nothing, then it is engaged only when its net value is positive. If it is compared to a set of alternative actions, then it is engaged only when its net value is above the others.   NetValue(Actioni)= Benefit[Goal(Actioni)]-Cost(Actioni) (1) The benefit term corresponds to the value of the action goal, i.e. how good it is for the agent to engage this action. Importantly, values may be attached to actions in two different ways. On the one hand, the action may be inherently valuable, like when reading a book because it is fun. In this case the goal is the action itself, and the motivation is said to be intrinsic. On the other hand, the action may be good because it leads to a valuable outcome, like when reading a book is required to pass an examination. In this case the goal is the exam success, the motivation is extrinsic, and the behaviour qualified as instrumental. There is therefore no major conceptual difference between extrinsic and intrinsic motivation from the perspective of decision theory. Note that the goal can be multidimensional and include both positive and negative elements, which may be called gains and losses or rewards and punishments. For instance, being the captain of a team can be set as a goal because it has high positive value on the dimensions of power and self-esteem, which may overcome the negative values related to duties and responsibilities. Naturally, a given anticipated state of the world can only be set as a goal if the positive values (of rewards) exceed the negative values (of punishments). The action does not necessarily lead to the goal, i.e. the valuable state anticipated by the agent, in a direct and deterministic way. For the benefit to be positive, actions only need to bring the agent closer to the goal, by decreasing either the uncertainty or the delay attached to goal reaching. In fact, subjective estimates show that people indeed discount the goal value by both delay and uncertainty. There is a huge literature on delay and uncertainty discounting, the latter being also linked to the notion of risk, defined as the variance of possible outcomes (Frederick et al., 2002; Green and Myerson, 2004). Critically, however, delay and uncertainty are not action costs: they are only modulators of goal value. Action costs are induced by the allocation of resources required for performing the action. They can be of two sorts: time and effort. Time is a cost because if time is spent on a given course of action it is no longer available for other actions that might be profitable. Of course, this opportunity cost of time only matters for actions that cannot be executed simultaneously. Effort is a cost because effortful actions consume resources that might be needed later and will have to be restored through some other costly actions. This is obvious in the case of physical effort, which consumes energetic resources and fatigues the muscles such that they may not operate efficiently for later purposes. It is not that obvious in the case of mental effort, for which the idea of a biological resource being consumed is still debated (Inzlicht et al., 2014; Botvinick and Braver, 2015). Indeed most of, if not all, the energy consumed by the brain is used for maintaining spontaneous activity, i.e. activity that is not related to a particular cognitive task (Raichle and Gusnard, 2005). This has led some authors to argue that people avoid mental effort because of the opportunity cost induced by cognitive resource allocation. The idea is that when we engage central cognitive modules in a given task, they are no longer available for other possible beneficial tasks (Kurzban et al., 2013; Boureau et al., 2015). In any case, people tend to avoid mental effort (Kool et al., 2010; Westbrook et al., 2013), so we must assume that it does entail a cost, which still needs to be specified at the biological and/or functional level. Action costs can be taken as measures of motivation intensity. This is because the action is only engaged if the net value is positive. Therefore, the highest cost that a person is willing to accept, in order to reach a particular goal, is equal to the goal value. In principle, motivation could be assessed by measuring the time that the person would be willing to invest to reach the goal. Yet in practice, researchers have focused on effort when developing behavioural paradigms to assess motivation. Behavioural tests By definition, it is impossible to vary intrinsic motivation and assess the behavioural consequences without changing the task to be performed, which may introduce a problematic confound. It is much simpler to manipulate motivation extrinsically, by varying the outcome associated with a given task or action, which can require more or less effort. The possibility that effortful actions trigger intrinsic motivation might be an issue in theory, but in practice empirical studies have shown that animals and persons tend to follow the law of least effort (Hull, 1943; Zipf, 1949). Experimental paradigms have been originally developed to test animals, mostly rodents, and were later adapted to humans. Two categories may be distinguished, depending on whether a choice is explicitly implemented or not. Interestingly, the processes targeted by the two sorts of tasks have been named differently, with an emphasis either on effort or reward (e.g. ‘effort-based decision’ and ‘incentive motivation’), although paradigms invariably involve the manipulation of both reward and effort levels (Table 2). Table 2 A synthetic view of motivational tests used in humans Type of …  Main options  … task  Selection within continuous range (Schmidt et al., 2008) Binary choice (Treadway et al., 2012) Willingness to accept (Chong et al., 2015)    … effort  Physical effort:  power grip peak (Pessiglione et al., 2007) or duration (Meyniel et al., 2013) number of button presses (Treadway et al., 2009) Mental effort:  N-back (Kool et al., 2010) numerical Stroop (Schmidt et al., 2012)    … reward  Real reward such as money (Le Bouc and Pessiglione, 2013) Virtual reward such as apples (Bonnelle et al., 2015) Subliminal reward (real or virtual money) (Schmidt et al., 2010)    … model  Hyperbolic or exponential discounting (Prevost et al., 2010) Parabolic (Hartmann et al., 2013) or sigmoidal discounting (Klein-Flugge et al., 2015) Subtraction of supralinear cost function (Le Bouc et al., 2016)  Type of …  Main options  … task  Selection within continuous range (Schmidt et al., 2008) Binary choice (Treadway et al., 2012) Willingness to accept (Chong et al., 2015)    … effort  Physical effort:  power grip peak (Pessiglione et al., 2007) or duration (Meyniel et al., 2013) number of button presses (Treadway et al., 2009) Mental effort:  N-back (Kool et al., 2010) numerical Stroop (Schmidt et al., 2012)    … reward  Real reward such as money (Le Bouc and Pessiglione, 2013) Virtual reward such as apples (Bonnelle et al., 2015) Subliminal reward (real or virtual money) (Schmidt et al., 2010)    … model  Hyperbolic or exponential discounting (Prevost et al., 2010) Parabolic (Hartmann et al., 2013) or sigmoidal discounting (Klein-Flugge et al., 2015) Subtraction of supralinear cost function (Le Bouc et al., 2016)  The table lists the main options (for the type of task, effort, reward, and model) that have been implemented in behavioural paradigms designed to assess motivation in human participants. References suggest one paper in which the corresponding option has been implemented. The advantages and drawbacks of the different options are explained in the text. Note that the list is non-exhaustive and that references are somewhat arbitrary. Binary choice In effort-based decision tasks, subjects have a choice between two options: producing little effort for small reward, or producing a higher effort for a bigger reward. This has been operationalized in rodents, for instance with a T-maze (Fig. 2) where a big reward (more food pellets) is placed in one arm behind a barrier, whereas the barrier-free arm only leads to a small reward (Salamone et al., 2007). Another example would be an operant box with two levers, one requiring more presses but providing more food pellets than the other (Walton et al., 2006). The number of food pellets (benefit) and the height of the barrier or the number of lever presses (cost) can be varied so as to determine the cost that the animal is willing to accept in order to get one food pellet, which gives a measure for the subjective reward value. Figure 2 View largeDownload slide Behavioural tasks assessing motivation as a reward/effort trade-off. Motivation intensity can be quantified as the amount of effort that the agent is willing to expend in order to obtain a potential reward. This trade-off between effort and reward has been operationalized in binary choice tasks (left) or free operant tasks (right). In animals (top), binary choice is typically implemented in a T-maze, with one branch representing the small reward/low effort option and the other branch the bigger reward (more food)/higher effort (due to the barrier) option. Free operant behaviour is classically assessed in a Skinner box where pressing the lever triggers food delivery. When the ratio between the number of lever presses and the number of food pellets is fixed, the reward obtained is simply proportional to the effort produced, such that the reward/effort levels are freely selected within a continuum. Note that the same sort of apparatus can be used to implement binary choice, with two levers associated with different ratios between effort and reward levels (not shown). Thus, binary choice can be envisaged as a special case of free operant behaviour, for which a large range of options (and not just two) are available. This can also be seen in the behavioural tests adapted to human participants (bottom), where effort is produced on a power grip and reward is given in monetary units. The figure shows one key screenshot of the visual animation presented to participants in a task trial. The vertical orange bar provides a visual feedback on the force produced, within a grid that is scaled to the participant’s maximal force. In the free operant version (sometimes called incentive motivation task), all grip forces between zero and maximal force may be selected, for a proportional reward that ranges from zero to full incentive (€1 in the illustrated example). In the binary choice version (sometimes called effort discounting task), only two reward-effort combinations are available (force levels are indicated by orange horizontal targets and reward levels by coin images). This behavioural paradigm can also be adapted to measure directly the willingness-to-accept a given effort for a given reward. In this case (not shown), a single option is displayed on the screen, as a target/coin association that the participant chooses to accept or decline. Figure 2 View largeDownload slide Behavioural tasks assessing motivation as a reward/effort trade-off. Motivation intensity can be quantified as the amount of effort that the agent is willing to expend in order to obtain a potential reward. This trade-off between effort and reward has been operationalized in binary choice tasks (left) or free operant tasks (right). In animals (top), binary choice is typically implemented in a T-maze, with one branch representing the small reward/low effort option and the other branch the bigger reward (more food)/higher effort (due to the barrier) option. Free operant behaviour is classically assessed in a Skinner box where pressing the lever triggers food delivery. When the ratio between the number of lever presses and the number of food pellets is fixed, the reward obtained is simply proportional to the effort produced, such that the reward/effort levels are freely selected within a continuum. Note that the same sort of apparatus can be used to implement binary choice, with two levers associated with different ratios between effort and reward levels (not shown). Thus, binary choice can be envisaged as a special case of free operant behaviour, for which a large range of options (and not just two) are available. This can also be seen in the behavioural tests adapted to human participants (bottom), where effort is produced on a power grip and reward is given in monetary units. The figure shows one key screenshot of the visual animation presented to participants in a task trial. The vertical orange bar provides a visual feedback on the force produced, within a grid that is scaled to the participant’s maximal force. In the free operant version (sometimes called incentive motivation task), all grip forces between zero and maximal force may be selected, for a proportional reward that ranges from zero to full incentive (€1 in the illustrated example). In the binary choice version (sometimes called effort discounting task), only two reward-effort combinations are available (force levels are indicated by orange horizontal targets and reward levels by coin images). This behavioural paradigm can also be adapted to measure directly the willingness-to-accept a given effort for a given reward. In this case (not shown), a single option is displayed on the screen, as a target/coin association that the participant chooses to accept or decline. Equivalent paradigms have been developed for human subjects, with effort being manipulated in terms of, for example, the force to be exerted on a handgrip (Fig. 2) or the number of clicks on a mouse (Treadway et al., 2009; Prevost et al., 2010). Compared to all the other measures, the grip task has the advantage of isolating effort from delay. This is because in standard designs, participants are instructed to produce short pulses, such that the effort cost (linked to peak force) can vary while keeping the duration roughly constant. This prevents situations in which enhancing effort postpones the reward, and thus increases temporal discounting of reward. Another potential confound is risk, if participants are uncertain about their ability to produce the required effort. This is generally avoided by restricting the range of forces to what participants can achieve with a 100% chance of success. The rewards used in human studies are generally more abstract than in animals: typically money or even points (tokens). They are generally thought to involve the same brain system as primary rewards, as implied by the notion of ‘common neural currency’ (Levy and Glimcher, 2012). Their advantage is 2-fold: first they are easy to quantify and second they prevent the issue of satiety, which may change the reward value during the course of the experiment. As famously argued by economists, marginal utility might be a decreasing function of monetary amount, meaning that a same objective gain would appears as less valuable (subjectively), once significant money has been accumulated. However, this effect likely remains limited given the small incentive ranges spanned in standard experiments. Willingness to accept In incentive motivation (or instrumental behaviour activation) tasks, there is no explicit choice in the sense that the alternative options are not explicitly laid down. For example, in paradigms using progressive ratio schedules, the number of lever presses required for a given number of food pellets is increased from one trial block to the next, until the animal ceases responding. As seen before, the number of lever presses at break point can be taken as a direct measure of the motivation induced by the food reward (i.e. the incentive). This sort of task can also be interpreted in the framework of decision theory, as there is an implicit decision to make, between pressing the lever or not. In other words, there is a hidden option that is just doing nothing. In fact this option is always present in animal experiments, which may stop performing the task at any time, for instance by running away or by breaking fixation, such that these ‘errors’ can be interpreted as motivated choices (Minamimoto et al., 2012). This sort of choices, with a single action that may be performed or not, can be made more explicit in humans, in the form of ‘accept’ versus ‘decline’ alternatives. Such tasks measure the willingness to accept performing a unique type of action for various combinations of effort and reward levels (Bonnelle et al., 2015). It could be argued that such yes/no decisions are more natural for the brain, because in the ancestral environment in which it has evolved (before the emergence of stores and restaurants), options do not appear simultaneously (Stephens and Krebs, 1986). When a new option arises, the agent is likely to be already pursuing a goal, such that the decision is whether to keep with the current course of action or to switch to the new alternative. Selection within a continuous range Another paradigm where choices are implicit are free operant tasks using fixed ratio schedules, meaning that the number of food pellets is proportional to the number of lever presses (Fig. 2). From a decision-theoretic perspective, animals are viewed as making decisions between pressing and not pressing at every time step, or choosing the time interval between presses, which may give a continuous set of numerous hidden options, depending on the resolution of time estimates and the maximal interval allowed (Niyogi et al., 2014). In a human version of this free operant paradigm (Fig. 2), participants are first shown a given incentive level (amount of money) and then asked to squeeze a handgrip, knowing that payoff will be proportional to peak force (Schmidt et al., 2008). More precisely, the payoff is calculated as the fraction of the incentive corresponding to the proportion of maximal force that subjects produce, such that they get half the incentive if they produce half their maximal force. Here again, the task (sometimes called incentive motivation task) can be viewed as a continuous version of the binary choice task (sometimes called effort discounting task), where the set of alternative options includes all possible forces between zero and maximal force. An interesting variant of this paradigm can be obtained by indexing the payoff not on peak force but on effort duration (Meyniel et al., 2013). In this case, a target force level is imposed and payoff accumulates, at a speed proportional to incentive level, as long as the force being produced by the participant is above the target. The duration of the trial (typically 30 s) is too long for the force to be maintained throughout the end, so subjects have to release the grip at some point, take a break to recover from fatigue and then start squeezing again to get more money. Effort production in this task can be interpreted as resulting from decisions about the durations of effort and rest periods, which in principle can take any value from zero to trial length. Specificities of testing humans Testing humans instead of animals present a number of advantages. One is that verbal instructions, associated with some familiarization with the task, may permit to investigate steady performance (i.e. with limited learning effects), without overtraining the subjects, which might make performance more like a habit. This is important because producing habitual behaviour has been shown to involve brain circuits different from those underpinning goal-directed behaviour (O'Doherty, 2016). A second advantage is the possibility to use virtual reward and effort. Indeed, consequential choices, where subjects have to perform the effort and truly receive the reward, can be reduced to a random fraction or even suppressed. This opens the possibility of presenting on every trial new items that resemble the sort of effort and reward we encounter in everyday life (an offer could be, for instance: would you clean up your room if I get you an ice cream?). Previous studies have shown that using virtual compared to real reward make little difference on effort production or intertemporal choice (Bickel et al., 2009; Schmidt et al., 2010). Yet it might not be neutral for decisions to propose effort items that are never actually implemented during the task. Some patients might have issues with anticipated costs, on which a priori decisions are based, and others with experienced costs, which would influence decisions posterior to effort production. Manipulating anticipated versus experienced costs has indeed been shown to result in distinct behavioural patterns (Meyniel et al., 2014). A third advantage is the accessibility of mental effort, which may be much more amenable to experimental investigation. Mental effort generally means attentional effort and has been manipulated by varying the difficulty of executive tasks such as Stroop, N-back, or task-switching. Just as physical effort, mental effort can be implemented in decision-making or incentive motivation tasks, i.e. with explicit or implicit choices. In decision tasks, it has been well-established that healthy subjects prefer to avoid difficult executive tasks (e.g. 3-back compared to 1-back), and only accept to perform them if the expected reward is significantly bigger than the one associated with an easy version of the same task (Kool et al., 2010; Westbrook et al., 2013; Apps et al., 2015). However, incentive effects on performance in tasks involving attentional effort appeared to be weaker than those observed with physical effort (Schmidt et al., 2012). This might mean that mental effort is more difficult to adjust, or less costly compared to physical effort. Finally, testing humans enables the assessment of possible dissociations between conscious and subconscious effects of incentive levels. This can be done by masking the cues indicating incentive levels to subjects, using standard subliminal presentation procedures. It has been demonstrated that subjects produce more effort in the grip task for higher incentives, even if they cannot report which incentive was presented (Pessiglione et al., 2007). This effect was nonetheless tiny compared to that observed in supraliminal conditions, where strategic adaptations (saving resources for when they matter) are likely to have an impact. In both cases (subliminal and supraliminal), the incentive motivation effect was associated with an increase in skin conductance response, a measure of autonomic activity. It is not entirely clear whether subliminal motivation effects correspond to a (subconscious) instrumental adjustment of behaviour, or to a more basic appetitive reflex, or even to emotional arousal. Indeed, when presenting emotional pictures incidentally and orthogonally to incentive levels (Schmidt et al., 2009), emotional arousal was found to increase effort production in a manner that was independent from motivational effects (no interaction). Note however that, although appetitive and emotional reactions to incentives may affect effort production and related decisions in many paradigms, their effect size seems limited in comparison to instrumental effects. Computational modelling Even in simple behavioural readouts of motivation, several factors might play a role. For instance, a shift in the propensity to favour high reward–high effort options could be due to an increased sensitivity to reward or a decreased sensitivity to effort. Computational models may be helpful for disentangling between such alternative explanations of behavioural changes. Modelling the cost/benefit trade-off Computational models are algorithms that perform the task imposed to subjects, meaning that they generate behavioural outputs through a limited series of mathematical operations applied to the experimental factors. Decision theory provides a generic and normative framework for deriving the computational models that can be used to simulate the behaviour in specific motivation tests. If in these experimental tests, goals can be reduced to the rewards that actions provide, and if actions can be reduced to the amount of effort that they involve, then Equation (1) can be simplified to:   V(Ei)=R(Ei)−C(Ei) (2) with V(Ei) being the net value of producing effort Ei, R(Ei) the reward associated to effort Ei and C(Ei) the cost of producing effort Ei. The contingencies between reward and effort are directly specified by the task, for example €10 can be associated with reaching 80% of maximal grip force. With money, reward level is an objective amount (such as €10) that can be directly entered in the equation, if we ignore distortions such as decreasing marginal utility. The main difficulty in establishing such models is to specify the cost of the required effort (80% of maximal force in the example). Pioneering investigations of how rewards are subjectively devalued by effort were inspired by delay discounting models, which account for how rewards are subjectively devalued by delay of delivery (Green and Myerson, 2004; Prevost et al., 2010). Non-linear functions such as exponential or hyperbolic decay with time are the mathematical forms most commonly used for delay discounting. Note that these functions can only give positive values. This is problematic in the case of effort because it would mean that agents always prefer producing the effort (even climbing a mountain for a peanut) than doing nothing. As this would be absurd, we focused in this review on a subtractive form of the value function, following on recent formalizations (Manohar et al., 2015; Le Bouc et al., 2016). This is not anecdotal, since it opens the possibility of attaching negative values to the production of efforts that are not rewarded enough. Discounting can nevertheless be non-linear with this subtractive form, depending on the shape of the cost function. Cost functions have been a matter of debate in recent years. In the case of physical effort, a consensus seems to emerge around the notion that discounting should be concave, as in parabolic functions (Hartmann et al., 2013), or at least initially concave, as in sigmoidal functions (Klein-Flugge et al., 2015). Interestingly, classical formalization in motor control theory, where action cost is typically defined as the integral of squared motor command, also predicts that cost should be a supralinear function of the objective muscle contraction (Rigoux and Guigon, 2012). These studies therefore converge to the conclusion that physical effort cost increases faster than a motor output such as grip force. The cost function might be different in the case of mental effort (Westbrook et al., 2013; Bonnelle et al., 2015), for which the essence of effort cost remains controversial. As an illustration, we describe below a net value function that could be integrated in a computational model to account for the behaviour in the incentive motivation test, where the payoff is proportional to the reward at stake and to the force produced:   V(Fi)=(1+kr.R).Fi−kc.(1+kf.T).Fi/(1−Fi) (3) with Fi being the considered peak force (in proportion to maximal force), R the potential reward (incentive level) and T the trial index. Note that Fi is the controlled variable that the agent must set (in order to maximize the net value), while R and T are experimental factors imposed by the task design. Simulations of costs and benefits generated by this value function, and the associated behavioural patterns, are shown in Fig. 3. Figure 3 View largeDownload slide Simulation of a motivation model. Simulation results were obtained with a model that can be approximated by Equation (3). It was applied to the incentive motivation task that is illustrated in Fig. 2 (bottom right) and used in a recent publication (Le Bouc et al., 2016), with six reward levels randomly distributed over 60 trials. (A) Simulated hidden variables. In a given trial, the model anticipates for each possible force peak (a proxy for effort level), the associated benefit (left) and the associated cost (right). The expected benefit is proportional to the force peak, with a slope that depends on the reward level offered in the present trial, i.e. the payoff corresponding to maximal force production (only four levels are illustrated). The expected cost is a supralinear function of force peak, with a slope that depends on fatigue level, i.e. the number of trials completed so far (two fatigue levels are illustrated, with the red line and red dots showing the expected cost at the beginning and at the end of the task, respectively). The net value is obtained by subtracting costs from benefits. The predicted behaviour is the optimal force peak (for which the net value is maximal), as illustrated by the green arrows. Note that effort expenditure (predicted force peak) increases with reward level, reproducing the incentive motivation effect. (B) Simulated behavioural patterns. Top and bottom panels show how the predicted behaviour varies with the two task factors: reward level and trial index. The different columns show how the predicted behaviour varies when changing one single free parameter (grey versus black lines). While changing the sensitivity to effort cost (Kc) globally shifts effort production up or down, changing the sensitivity to reward (Kr) affects the impact of reward level, and changing the sensitivity to fatigue (Kf) affects the impact of trial index. Figure 3 View largeDownload slide Simulation of a motivation model. Simulation results were obtained with a model that can be approximated by Equation (3). It was applied to the incentive motivation task that is illustrated in Fig. 2 (bottom right) and used in a recent publication (Le Bouc et al., 2016), with six reward levels randomly distributed over 60 trials. (A) Simulated hidden variables. In a given trial, the model anticipates for each possible force peak (a proxy for effort level), the associated benefit (left) and the associated cost (right). The expected benefit is proportional to the force peak, with a slope that depends on the reward level offered in the present trial, i.e. the payoff corresponding to maximal force production (only four levels are illustrated). The expected cost is a supralinear function of force peak, with a slope that depends on fatigue level, i.e. the number of trials completed so far (two fatigue levels are illustrated, with the red line and red dots showing the expected cost at the beginning and at the end of the task, respectively). The net value is obtained by subtracting costs from benefits. The predicted behaviour is the optimal force peak (for which the net value is maximal), as illustrated by the green arrows. Note that effort expenditure (predicted force peak) increases with reward level, reproducing the incentive motivation effect. (B) Simulated behavioural patterns. Top and bottom panels show how the predicted behaviour varies with the two task factors: reward level and trial index. The different columns show how the predicted behaviour varies when changing one single free parameter (grey versus black lines). While changing the sensitivity to effort cost (Kc) globally shifts effort production up or down, changing the sensitivity to reward (Kr) affects the impact of reward level, and changing the sensitivity to fatigue (Kf) affects the impact of trial index. The benefit term, (1+kr.R).Fi, accounts for the contingency imposed by experimental design, according to which exerting more force linearly increases the monetary payoff. Such contingency might be present in many real-life situations where exerting more effort would enhance the probability or reduce the delay of reward delivery. Other contingencies could be envisaged, for instance an all-or-none payoff depending on whether a target such as a force window is hit or missed. In this case, the benefit term could be formalized as krR.P(Fi), with P(Fi) being the probability of hitting the target given the considered peak force and some motor noise. Similar formulations can be applied to situations where the outcome is not a potential reward but a potential punishment such as monetary loss. The benefit term also includes a constant, which may account for the fact that producing more force has positive outcomes other than money, for instance it could make an impression on the experimenter. This is consistent with the repeated observation that subjects squeeze the grip in this task even for negligible monetary incentives (Schmidt et al., 2008; Le Bouc et al., 2016). The cost term includes an explosive cost function, kc.Fi/(1-Fi), modulated by a fatigue function, (1+kf.T). The cost function is a simple approximation of the equation derived from motor control theory (Rigoux and Guigon, 2012; Lebouc et al., 2016). Note that if F is expressed as a proportion of maximal force, then cost is null when no force is produced, and infinite when force approaches the theoretical maximum. This maximum may correspond to what can be expected at best from the musculature of the arm. In the case of handgrip, previous studies (Forbes et al., 1988; Hsu et al., 1993; Neu et al., 2002) have shown that maximal force can be approximated from simple measures of the forearm length, circumference and skin width (a proxy for non-muscular tissues). Fatigue is modelled as a linear increase with the number of trials achieved so far, for the sake of simplicity. This fatigue function captures the phenomenon that a same force is more and more costly to produce as fatigue progressively kicks in. More complex functions of trial index could be envisaged, and cumulative effort over trials could be a better proxy than mere trial index. We also note that a symmetrical function could be included in the benefit term to account for the phenomenon of satiety, according to which incentive effects would diminish with trial index or cumulative reward. We do not imply that the simple formulation adopted in Equation (3) is the unique possibility or that it provides the best account of motivational processes. We chose this illustration because it respects the fundamental principles of decision theory and because it is sufficient to understand the model fitting approach and hence the interests of computational phenotyping. Fitting model free parameters The net value function suggested in Equation (3) follows on a normative principle, in the sense that it specifies what subjects should do in order to maximize benefits and minimize costs. However, it does not discard the possibility that individuals may vary in their subjective attitude towards various dimensions such as reward, effort, fatigue etc. This subjectivity is enabled by the constants (the k parameters), which account for variations in how agents weight the various factors that impact cost and benefit terms. Constants in computational models are viewed as free parameters because they can be adjusted to fit (as closely as possible) the behaviour exhibited by a given subject in a given test. Value functions can be fitted directly to behavioural measures in choice paradigms that allow inferring indifference points, i.e. reward–effort combinations that are selected in 50% of trials when confronted two-by-two. The fitting procedure in this case consists of searching the set of free parameters that yield a theoretical value function whose distance from indifference points is minimal, as classically implemented in least squares methods. Indifference points can be determined through an adaptive design, such as a stair-case procedure, that increments effort and/or rewards levels based on observed choices (Westbrook et al., 2013; Klein-Flugge et al., 2015; Le Bouc et al., 2016). Another approach is to integrate in the model a function that generates the behaviour, in accordance to a value maximization principle. In the incentive motivation test, where any force can be chosen between zero and maximum, the predicted behaviour is simply the peak force that gives the maximal net value (for which the value derivative is null): F* = argmax V(F). If noise is incorporated in the model, then a probability (or likelihood) can be assigned to every observable peak force. Here, model fitting consists of searching for the set of free parameters that maximize the log likelihood of the peak force series produced across trials. This is implemented in a variety of optimization algorithms, from basic grid search methods to Bayesian model inversion techniques. Similar computational modelling and fitting approaches can be applied to binary choice, which can be seen as a special case of the incentive motivation problem, where the option set is reduced to just one pair. In this case, the two option values, which can be calculated through Equation (3), are generally entered into a softmax function, which gives the likelihood of the observed choice: P(Fc) = 1/(1+exp(β.(V(Fu)-V(Fc)))), with Fc and Fu being the chosen and unchosen forces, respectively and β a free parameter termed ‘inverse temperature’ that captures the noise in the choice process. The shape of the softmax function is a sigmoid that gives a probability of 0.5 when the two option values are equal, and converges to 1 or 0 when one option gets much better than the other. Fitting the parameters of the net value function thus reduces to (non-linear) logistic regression, which is classically employed to account for binary data. Eventually, adjusted parameters, or parameter estimates, obtained by fitting the model to the data, characterize the subject’s behaviour in a testing session. They provide a computational phenotype that quantifies individual attitudes such as sensitivity to reward attraction (kr), effort cost (kc), fatigue effect (kf), etc. These models are therefore not purely normative, in the sense that they do not determine what should be done once and for all: they instead take into account interindividual differences. This does not imply that they represent trait measures, as a same subject in different states might produce different behaviours, which would be best explained by different parameters. Changes in parameter estimates can therefore be used to evaluate the impact of a disease, or the effect of a treatment, or even normal fluctuations (in mood for instance). Disease and treatment effects can also be assessed in a subtly different way, through Bayesian model comparison, which provides a metric to elect the most plausible model given the behavioural data. The idea here is to provide qualitative, rather than quantitative, differences. For instance, one may want to compare models in which the disease affects sensitivity to reward versus sensitivity to effort, to discriminate between different forms of apathy (see application to Parkinson’s disease in Fig. 4). Figure 4 View largeDownload slide Computational characterization of motivation deficit. The figure illustrates the computational approach of motivation deficit, taking the example of dopamine depletion in Parkinson’s disease (adapted from Le Bouc et al., 2016). Behavioural data were acquired in a group of Parkinson’s disease patients tested ON and OFF dopaminergic medication, as well as in matched healthy controls, using the binary choice and free operant tasks displayed in Fig. 2. The model fitted to behavioural data can be approximated by Equation (3), and includes the same free parameters as in the simulations (Fig. 3). (A) Model-free results. Graphs show inter-subject means and standard errors for the force selected in the choice task, or freely generated in the motivation task, as a function of incentive level (from 0.1 to 5€). The slopes appeared to differ between groups, an effect that was captured by computational modelling. (B) Comparison of parameter estimates. Graphs show inter-subject means and standard errors for posterior estimates of free parameters, after model fitting through Bayesian inversion. Parameter estimates were then compared between groups using t-tests. Only parameter Kr differed significantly between Parkinson’s disease patients and controls, and between ON and OFF patients. (C) Clinico-computational correlation. The effects of dopaminergic medication on the computational parameter Kr and on the Starkstein apathy score were correlated across patients. This supports the idea that dopaminergic medication alleviates the motivation deficit by increasing reward sensitivity (as opposed to decreasing effort cost or susceptibility to fatigue). (D) Bayesian model selection. If we only consider the three parameters Kr, Kc and Kf, there are 23 = 8 possible models (each parameter can be affected or not by medication). Then for each parameter we compare the four models in which medication has an effect to the four models in which there is no effect. Exceedance probability suggests that a medication effect on Kr is highly plausible (compared to no effect), whereas an absence of effect was much more plausible for Kc and Kf. Thus, model selection leads to the same conclusion as comparing parameters estimates: the effect of dopaminergic medication is a selective modulation of Kr. PD = Parkinson’s disease. Figure 4 View largeDownload slide Computational characterization of motivation deficit. The figure illustrates the computational approach of motivation deficit, taking the example of dopamine depletion in Parkinson’s disease (adapted from Le Bouc et al., 2016). Behavioural data were acquired in a group of Parkinson’s disease patients tested ON and OFF dopaminergic medication, as well as in matched healthy controls, using the binary choice and free operant tasks displayed in Fig. 2. The model fitted to behavioural data can be approximated by Equation (3), and includes the same free parameters as in the simulations (Fig. 3). (A) Model-free results. Graphs show inter-subject means and standard errors for the force selected in the choice task, or freely generated in the motivation task, as a function of incentive level (from 0.1 to 5€). The slopes appeared to differ between groups, an effect that was captured by computational modelling. (B) Comparison of parameter estimates. Graphs show inter-subject means and standard errors for posterior estimates of free parameters, after model fitting through Bayesian inversion. Parameter estimates were then compared between groups using t-tests. Only parameter Kr differed significantly between Parkinson’s disease patients and controls, and between ON and OFF patients. (C) Clinico-computational correlation. The effects of dopaminergic medication on the computational parameter Kr and on the Starkstein apathy score were correlated across patients. This supports the idea that dopaminergic medication alleviates the motivation deficit by increasing reward sensitivity (as opposed to decreasing effort cost or susceptibility to fatigue). (D) Bayesian model selection. If we only consider the three parameters Kr, Kc and Kf, there are 23 = 8 possible models (each parameter can be affected or not by medication). Then for each parameter we compare the four models in which medication has an effect to the four models in which there is no effect. Exceedance probability suggests that a medication effect on Kr is highly plausible (compared to no effect), whereas an absence of effect was much more plausible for Kc and Kf. Thus, model selection leads to the same conclusion as comparing parameters estimates: the effect of dopaminergic medication is a selective modulation of Kr. PD = Parkinson’s disease. Although the sort of computational models exposed here provide reasonable accounts for optimization processes, they are silent about how values are generated. When the anticipated outcomes are expressed in objective metrics such as a number of pellets or a sum in euros, a slight subjective distortion might provide a good enough approximation. But when they come to episodes embedded in social contexts, such as family vacation projects, current models fall short of suggesting how values are constructed. Nevertheless, decision-theoretic models can be used to optimize action selection if proxies for subjective values are obtained from participants through likeability or desirability ratings on analogue scales. Note, however, that decision theory is agnostic about whether values have an affective component or not. The expected value (at the time of choice) and the experienced value (at the time of consumption) do not necessarily match what could be subjectively felt as desire (or dread) and pleasure (or pain). What is important for natural selection is that behavioural policies maximize reproductive success. Desire and pleasure might be just proxies for how good a given state is, i.e. how much it favours eventual reproduction. They might partially depart from the values that actually guide the behaviour, through the maximization mechanisms that the brain has evolved and which may be modelled by decision theory. Similar issues may arise when autonomic data, such as pupil diameter or skin conductance, are used to fit computational models, as was done in passive paradigms, i.e. in the absence of behavioural data (Petrovic et al., 2008). The question is to what extent autonomic responses represent valid proxies for the values that guide the behaviour. It has been shown that skin conductance responses were correlated with the magnitude or probability of both reward and punishment (LaBar et al., 1998; Critchley et al., 2001; Delgado et al., 2006; Schmidt et al., 2009; Gergelyfi et al., 2015). Thus, skin conductance could be used as a proxy for the absolute value of reward or punishment separately, but would be off for actions that combine positive and negative outcomes. Besides, it remains unknown whether skin conductance, which reflects arousal levels, strictly matches the value estimates that guide decisions. Along the same lines, pupil dilation is generally considered as a marker of effort, both physical and mental (Kahneman and Wright, 1971; Beatty and Wagoner, 1978; Piquado et al., 2010; Alnaes et al., 2014; Zenon et al., 2014). Thus, pupil size could be taken as a proxy for effort level, although it remains unclear how closely it would follow the effort cost estimate integrated in the net value equation that guides decisions. The alternative is that pupil size may reflect the amount of effort eventually invested in the action when it is performed, and not the amount of effort that is anticipated during decision-making. It should also be stressed that the dissociation between skin conductance and pupil size as reflecting outcome value and effort cost is not clear-cut, as both measures may reflect a common form of autonomic arousal. Finally, we have restricted our computational presentation to the basics of decision theory, leaving aside learning processes. It is nonetheless expected that learning should occur at many levels in the tests used to assess motivation, not only with respect to outcome value but also with regards to sensorimotor contingencies. Learning effects can be partially controlled through training sessions on the task, or captured by relevant learning models. A presentation of these models would, however, go far beyond the scope of this review. Neural implementation In this section, we briefly review studies that investigated motivation as a trade-off between cost and benefit—effort and reward, typically. We focus on work in primates that might have a direct translation to clinics, even if this work was largely inspired by earlier studies in rodents, for which we refer readers to a recent review (Salamone et al., 2016). We first explore activation studies that link brain activity to reward/effort optimization, in both humans and monkeys. We then review the behavioural consequences of lesions and pharmacological manipulations in animals. What can be learned from lesions and pharmacological treatments in human clinical conditions is discussed in the next section. Meta-analysis of functional MRI studies (Fig. 5) helps with delineating a set of brain regions involved in reward and effort processing. We deliberately set a conservative statistical threshold to focus on key nodes, so we do not pretend to provide an exhaustive description. The reward network includes the orbitofrontal cortex (mostly the medial part), the ventral striatum bilaterally and the midbrain (around dopaminergic nuclei). The effort network includes principally the anterior cingulate cortex and bilateral anterior insula. In the following we briefly consider the cortical, subcortical and neuromodulatory components of these networks. Figure 5 View largeDownload slide Meta-analysis of reward/effort neural representation. Statistical maps show with colour-coded z-score the results of functional MRI meta-analysis performed on the Neurosynth platform. As the number of functional MRI studies employing the keywords ‘reward’ and ‘effort’ was very different, we used reverse and forward inference, respectively. The slices were selected so as to display the most noticeable clusters, and superimposed on canonical anatomical template. The [x, y, z] coordinates refer to the Montreal Neurological Institute (MNI) space. aI = anterior insula; vS = ventral striatum. Note that less significant clusters were observed in the posterior and anterior cingulate cortex for reward, and in the inferior parietal and inferior frontal cortex for effort. Figure 5 View largeDownload slide Meta-analysis of reward/effort neural representation. Statistical maps show with colour-coded z-score the results of functional MRI meta-analysis performed on the Neurosynth platform. As the number of functional MRI studies employing the keywords ‘reward’ and ‘effort’ was very different, we used reverse and forward inference, respectively. The slices were selected so as to display the most noticeable clusters, and superimposed on canonical anatomical template. The [x, y, z] coordinates refer to the Montreal Neurological Institute (MNI) space. aI = anterior insula; vS = ventral striatum. Note that less significant clusters were observed in the posterior and anterior cingulate cortex for reward, and in the inferior parietal and inferior frontal cortex for effort. Cortical structures The medial part of the orbitofrontal cortex (or ventromedial prefrontal cortex, vmPFC) has been repeatedly shown to represent reward values in humans (Peters and Buchel, 2010; Bartra et al., 2013; Clithero and Rangel, 2014). More precisely, vmPFC haemodynamic response was positively correlated with the stimulus value, whether it was an objective value such as monetary amount, or a subjective value such as likeability rating. Accordingly, single-unit activity in the monkey vmPFC was strongly associated with stimulus value, integrating subjective aspects such as factors related to the internal state (satiety) or representations stored in memory (Bouret and Richmond, 2010; Strait et al., 2014; Abitbol et al., 2015). Single-unit recordings in monkeys have also established correlations (both positive and negative) with stimulus value in more lateral parts of the orbitofrontal cortex (lOFC) (Padoa-Schioppa and Cai, 2011; Wallis, 2011). In contrast, physical effort levels associated to actions were positively correlated with the dorsal anterior cingulate cortex (dACC) haemodynamic response, which also correlated negatively with the reward levels associated to action outcomes (Burke et al., 2013; Kurniawan et al., 2013; Skvortsova et al., 2014; Bonnelle et al., 2015; Scholl et al., 2015). These results are consistent with a role for the dACC in integrating costs and benefits and computing the net value of actions, whereas the role of the vmPFC might be confined to the outcome space, ignoring action costs. Physical effort levels were also found to be reflected in the anterior insula, together with outcomes bearing strong aversive valence such as pain or punishment (Seymour et al., 2005; Pessiglione et al., 2006; Samanez-Larkin et al., 2008; Prevost et al., 2010; Skvortsova et al., 2014). Interestingly, cognitive effort appeared to recruit the same brain areas, notably the dACC (Schouppe et al., 2014; Massar et al., 2015). The differential involvement of the ACC and vmPFC/lOFC in effort representations was also observed in primate studies. Indeed single cell activities were more sensitive to the amount of effort needed to obtain a reward in the ACC compared to the lOFC/vmPFC (Shidara and Richmond, 2002; Kennerley et al., 2009; San-Galli et al., 2016). Similar conclusions were reached from lesion studies in rats: lesions of the ACC decreased the willingness to climb a barrier to get the reward (Walton et al., 2003). Furthermore, a double dissociation was found between ACC and OFC lesions, which impaired effort-based and delay-based decisions, respectively (Rudebeck et al., 2006). The same dissociation has been observed in humans, with the delay of reward delivery modulating value representation in the vmPFC, and the effort associated to the action requested to obtain the reward modulating value representation in the dACC (Prevost et al., 2010). Subcortical structures In functional MRI studies using incentive motivation tasks, the ventral striatum and pallidum were found to activate with the amount of reward at stake and to drive motor performance (Knutson et al., 2001; Schmidt et al., 2012; Kroemer et al., 2014). This activation of the ventral striato-pallidum complex was observed in subliminal conditions, where subjects were unaware of incentive level (Pessiglione et al., 2007), and was dissociated from emotional arousal (Schmidt et al., 2009). It was also shown to drive both physical and mental effort, depending on the task demand (Schmidt et al., 2012). The ventral striatum and pallidum are part of the limbic circuit of the basal ganglia, which receives projections from the orbitofrontal cortex (Brown and Pluck, 2000; Haber, 2003). Nevertheless, their role in incentive motivation seems different from that of the orbitofrontal cortex, as they increase their activity when subjects produce more effort (to get more reward). Similar conclusions have been drawn from electrophysiological studies in monkeys (Tachibana and Hikosaka, 2012). However, when reward and effort levels have to be integrated in order to make a choice (and not to drive performance), activity in the striatum appeared to signal the net value, i.e. reward minus effort (Botvinick et al., 2009; Croxson et al., 2009; Kurniawan et al., 2010). Another line of research examined the role of dopamine in the nucleus accumbens, a structure homologous to the ventral striatum in rodents (see Salamone et al., 2007 for a review). Increasing and decreasing dopamine levels shifted the choices toward high effort–large reward and low effort–small reward options, respectively. This result could be explained either by dopamine enhancing reward attractiveness, or by dopamine diminishing effort cost. Voltammetry studies in rodents suggested that dopamine release in the nucleus accumbens scales with reward but not with effort level (Gan et al., 2010; Hollon et al., 2014; Hamid et al., 2016). This is in-line with single cell electrophysiological recordings in monkeys that showed a relatively limited sensitivity of dopamine neurons to information about upcoming physical effort, compared to expected reward (Pasquereau and Turner, 2013; Varazzani et al., 2015). This relative lack of sensitivity to effort by dopamine neurons contrasts with their strong sensitivity to other forms of cost that relate to reward delivery, such as risk or delay (Kobayashi and Schultz, 2008; Lak et al., 2014). These results suggest that dopamine helps to cope with effortful actions by signalling reward availability, without contributing to the estimation of effort cost. This is compatible with the energizing effects of dopamine enhancers that have been observed in humans (Wardle et al., 2011) (see also next section), or the reduced motivation following acute phenylalanine/tyrosine depletion, which reduces dopamine level (Venugopalan et al., 2011). If not dopamine, other neuromodulators might be involved in signalling effort cost, such as noradrenaline and serotonin. Electrophysiological recordings showed that when monkeys initiate an action, the firing of neurons in the locus coeruleus scales positively with the amount of force required (Bouret and Richmond, 2015; Varazzani et al., 2015). This is compatible with the idea that noradrenaline release signals the amount of effort required by upcoming actions. However, locus coeruleus neurons displayed virtually no response to information about effort provided by a visual cue, in contrast to what has been reported for ACC neurons in similar tasks (Kennerley et al., 2009). One interpretation is that noradrenaline is important at the time of action, when the effort needs to be produced, but not in the earlier decision to make the effort or not. Evidence for a role of serotonin in the processing of effort cost comes from pharmacological manipulation in healthy humans. Increasing serotonin level with antidepressants (SSRIs) prolonged effort duration in an incentive motivation task, an effect that was computationally captured by a reduction of effort cost (Meyniel et al., 2016). This is in agreement with a more general role for serotonin in overcoming costs, as was previously shown for the delay subjects have to wait before obtaining the reward (Miyazaki et al., 2012; Fonseca et al., 2015). Finally, opioids might also play a role in moderating the impact of experienced or anticipated effort, which could be seen as a prolongation of their role in attenuating pain experience. A tentative neuro-computational scenario It is tempting to establish links between the net value equation that adjusts the reward/effort trade-off and the findings reviewed in this section. A speculative but reasonable summary might be that (i) some regions, such as the vmPFC, compute the benefit term; (ii) other regions, such as the dACC, integrate the effort cost and signal the net value; and (iii) neuromodulators adjust the weights of reward and effort in the net value estimation. In Fig. 6, we go further and suggest a possible implementation of the computational mechanism that finds the optimal behaviour by maximizing the net value equation. For illustration, we take the case of the incentive force task (Le Bouc et al., 2016), a subcase of free operant paradigms (Fig. 2), to which Equation (3) can be applied. The problem for the brain is to find the force F* that maximizes the net value function V(F), without explicitly computing the value of all admissible forces. One solution is adjusting force dynamically so as to operate a gradient ascent on the value function, as follows:   ∂F∂t=λ∂V∂F (4) where the temporal derivative of force is obtained by multiplying the derivative of value with respect to force by a parameter λ that controls the rate of convergence of the gradient ascent. By construction, the steady-state limt→∞F(t) of this ordinary differential equation is a local maximum F* of the value function V(F). Computationally speaking, Equation (4) requires the moment-to-moment calculation of the gradient ∂V/∂F=kr∂B/∂F−kc∂C/∂F of the value function V with respect to force F. This gradient is decomposed into two terms: the gradient of benefits and the gradient of costs with respect to force. The former measures the efficiency of effort investment, in terms of the net benefit accrued by a unitary change in force. The latter measures the susceptibility of costs, in terms of the net energy expenditure incurred when force is increased by a unitary amount. In turn, this scheme highlights the computational nodes that are critical for performing effort allocation in the brain. In brief, effort allocation requires brain systems specialized for (i) evaluating the benefit and cost gradients; (ii) aggregating those to form the net gradient; and (iii) transforming the latter into the appropriate dynamical adjustment of instantaneous force production. In addition, the scheme may involve a feedback loop of proprioceptive and/or afferent copies of sensorimotor signals that enable the evaluation of the cost gradient. Note that in this particular design, the benefit gradient is constant and equal to the potential reward R (i.e. incentive level). Figure 6 View largeDownload slide A speculative implementation of motivation process in the human brain. The figure illustrates the mechanisms that the brain might implement to generate behaviour in a typical incentive motivation task, where one behavioural output must be selected within a continuous range. Under decision theory, the behavioural output must maximize the benefit B and minimize the cost C. In this example (see incentive force task in Le Bouc et al., 2016), the behavioural output is a force F, the benefit is a fraction of the reward R corresponding to F (relative to maximal force Fmax), and the cost is a supralinear function of F. The problem for the brain is therefore to find the force F that maximizes the net value (V = B – C) (see explanation in the text). (A) Plausible anatomical locations for the representation of computational variables involved in the model. The pivotal region would be the dACC, which would integrate the goal value conveyed by ventral fronto-striatal circuits and the effort cost transmitted by the anterior insula. The dACC would then send the net value to premotor and motor regions, which would elaborate a motor command for the muscles. Muscle contraction would on the one hand have an added value, and on the other entail an effort cost, which would be integrated in the corresponding brain regions. Within a particular trial, the behaviour would be generated through several loops during which all representations are updated. Note that the loop can be internally simulated (with no overt movement) if the motor regions inform directly (by an efferent copy of the motor command) the perceptual regions, from which effort cost can be estimated. There is no consensus about the role of neuromodulators in this machinery—here we illustrate a tentative functional repartition where dopamine modulates the goal value, 5HT the effort cost and noradrenaline the net value. (B) Mechanistic derivation of optimal behaviour in a putative brain-scale network. The key mechanism is a gradient ascent: the temporal derivative of force (generated in motor regions) would scale with the derivative of net value with respect to force (aggregated in the dACC by subtracting cost derivative from benefit derivative). In the incentive motivation task, the derivative of the benefit is proportional to reward R (i.e. incentive level). The effective connectivity in the network would correspond to the weighting of the different factors influencing the net value, in particular reward (kr) and effort (kc). The dynamics at longer time scales (across trials), which could give rise to fatigue or satiety effects, is not represented. 5HT = serotonin; B = expected benefit; C = expected cost; DA = dopamine; dACC = dorsal anterior cingulate cortex; F = produced force; Fp = perceived force; M1 = primary motor cortex; NA = noradrenaline; PM = premotor cortex; R = reward level; SI-II = primary and secondary somatosensory cortex; V = net value; vmPFC = ventromedial prefrontal cortex; VS = ventral striatum. Figure 6 View largeDownload slide A speculative implementation of motivation process in the human brain. The figure illustrates the mechanisms that the brain might implement to generate behaviour in a typical incentive motivation task, where one behavioural output must be selected within a continuous range. Under decision theory, the behavioural output must maximize the benefit B and minimize the cost C. In this example (see incentive force task in Le Bouc et al., 2016), the behavioural output is a force F, the benefit is a fraction of the reward R corresponding to F (relative to maximal force Fmax), and the cost is a supralinear function of F. The problem for the brain is therefore to find the force F that maximizes the net value (V = B – C) (see explanation in the text). (A) Plausible anatomical locations for the representation of computational variables involved in the model. The pivotal region would be the dACC, which would integrate the goal value conveyed by ventral fronto-striatal circuits and the effort cost transmitted by the anterior insula. The dACC would then send the net value to premotor and motor regions, which would elaborate a motor command for the muscles. Muscle contraction would on the one hand have an added value, and on the other entail an effort cost, which would be integrated in the corresponding brain regions. Within a particular trial, the behaviour would be generated through several loops during which all representations are updated. Note that the loop can be internally simulated (with no overt movement) if the motor regions inform directly (by an efferent copy of the motor command) the perceptual regions, from which effort cost can be estimated. There is no consensus about the role of neuromodulators in this machinery—here we illustrate a tentative functional repartition where dopamine modulates the goal value, 5HT the effort cost and noradrenaline the net value. (B) Mechanistic derivation of optimal behaviour in a putative brain-scale network. The key mechanism is a gradient ascent: the temporal derivative of force (generated in motor regions) would scale with the derivative of net value with respect to force (aggregated in the dACC by subtracting cost derivative from benefit derivative). In the incentive motivation task, the derivative of the benefit is proportional to reward R (i.e. incentive level). The effective connectivity in the network would correspond to the weighting of the different factors influencing the net value, in particular reward (kr) and effort (kc). The dynamics at longer time scales (across trials), which could give rise to fatigue or satiety effects, is not represented. 5HT = serotonin; B = expected benefit; C = expected cost; DA = dopamine; dACC = dorsal anterior cingulate cortex; F = produced force; Fp = perceived force; M1 = primary motor cortex; NA = noradrenaline; PM = premotor cortex; R = reward level; SI-II = primary and secondary somatosensory cortex; V = net value; vmPFC = ventromedial prefrontal cortex; VS = ventral striatum. These theoretical brain systems can be mapped onto possible anatomical locations based on the neuroscientific literature on the reward/effort trade-off (Fig. 6). This neural implementation is of course highly speculative but may serve to derive predictions about the sort of behavioural deficits that should be induced by specific brain damage. Lastly, this computational perspective suggests that the coupling strength between the aggregation system and the benefit and cost systems corresponds to the control parameters kr and kc. In other terms, induced changes in effective connectivity between these systems should induce concomitant variations in the way individuals trade costs against benefits. This mechanistic interpretation is interesting because it ties together the known biological impact of neuromodulators onto synaptic plasticity and the behavioural effect of related pharmacological drugs onto effort allocation. Clinical manifestation The different neural structures involved in the reward/effort trade-off can be dysfunctional in a number of pathological conditions, and/or targeted by therapeutic interventions. Some specific motivational dysfunctions have been identified, in both neurology and psychiatry, and distinguished from motor or cognitive disabilities that might also affect the behaviour. We review this work, with a special emphasis on studies that rely upon computational modelling. Neurology Motivation disorders have been observed following focal lesions (stroke or tumour) and degenerative diseases. The most frequent disorder is apathy, which is generally defined as a reduction of goal-directed behaviour, and present in 50% of cases after traumatic brain injury (Arnould et al., 2013), 40% in Parkinson’s disease (Starkstein et al., 1992), 50% in pre-dementia Alzheimer’s disease (Landes et al., 2005), and 35% in patients after vascular lesions (Caeiro et al., 2013; van Dalen et al., 2013). Apathy alters functional recovery during rehabilitation after vascular lesions, and represents an independent predictor of poor functional outcome (Hama et al., 2007; Mayo et al., 2009; Caeiro et al., 2013), as well as everyday life dependency (van Almenkerk et al., 2015). Apathy has a negative impact on quality of life and can also lead to significant burden for caregivers (Aarsland et al., 1999; Samus et al., 2005). The condition in which reward and effort processing have been the most investigated is Parkinson’s disease, which is characterized by a progressive degeneration of dopaminergic neurons, prominent motor symptoms such as akinesia, and frequent non-motor symptoms such as apathy. In Parkinson’s disease, apathy might result from dopamine depletion in the mesocorticolimbic pathway, which connects the ventral tegmental area to ventral striatum, orbitofrontal and anterior cingulate cortex (Remy et al., 2005; Thobois et al., 2010; Brown et al., 2012; Martinez-Horta et al., 2014). Comparing Parkinson’s disease patients with and without dopaminergic medication has brought some insight into the role of dopamine in the reward/effort trade-off. Dopaminergic medication has been shown to increase the amount of effort that Parkinson’s disease patients were willing to produce for a given reward in a progressive ratio schedule task that involved clicking on a keyboard to earn money (Porat et al., 2014), and in an accept versus decline choice task that involved squeezing a handgrip to earn fictive rewards (Chong et al., 2016). Dopaminergic medication has also been shown to bias choices toward high effort–big reward option in a binary choice task that traded handgrip force against monetary payoff, and to enhance effort production in a free operant (or incentive motivation) task that involved squeezing a handgrip to win monetary rewards, knowing that payoff would be proportional to force at peak (Le Bouc et al., 2016). Thus, dopaminergic enhancers have yielded consistent effects on effort expenditure for rewards across a variety of experimental paradigms, in line with what was observed in rodents (Salamone et al., 2007). Similar effects have also been obtained with high-frequency stimulation of the subthalamic nucleus in Parkinson’s disease patients (Palminteri et al., 2013; Zenon et al., 2016b). Computational modelling showed that the energizing effect of dopamine enhancers was best captured by an increased sensitivity to reward magnitude, and not a decreased sensitivity to effort cost (Le Bouc et al., 2016). This computational effect of dopaminergic medication could account for both the enhanced propensity to select high effort–big reward option in the choice task and the enhanced force production in the motivation task. Thus, the same computational mechanism could explain the orientation (action selection) and the intensity (action vigour) of the behaviour. Dopaminergic medication also had an effect on the motor activation rate, a computational parameter used in optimal control theory that serves to control the speed with which force is produced, irrespective of reward level. Crucially, this motor effect was independent (uncorrelated across patients) from the motivational effect on reward magnitude. Moreover, medication effects on motor and motivational parameters were correlated with improvement of motor symptoms and apathy, respectively. Such dissociation shows that computational analysis can help disentangle motor and motivational disorders in conditions where it is not obvious to tell whether the patient cannot or simply does not want to produce a behaviour. This result is compatible with other model-based studies of Parkinson’s disease patients’ behaviour, which also pointed to decreased reward sensitivity, irrespective of costs (Manohar et al., 2015; Zenon et al., 2016a). We note that dopamine receptor agonists may not only improve apathy but also induce impulse control disorders (ICDs) in a significant proportion of Parkinson’s disease patients (∼17% according to Voon et al., 2017). ICDs can include gambling disorder, binge eating, compulsive shopping, compulsive sexual behaviours and compulsive medication use (dopamine dysregulation syndrome). In decision-making tasks, ICDs have been associated with enhanced sensitivity to delay, resulting in more impulsive choices (Housden et al., 2010; Voon et al., 2010b; Joutsa et al., 2015), or with higher propensity to make risky choices (Djamshidian et al., 2010; Voon et al., 2011). However, ICDs could also be explained by dysfunction of learning mechanisms, such as an imbalance between positive and negative reinforcement (Voon et al., 2010a; Piray et al., 2014). As learning models go beyond our scope, we refer readers to a recent review of putative mechanisms underlying ICDs (Voon et al., 2017). Another case where apathy has been explored using reward/effort paradigms is the so-called autoactivation deficit. This syndrome is characterized by a severe form of apathy following bilateral lesions in the striato-pallidal complex (Laplane et al., 1989). Patients present a complete lack of self-initiated behaviour that contrasts with the preserved motor and cognitive abilities that they can exhibit when solicited by another person (Laplane and Dubois, 2001). These patients have been explored in the first study that assessed reward/effort trade-off in neurological conditions (Schmidt et al., 2008). In the incentive motivation task, patients were free to produce any handgrip force knowing that (fictive) payoff would be in proportion, whereas in the instructed control task, patients were explicitly told how much to squeeze the handgrip. Patients showed a preserved ability to modulate their force according to the instructions, a preserved autonomic (skin conductance) response to incentive cues (coin images), but an inability to modulate their force according to monetary incentives (Schmidt et al., 2008). These results suggested that striato-pallidal lesions do not impair motor control, nor do they impair the affective evaluation of monetary incentive, but rather the process that translates potential incentives into motor activation (Schmidt et al., 2008). Interestingly, patients still produced a force in the incentive motivation task, suggesting that only the financial part of the benefit term was affected in the net value function. Further computational analyses might help specify the motivational dysfunction that characterizes the autoactivation deficit. It has been shown that dopaminergic medication could help restore motivational function in these patients (Adam et al., 2013), which might indicate that apathy in autoactivation deficit is qualitatively similar, but quantitatively more severe, than apathy in Parkinson’s disease. Finally, apathy has been described following lesions of the medial wall, which can include the ACC and vmPFC. Unfortunately, few studies have investigated the reward/effort trade-off in these patients. It has been reported that ACC lesions attenuate the conscious feeling of mental effort (Naccache et al., 2005), without impairing cognitive control performance. This is consistent with the report that electrical stimulation of ACC induces the subjective impression of an imminent challenge and the determined intention to cope with it (Parvizi et al., 2013). Examination of a large cohort of patients showed that frontal medial damage was associated with a reduced effect of reward on invigorating saccade velocity (Manohar and Husain, 2016). Somewhat surprisingly, this study reported an increased sensitivity to reward following specific (subgenual) vmPFC lesions, and a trend in the opposite direction following ventral striatum lesions. Further studies using proper physical or mental effort would be needed to precise the effect of such lesions on the net value function. Psychiatry Disorders of motivation are frequently observed in psychiatric diseases. Even when they are not the most apparent symptoms, they may interfere with cognitive abilities and impede functional outcomes. Indeed, two of the most frequent and devastating psychiatric diseases, schizophrenia and depression, include lack of motivation in their definition. According to the DSM-5, one of the two main negative symptoms in schizophrenia is avolition—a decrease in motivated self-initiated purposeful activities (American Psychiatric Association, 2013). Similarly, a diagnosis of major depressive disorder requires either depressed mood or markedly diminished interest or pleasure in all, or almost all, activities most of the day, nearly every day. Conversely, one of the criteria of manic/hypomanic episodes is an increase in goal-directed activity (even if psychomotor agitation, i.e. purposeless non-goal-directed activity, is most often observed as severity increases). Beyond motivation deficit and excessive motivation, several psychiatric diseases are partly defined by atypical interest (e.g. autism spectrum disorders) or deviant motivation (e.g. addictive disorders). In this section, we will mainly focus on schizophrenia and depression, where reward/effort-based decision-making was particularly investigated in the recent years. From an historical point of view, the very first descriptions of schizophrenia by Kraepelin and Bleuler already emphasized the motivation deficit, which has been considered a cardinal and crucial component of negative symptoms (Kirkpatrick et al., 2006; Foussias and Remington, 2010). However, negative symptoms, which are by definition less spectacular and de facto less responsive to antipsychotics than delusions or hallucinations, were largely understudied until quite recently. Nevertheless, recent research put forward the critical importance of avolition, which has been shown to be one of the best predictors of functional outcome (Bowie et al., 2006; Foussias et al., 2011; Konstantakopoulos et al., 2011; Evensen et al., 2012; Fervaha et al., 2013a, 2014a, b), not only at the chronic stage of the disease but also during first episodes (Faerden et al., 2009, 2010; Chang et al., 2016) or even in ultra-high risk patients (Lam et al., 2015). Moreover, the lack of motivation could partly account for other dimensions of the disease, such as cognitive dysfunction, which are usually evaluated through neuropsychological tests that require participants to be motivated for a correct performance (Fervaha et al., 2014c). Regarding behavioural tests of motivation, many studies revealed a reward processing deficit using desirability/pleasantness ratings, intertemporal choices or reinforcement learning tasks (see Gold et al., 2008 for a review). Effort was introduced lately in a seminal study showing that patients with schizophrenia are less likely to favour high effort, especially when paired with largest and most certain monetary rewards (Gold et al., 2013). This shift in the reward/effort trade-off has been replicated by several studies (Fervaha et al., 2013c; Barch et al., 2014; Hartmann et al., 2015; Treadway et al., 2015; Wang et al., 2015; McCarthy et al., 2016; Strauss et al., 2016), across various types of tasks (binary choice, willingness to accept) and efforts (key pressing, handgrip squeezing), with only one negative result (Docx et al., 2015). It has also been extended to cognitive (rather than motor) effort (Wolf et al., 2014; Culbreth et al., 2016), again with one negative finding (Gold et al., 2015a). Furthermore, most studies found a correlation between the willingness to produce efforts and the severity of negative symptoms assessed with standard clinical scales (Gold et al., 2015b). Interestingly, a recent study compared five different reward/effort trade-off tasks in a large sample of 94 patients with schizophrenia, with a 4-week retest (Reddy et al., 2015). All tasks, with the notable exception of the perceptual categorization task, discriminated between patients and controls, with good enough reliability, but relatively modest correlation with negative symptoms (Horan et al., 2015). Surprisingly, no correlation was found with treatment, although most medications used in schizophrenia target dopaminergic transmission. Indeed, other authors found a strong effect of drug type (particularly conventional antipsychotics) on the willingness to expend effort, but argued that different drugs generally mean different patients, and claimed for randomized trials (Gold et al., 2015b). Critically, and despite the growing amount of evidence pointing toward an altered reward/effort trade-off in schizophrenia (Fervaha et al., 2013b; Strauss et al., 2014; Green and Horan, 2015; Green et al., 2015), no existing study attempted to discriminate between decreased sensitivity to reward and increased sensitivity to effort, using computational modelling tools. The only step in this direction was taken in a study reporting that patients with schizophrenia produced the same amount of effort as controls on average, but that effort expenditure was less driven by reward, such that choices were suboptimal in terms of maximizing outcomes (Treadway et al., 2015). More quantitative analyses would be required to disclose the potential distortions of the net value function in schizophrenia, as well as the impact of treatments such as antidopaminergic medication. As in schizophrenia, motivation deficit is a frequent and burdensome symptom of depression, which is highly predictive of functional impairment and subjective wellbeing (Calabrese et al., 2014; Fervaha et al., 2016). Moreover, as for avolition in schizophrenia, motivation deficit is less responsive to conventional treatment, such as serotoninergic antidepressant in unipolar depression or mood stabilizer in bipolar disorder, than other dimensions of the disease (Calabrese et al., 2014). Over the past decade, most behavioural investigations focused on reward processing versus punishment processing (Eshel and Roiser, 2010; Huys et al., 2013; Admon and Pizzagalli, 2015; Chen et al., 2015). Typical results in depressed patients are excessive processing of negative stimuli, hyposensitivity to positive outcomes with decreased pleasurable experience, and blunted response to reward in the ventral striatum. Effort was first introduced with an incentive motivation test using a handgrip device (Cléry-Melin et al., 2011), which established that contrary to healthy controls, depressed patients would not produce more effort when more money is at stake. A follow-up study showed that a normal pattern of effort production in response to monetary incentives was restored after remission from depressive episode (Mauras et al., 2016). The diminished sensitivity of effort production to incentives has been replicated many times, using different tasks (binary choice, willingness to accept) and efforts (key pressing, handgrip squeezing, cognitive effort) (Sherdell et al., 2012; Treadway et al., 2012; Yang et al., 2014; Hershenberg et al., 2016). While the shift in the reward/effort trade-off was correlated with Beck Depression Inventory (BDI) score in healthy subjects (Treadway et al., 2009), and was found more pronounced in actual depression than in subsyndromal depression, two studies failed to find a correlation with global BDI score in depressed patients (Yang et al., 2014; Hershenberg et al., 2016). Thus, abnormal effort allocation might represent a specific dimension of depression rather than a consequence of depression severity. Indeed, the willingness to produce effort for a given reward was correlated with lack of motivation, as measured by apathy scales, in both healthy participants and depressed patients (Bonnelle et al., 2015; Mauras et al., 2016). The link between anhedonia and reward/effort-based decision-making is less clear and still a matter of debate (Treadway and Zald, 2011; Der-Avakian and Markou, 2012; Rizvi et al., 2016). Regarding diagnostic, only one study compared unipolar and bipolar depressed patients, and found no difference between these two populations, nor between the medication classes used to treat them (Hershenberg et al., 2016). A decreased reward/effort ratio was also found in drug-naïve patients (Yang et al., 2014), which is an important control since serotoninergic medication could impact reward or effort processing. Again, while consistent evidence was found for an alteration of reward/effort-based decision-making in depression, no study could dissociate between the two candidate underlying mechanisms, namely increased sensitivity to effort or decreased sensitivity to reward. Thus, the question remains to what extent these apparently similar reward/effort trade-off alterations in schizophrenia and depression are indeed supported by the same cognitive distortion and neurobiological pathophysiology. Importantly, decreased reward/effort ratio is not an unspecific feature that would be observed in any psychiatric disease. Indeed this ratio was reported to be unaltered for instance in pathological gamblers (Fauth-Buhler et al., 2014), or even increased in autism spectrum disorder (Damiano et al., 2012). Unfortunately, reward/effort-based decision-making has not been explored in patients for whom an excessive motivation would be predicted from the clinical presentation, such as during hypomanic episodes of bipolar disorder. Conclusions and perspectives Compared to psychometric scoring that assesses motivation deficit as a whole, computational phenotyping based on behavioural tests is a promising avenue for clinical neuroscience. Modelling the behaviour as generated from a value function (borrowed from decision theory) enables a normative decomposition of motivation into subcomponents that may possess anatomo-functional specificity at the neural level. For instance, the benefit term has been associated with cortical regions such as the vmPFC, whereas other regions like the dACC would integrate the costs. Also, the weights of costs and benefits might be determined by different neuromodulators such as dopamine, noradrenaline and serotonin. Thus, computational phenotyping may help discriminate between distinct mechanisms underlying motivation deficits, such as hyposensitivity to benefits and hypersensitivity to costs, or abnormal susceptibility to fatigue. It may also help dissociate motivation deficits from motor disorders, which seems crucial in a number of neuro-psychiatric diseases. An overarching objective for computational neuropsychiatry would be to build a quantitative model of motivation that integrates the neural and algorithmic levels. From virtual lesions of this grand model, one could derive predictions about specific behavioural patterns that should be observed in patients. Reciprocally, by fitting the model to observed patients’ behaviour, one could make inferences about the underlying pathophysiology. The hope here is that computational approach would (i) provide unprecedented insight into the disease; and (ii) improve on differential diagnostic and therapeutic orientation (e.g. assuming that hyposensitivity to reward should not require the same treatment as hypersensitivity to effort). There are nonetheless limitations to this approach. First, the estimation of computational parameters may depend upon the paradigms used to probe the behaviour. Instead of faithfully representing the patient’s state, parameters might be susceptible to confounding details of the tasks. A related issue is that quantification of motivation deficit may depend upon the particular goals suggested in behavioural tests. Goal values are subjective and assigning a low value to one particular goal is not necessarily pathological, as long as other goals receive high values. This may be problematic because most tasks target a reduced set of interests, typically motivation for performance or payoff (the goal is to accumulate points or money). Yet patients may not be strongly motivated by that sort of goal but still driven by other goals, such as making their relatives happy. Note that decision theory is only useful to capture disorders characterized by abnormal motivation intensity, not those defined by abnormal motivation contents (e.g. atypical interests in autism spectrum disorder). Finally, because all behavioural tasks suggest goals (as in ‘would you make an effort to get this reward?’), they might miss motivation deficits in which the generation of goals (and not goal valuation) is precisely the problem. One could envisage the case of a patient who would like to engage in some activity but the idea of doing this activity would simply not come to mind. Other types of tests would certainly be required in order to detect this type of motivation deficit. Funding No funding was received towards this work. Abbreviations Abbreviations dACC dorsal anterior cingulate cortex vmPFC ventromedial prefrontal cortex References Aarsland D, Larsen JP, Lim NG, Janvin C, Karlsen K, Tandberg E, et al.   Range of neuropsychiatric disturbances in patients with Parkinson's disease. J Neurol Neurosurg Psychiatry  1999; 67: 492– 6. Google Scholar CrossRef Search ADS PubMed  Abitbol R, Lebreton M, Hollard G, Richmond BJ, Bouret S, Pessiglione M. Neural mechanisms underlying contextual dependency of subjective values: converging evidence from monkeys and humans. J Neurosci  2015; 35: 2308– 20. Google Scholar CrossRef Search ADS PubMed  Adam R, Leff A, Sinha N, Turner C, Bays P, Draganski B, et al.   Dopamine reverses reward insensitivity in apathy following globus pallidus lesions. Cortex  2013; 49: 1292– 303. Google Scholar CrossRef Search ADS PubMed  Admon R, Pizzagalli DA. Dysfunctional reward processing in depression. Curr Opin Psychol  2015; 4: 114– 18. Google Scholar CrossRef Search ADS PubMed  Alnaes D, Sneve MH, Espeseth T, Endestad T, van de Pavert SH, Laeng B. Pupil size signals mental effort deployed during multiple object tracking and predicts brain activity in the dorsal attention network and the locus coeruleus. J Vis  2014; 14: 1– 20. Google Scholar CrossRef Search ADS PubMed  American Psychiatric Association. Diagnostic and statistical manual of mental disorders . 5th edn. Arlington, VA: American Psychiatric Publishing; 2013. Apps MA, Grima LL, Manohar S, Husain M. The role of cognitive effort in subjective reward devaluation and risky decision-making. Sci Rep  2015; 5: 16880. Google Scholar CrossRef Search ADS PubMed  Arnould A, Rochat L, Azouvi P, Van der Linden M. A multidimensional approach to apathy after traumatic brain injury. Neuropsychol Rev  2013; 23: 210– 33. Google Scholar CrossRef Search ADS PubMed  Barch DM, Treadway MT, Schoen N. Effort, anhedonia, and function in schizophrenia: reduced effort allocation predicts amotivation and functional impairment. J Abnorm Psychol  2014; 123: 387– 97. Google Scholar CrossRef Search ADS PubMed  Bartra O, McGuire JT, Kable JW. The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. Neuroimage  2013; 76: 412– 27. Google Scholar CrossRef Search ADS PubMed  Beatty J, Wagoner BL. Pupillometric signs of brain activation vary with level of cognitive processing. Science  1978; 199: 1216– 18. Google Scholar CrossRef Search ADS PubMed  Bickel WK, Pitcock JA, Yi R, Angtuaco EJ. Congruence of BOLD response across intertemporal choice conditions: fictive and real money gains and losses. J Neurosci  2009; 29: 8839– 46. Google Scholar CrossRef Search ADS PubMed  Bonnelle V, Veromann KR, Burnett Heyes S, Lo Sterzo E, Manohar S, Husain M. Characterization of reward and effort mechanisms in apathy. J Physiol Paris  2015; 109: 16– 26. Google Scholar CrossRef Search ADS PubMed  Botvinick M, Braver T. Motivation and cognitive control: from behavior to neural mechanism. Annu Rev Psychol  2015; 66: 83– 113. Google Scholar CrossRef Search ADS PubMed  Botvinick MM, Huffstetler S, McGuire JT. Effort discounting in human nucleus accumbens. Cogn Affect Behav Neurosci  2009; 9: 16– 27. Google Scholar CrossRef Search ADS PubMed  Boureau YL, Sokol-Hessner P, Daw ND. Deciding how to decide: self-control and meta-decision making. Trends Cogn Sci  2015; 19: 700– 10. Google Scholar CrossRef Search ADS PubMed  Bouret S, Richmond BJ. Ventromedial and orbital prefrontal neurons differentially encode internally and externally driven motivational values in monkeys. J Neurosci  2010; 30: 8591– 601. Google Scholar CrossRef Search ADS PubMed  Bouret S, Richmond BJ. Sensitivity of locus ceruleus neurons to reward value for goal-directed actions. J Neurosci  2015; 35: 4005– 14. Google Scholar CrossRef Search ADS PubMed  Bowie CR, Reichenberg A, Patterson TL, Heaton RK, Harvey PD. Determinants of real-world functional performance in schizophrenia subjects: correlations with cognition, functional capacity, and symptoms. Am J Psychiatry  2006; 163: 418– 25. Google Scholar CrossRef Search ADS PubMed  Brown CA, Campbell MC, Karimi M, Tabbal SD, Loftin SK, Tian LL, et al.   Dopamine pathway loss in nucleus accumbens and ventral tegmental area predicts apathetic behavior in MPTP-lesioned monkeys. Exp Neurol  2012; 236: 190– 7. Google Scholar CrossRef Search ADS PubMed  Brown RG, Pluck G. Negative symptoms: the ‘pathology' of motivation and goal-directed behaviour. Trends Neurosci  2000; 23: 412– 17. Google Scholar CrossRef Search ADS PubMed  Burke CJ, Brunger C, Kahnt T, Park SQ, Tobler PN. Neural integration of risk and effort costs by the frontal pole: only upon request. J Neurosci  2013; 33: 1706– 13a. Google Scholar CrossRef Search ADS PubMed  Caeiro L, Ferro JM, Costa J. Apathy secondary to stroke: a systematic review and meta-analysis. Cerebrovasc Dis  2013; 35: 23– 39. Google Scholar CrossRef Search ADS PubMed  Calabrese JR, Fava M, Garibaldi G, Grunze H, Krystal AD, Laughren T, et al.   Methodological approaches and magnitude of the clinical unmet need associated with amotivation in mood disorders. J Affect Disord  2014; 168: 439– 51. Google Scholar CrossRef Search ADS PubMed  Chang WC, Hui CL, Chan SK, Lee EH, Chen EY. Impact of avolition and cognitive impairment on functional outcome in first-episode schizophrenia-spectrum disorder: a prospective one-year follow-up study. Schizophr Res  2016; 170: 318– 21. Google Scholar CrossRef Search ADS PubMed  Chen C, Takahashi T, Nakagawa S, Inoue T, Kusumi I. Reinforcement learning in depression: a review of computational research. Neurosci Biobehav Rev  2015; 55: 247– 67. Google Scholar CrossRef Search ADS PubMed  Chong TT, Bonnelle V, Husain M. Quantifying motivation with effort-based decision-making paradigms in health and disease. Prog Brain Res  2016; 229: 71– 100. Google Scholar CrossRef Search ADS PubMed  Chong TT, Bonnelle V, Manohar S, Veromann KR, Muhammed K, Tofaris GK, et al.   Dopamine enhances willingness to exert effort for reward in Parkinson's disease. Cortex  2015; 69: 40– 6. Google Scholar CrossRef Search ADS PubMed  Cléry-Melin ML, Schmidt L, Lafargue G, Baup N, Fossati P, Pessiglione M. Why don't you try harder? An investigation of effort production in major depression. PLoS One  2011; 6: e23178. Google Scholar CrossRef Search ADS PubMed  Clithero JA, Rangel A. Informatic parcellation of the network involved in the computation of subjective value. Soc Cogn Affect Neurosci  2014; 9: 1289– 302. Google Scholar CrossRef Search ADS PubMed  Critchley HD, Mathias CJ, Dolan RJ. Neural activity in the human brain relating to uncertainty and arousal during anticipation. Neuron  2001; 29: 537– 45. Google Scholar CrossRef Search ADS PubMed  Croxson PL, Walton ME, O'Reilly JX, Behrens TE, Rushworth MF. Effort-based cost-benefit valuation and the human brain. J Neurosci  2009; 29: 4531– 41. Google Scholar CrossRef Search ADS PubMed  Culbreth A, Westbrook A, Barch D. Negative symptoms are associated with an increased subjective cost of cognitive effort. J Abnorm Psychol  2016; 125: 528– 36. Google Scholar CrossRef Search ADS PubMed  Damiano CR, Aloi J, Treadway M, Bodfish JW, Dichter GS. Adults with autism spectrum disorders exhibit decreased sensitivity to reward parameters when making effort-based decisions. J Neurodev Disord  2012; 4: 13. Google Scholar CrossRef Search ADS PubMed  Delgado MR, Labouliere CD, Phelps EA. Fear of losing money? Aversive conditioning with secondary reinforcers. Soc Cogn Affect Neurosci  2006; 1: 250– 9. Google Scholar CrossRef Search ADS PubMed  Der-Avakian A, Markou A. The neurobiology of anhedonia and other reward-related deficits. Trends Neurosci  2012; 35: 68– 77. Google Scholar CrossRef Search ADS PubMed  Djamshidian A, Jha A, O'Sullivan SS, Silveira-Moriyama L, Jacobson C, Brown P, et al.   Risk and learning in impulsive and nonimpulsive patients with Parkinson's disease. Mov Disord  2010; 25: 2203– 10. Google Scholar CrossRef Search ADS PubMed  Docx L, de la Asuncion J, Sabbe B, Hoste L, Baeten R, Warnaerts N, et al.   Effort discounting and its association with negative symptoms in schizophrenia. Cogn Neuropsychiatry  2015; 20: 172– 85. Google Scholar CrossRef Search ADS PubMed  Drijgers RL, Dujardin K, Reijnders JS, Defebvre L, Leentjens AF. Validation of diagnostic criteria for apathy in Parkinson's disease. Parkinsonism Relat Disord  2010; 16: 656– 60. Google Scholar CrossRef Search ADS PubMed  Eshel N, Roiser JP. Reward and punishment processing in depression. Biol Psychiatry  2010; 68: 118– 24. Google Scholar CrossRef Search ADS PubMed  Evensen J, Rossberg JI, Barder H, Haahr U, Hegelstad W, Joa I, et al.   Apathy in first episode psychosis patients: a ten year longitudinal follow-up study. Schizophr Res  2012; 136: 19– 24. Google Scholar CrossRef Search ADS PubMed  Faerden A, Finset A, Friis S, Agartz I, Barrett EA, Nesvag R, et al.   Apathy in first episode psychosis patients: one year follow up. Schizophr Res  2010; 116: 20– 6. Google Scholar CrossRef Search ADS PubMed  Faerden A, Friis S, Agartz I, Barrett EA, Nesvag R, Finset A, et al.   Apathy and functioning in first-episode psychosis. Psychiatr Serv  2009; 60: 1495– 503. Google Scholar CrossRef Search ADS PubMed  Fauth-Buhler M, Zois E, Vollstadt-Klein S, Lemenager T, Beutel M, Mann K. Insula and striatum activity in effort-related monetary reward processing in gambling disorder: the role of depressive symptomatology. Neuroimage Clin  2014; 6: 243– 51. Google Scholar CrossRef Search ADS PubMed  Fervaha G, Foussias G, Agid O, Remington G. Amotivation and functional outcomes in early schizophrenia. Psychiatry Res  2013a; 210: 665– 8. Google Scholar CrossRef Search ADS   Fervaha G, Foussias G, Agid O, Remington G. Neural substrates underlying effort computation in schizophrenia. Neurosci Biobehav Rev  2013b; 37 ( 10 Pt 2): 2649– 65. Google Scholar CrossRef Search ADS   Fervaha G, Foussias G, Agid O, Remington G. Impact of primary negative symptoms on functional outcomes in schizophrenia. Eur Psychiatry  2014a; 29: 449– 55. Google Scholar CrossRef Search ADS   Fervaha G, Foussias G, Agid O, Remington G. Motivational and neurocognitive deficits are central to the prediction of longitudinal functional outcome in schizophrenia. Acta Psychiatr Scand  2014b; 130: 290– 9. Google Scholar CrossRef Search ADS   Fervaha G, Foussias G, Takeuchi H, Agid O, Remington G. Motivational deficits in major depressive disorder: cross-sectional and longitudinal relationships with functional impairment and subjective well-being. Compr Psychiatry  2016; 66: 31– 8. Google Scholar CrossRef Search ADS PubMed  Fervaha G, Graff-Guerrero A, Zakzanis KK, Foussias G, Agid O, Remington G. Incentive motivation deficits in schizophrenia reflect effort computation impairments during cost-benefit decision-making. J Psychiatr Res  2013c; 47: 1590– 6. Google Scholar CrossRef Search ADS   Fervaha G, Zakzanis KK, Foussias G, Graff-Guerrero A, Agid O, Remington G. Motivational deficits and cognitive test performance in schizophrenia. JAMA Psychiatry  2014c; 71: 1058– 65. Google Scholar CrossRef Search ADS   Fonseca MS, Murakami M, Mainen ZF. Activation of dorsal raphe serotonergic neurons promotes waiting but is not reinforcing. Curr Biol  2015; 25: 306– 15. Google Scholar CrossRef Search ADS PubMed  Forbes GB, Brown MR, Griffiths HJ. Arm muscle plus bone area: anthropometry and CAT scan compared. Am J Clin Nutr  1988; 47: 929– 31. Google Scholar CrossRef Search ADS PubMed  Foussias G, Mann S, Zakzanis KK, van Reekum R, Agid O, Remington G. Prediction of longitudinal functional outcomes in schizophrenia: the impact of baseline motivational deficits. Schizophr Res  2011; 132: 24– 7. Google Scholar CrossRef Search ADS PubMed  Foussias G, Remington G. Negative symptoms in schizophrenia: avolition and Occam's razor. Schizophr Bull  2010; 36: 359– 69. Google Scholar CrossRef Search ADS PubMed  Frederick S, Loewenstein G, O'Donoghue T. Time discounting and time preference: a critical review. J Econ Lit  2002; 40: 351– 401. Google Scholar CrossRef Search ADS   Gan JO, Walton ME, Phillips PE. Dissociable cost and benefit encoding of future rewards by mesolimbic dopamine. Nat Neurosci  2010; 13: 25– 7. Google Scholar CrossRef Search ADS PubMed  Gergelyfi M, Jacob B, Olivier E, Zenon A. Dissociation between mental fatigue and motivational state during prolonged mental activity. Front Behav Neurosci  2015; 9: 176. Google Scholar CrossRef Search ADS PubMed  Gold JM, Kool W, Botvinick MM, Hubzin L, August S, Waltz JA. Cognitive effort avoidance and detection in people with schizophrenia. Cogn Affect Behav Neurosci  2015a; 15: 145– 54. Google Scholar CrossRef Search ADS   Gold JM, Strauss GP, Waltz JA, Robinson BM, Brown JK, Frank MJ. Negative symptoms of schizophrenia are associated with abnormal effort-cost computations. Biol Psychiatry  2013; 74: 130– 6. Google Scholar CrossRef Search ADS PubMed  Gold JM, Waltz JA, Frank MJ. Effort cost computation in schizophrenia: a commentary on the recent literature. Biol Psychiatry  2015b; 78: 747– 53. Google Scholar CrossRef Search ADS   Gold JM, Waltz JA, Prentice KJ, Morris SE, Heerey EA. Reward processing in schizophrenia: a deficit in the representation of value. Schizophr Bull  2008; 34: 835– 47. Google Scholar CrossRef Search ADS PubMed  Green L, Myerson J. A discounting framework for choice with delayed and probabilistic rewards. Psychol Bull  2004; 130: 769– 92. Google Scholar CrossRef Search ADS PubMed  Green MF, Horan WP. Effort-based decision making in schizophrenia: evaluation of paradigms to measure motivational deficits. Schizophr Bull  2015; 41: 1021– 3. Google Scholar CrossRef Search ADS PubMed  Green MF, Horan WP, Barch DM, Gold JM. Effort-based decision making: a novel approach for assessing motivation in schizophrenia. Schizophr Bull  2015; 41: 1035– 44. Google Scholar CrossRef Search ADS PubMed  Haber SN. The primate basal ganglia: parallel and integrative networks. J Chem Neuroanat  2003; 26: 317– 30. Google Scholar CrossRef Search ADS PubMed  Hama S, Yamashita H, Shigenobu M, Watanabe A, Hiramoto K, Kurisu K, et al.   Depression or apathy and functional recovery after stroke. Int J Geriatr Psychiatry  2007; 22: 1046– 51. Google Scholar CrossRef Search ADS PubMed  Hamid AA, Pettibone JR, Mabrouk OS, Hetrick VL, Schmidt R, Vander Weele CM, et al.   Mesolimbic dopamine signals the value of work. Nat Neurosci  2016; 19: 117– 26. Google Scholar CrossRef Search ADS PubMed  Hartmann MN, Hager OM, Reimann AV, Chumbley JR, Kirschner M, Seifritz E, et al.   Apathy but not diminished expression in schizophrenia is associated with discounting of monetary rewards by physical effort. Schizophr Bull  2015; 41: 503– 12. Google Scholar CrossRef Search ADS PubMed  Hartmann MN, Hager OM, Tobler PN, Kaiser S. Parabolic discounting of monetary rewards by physical effort. Behav Processes  2013; 100: 192– 6. Google Scholar CrossRef Search ADS PubMed  Hershenberg R, Satterthwaite TD, Daldal A, Katchmar N, Moore TM, Kable JW, et al.   Diminished effort on a progressive ratio task in both unipolar and bipolar depression. J Affect Disord  2016; 196: 97– 100. Google Scholar CrossRef Search ADS PubMed  Hollon NG, Arnold MM, Gan JO, Walton ME, Phillips PE. Dopamine-associated cached values are not sufficient as the basis for action selection. Proc Natl Acad Sci USA  2014; 111: 18357– 62. Google Scholar CrossRef Search ADS PubMed  Horan WP, Reddy LF, Barch DM, Buchanan RW, Dunayevich E, Gold JM, et al.   Effort-based decision-making paradigms for clinical trials in schizophrenia: part 2-external validity and correlates. Schizophr Bull  2015; 41: 1055– 65. Google Scholar CrossRef Search ADS PubMed  Housden CR, O'Sullivan SS, Joyce EM, Lees AJ, Roiser JP. Intact reward learning but elevated delay discounting in Parkinson's disease patients with impulsive-compulsive spectrum behaviors. Neuropsychopharmacology  2010; 35: 2155– 64. Google Scholar CrossRef Search ADS PubMed  Hsu ES, Patwardhan AG, Meade KP, Light TR, Martin WR. Cross-sectional geometrical properties and bone mineral contents of the human radius and ulna. J Biomech  1993; 26: 1307– 18. Google Scholar CrossRef Search ADS PubMed  Hull CL. Principles of behavior, an introduction to behavior theory . New York, NY; London: Appleton-Century Company; 1943. Huys QJ, Pizzagalli DA, Bogdan R, Dayan P. Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis. Biol Mood Anxiety Disord  2013; 3: 12. Google Scholar CrossRef Search ADS PubMed  Inzlicht M, Schmeichel BJ, Macrae CN. Why self-control seems (but may not be) limited. Trends Cogn Sci  2014; 18: 127– 33. Google Scholar CrossRef Search ADS PubMed  Joutsa J, Voon V, Johansson J, Niemela S, Bergman J, Kaasinen V. Dopaminergic function and intertemporal choice. Transl Psychiatry  2015; 5: e491. Google Scholar CrossRef Search ADS PubMed  Kahneman D, Wright P. Changes of pupil size and rehearsal strategies in a short-term memory task. Q J Exp Psychol  1971; 23: 187– 96. Google Scholar CrossRef Search ADS PubMed  Kennerley SW, Dahmubed AF, Lara AH, Wallis JD. Neurons in the frontal lobe encode the value of multiple decision variables. J Cogn Neurosci  2009; 21: 1162– 78. Google Scholar CrossRef Search ADS PubMed  Kirkpatrick B, Fenton WS, Carpenter WTJr, Marder SR. The NIMH-MATRICS consensus statement on negative symptoms. Schizophr Bull  2006; 32: 214– 19. Google Scholar CrossRef Search ADS PubMed  Klein-Flugge MC, Kennerley SW, Saraiva AC, Penny WD, Bestmann S. Behavioral modeling of human choices reveals dissociable effects of physical effort and temporal delay on reward devaluation. PLoS Comput Biol  2015; 11: e1004116. Google Scholar CrossRef Search ADS PubMed  Knutson B, Fong GW, Adams CM, Varner JL, Hommer D. Dissociation of reward anticipation and outcome with event-related fMRI. Neuroreport  2001; 12: 3683– 7. Google Scholar CrossRef Search ADS PubMed  Kobayashi S, Schultz W. Influence of reward delays on responses of dopamine neurons. J Neurosci  2008; 28: 7837– 46. Google Scholar CrossRef Search ADS PubMed  Konstantakopoulos G, Ploumpidis D, Oulis P, Patrikelis P, Soumani A, Papadimitriou GN, et al.   Apathy, cognitive deficits and functional impairment in schizophrenia. Schizophr Res  2011; 133: 193– 8. Google Scholar CrossRef Search ADS PubMed  Kool W, McGuire JT, Rosen ZB, Botvinick MM. Decision making and the avoidance of cognitive demand. J Exp Psychol Gen  2010; 139: 665– 82. Google Scholar CrossRef Search ADS PubMed  Kroemer NB, Guevara A, Ciocanea Teodorescu I, Wuttig F, Kobiella A, Smolka MN. Balancing reward and work: anticipatory brain activation in NAcc and VTA predict effort differentially. Neuroimage  2014; 102 Pt 2: 510– 19. Google Scholar CrossRef Search ADS PubMed  Kurniawan IT, Guitart-Masip M, Dayan P, Dolan RJ. Effort and valuation in the brain: the effects of anticipation and execution. J Neurosci  2013; 33: 6160– 9. Google Scholar CrossRef Search ADS PubMed  Kurniawan IT, Seymour B, Talmi D, Yoshida W, Chater N, Dolan RJ. Choosing to make an effort: the role of striatum in signaling physical effort of a chosen action. J Neurophysiol  2010; 104: 313– 21. Google Scholar CrossRef Search ADS PubMed  Kurzban R, Duckworth A, Kable JW, Myers J. An opportunity cost model of subjective effort and task performance. Behav Brain Sci  2013; 36: 661– 79. Google Scholar CrossRef Search ADS PubMed  LaBar KS, Gatenby JC, Gore JC, LeDoux JE, Phelps EA. Human amygdala activation during conditioned fear acquisition and extinction: a mixed-trial fMRI study. Neuron  1998; 20: 937– 45. Google Scholar CrossRef Search ADS PubMed  Lak A, Stauffer WR, Schultz W. Dopamine prediction error responses integrate subjective value from different reward dimensions. Proc Natl Acad Sci USA  2014; 111: 2343– 8. Google Scholar CrossRef Search ADS PubMed  Lam M, Abdul Rashid NA, Lee SA, Lim J, Foussias G, Fervaha G, et al.   Baseline social amotivation predicts 1-year functioning in UHR subjects: a validation and prospective investigation. Eur Neuropsychopharmacol  2015; 25: 2187– 96. Google Scholar CrossRef Search ADS PubMed  Landes AM, Sperry SD, Strauss ME. Prevalence of apathy, dysphoria, and depression in relation to dementia severity in Alzheimer's disease. J Neuropsychiatry Clin Neurosci  2005; 17: 342– 9. Google Scholar CrossRef Search ADS PubMed  Laplane D, Dubois B. Auto-activation deficit: a basal ganglia related syndrome. Mov Disord  2001; 16: 810– 14. Google Scholar CrossRef Search ADS PubMed  Laplane D, Levasseur M, Pillon B, Dubois B, Baulac M, Mazoyer B, et al.   Obsessive-compulsive and other behavioural changes with bilateral basal ganglia lesions. A neuropsychological, magnetic resonance imaging and positron tomography study. Brain  1989; 112 ( Pt 3): 699– 725. Google Scholar CrossRef Search ADS PubMed  Le Bouc R, Pessiglione M. Imaging social motivation: distinct brain mechanisms drive effort production during collaboration versus competition. J Neurosci  2013; 33: 15894– 902. Google Scholar CrossRef Search ADS PubMed  Le Bouc R, Rigoux L, Schmidt L, Degos B, Welter ML, Vidailhet M, et al.   Computational dissection of dopamine motor and motivational functions in humans. J Neurosci  2016; 36: 6623– 33. Google Scholar CrossRef Search ADS PubMed  Levy DJ, Glimcher PW. The root of all value: a neural common currency for choice. Curr Opin Neurobiol  2012; 22: 1027– 38. Google Scholar CrossRef Search ADS PubMed  Levy R, Dubois B. Apathy and the functional anatomy of the prefrontal cortex-basal ganglia circuits. Cereb Cortex  2006; 16: 916– 28. Google Scholar CrossRef Search ADS PubMed  Manohar SG, Chong TT, Apps MA, Batla A, Stamelou M, Jarman PR, et al.   Reward pays the cost of noise reduction in motor and cognitive control. Curr Biol  2015; 25: 1707– 16. Google Scholar CrossRef Search ADS PubMed  Manohar SG, Husain M. Human ventromedial prefrontal lesions alter incentivisation by reward. Cortex  2016; 76: 104– 20. Google Scholar CrossRef Search ADS PubMed  Marin RS. Differential diagnosis and classification of apathy. Am J Psychiatry  1990; 147: 22– 30. Google Scholar CrossRef Search ADS PubMed  Martinez-Horta S, Riba J, de Bobadilla RF, Pagonabarraga J, Pascual-Sedano B, Antonijoan RM, et al.   Apathy in Parkinson's disease: neurophysiological evidence of impaired incentive processing. J Neurosci  2014; 34: 5918– 26. Google Scholar CrossRef Search ADS PubMed  Massar SA, Libedinsky C, Weiyan C, Huettel SA, Chee MW. Separate and overlapping brain areas encode subjective value during delay and effort discounting. Neuroimage  2015; 120: 104– 13. Google Scholar CrossRef Search ADS PubMed  Mauras T, Masson M, Fossati P, Pessiglione M. Incentive sensitivity as a behavioral marker of clinical remission from major depressive episode. J Clin Psychiatry  2016; 77: e697– 703. Google Scholar CrossRef Search ADS PubMed  Mayo NE, Fellows LK, Scott SC, Cameron J, Wood-Dauphinee S. A longitudinal view of apathy and its impact after stroke. Stroke  2009; 40: 3299– 307. Google Scholar CrossRef Search ADS PubMed  McCarthy JM, Treadway MT, Bennett ME, Blanchard JJ. Inefficient effort allocation and negative symptoms in individuals with schizophrenia. Schizophr Res  2016; 170: 278– 84. Google Scholar CrossRef Search ADS PubMed  Meyniel F, Goodwin GM, Deakin JW, Klinge C, MacFadyen C, Milligan H, et al.   A specific role for serotonin in overcoming effort cost. Elife  2016; 5: e17282. Google Scholar CrossRef Search ADS PubMed  Meyniel F, Safra L, Pessiglione M. How the brain decides when to work and when to rest: dissociation of implicit-reactive from explicit-predictive computational processes. PLoS Comput Biol  2014; 10: e1003584. Google Scholar CrossRef Search ADS PubMed  Meyniel F, Sergent C, Rigoux L, Daunizeau J, Pessiglione M. Neurocomputational account of how the human brain decides when to have a break. Proc Natl Acad Sci USA  2013; 110: 2641– 6. Google Scholar CrossRef Search ADS PubMed  Minamimoto T, Hori Y, Richmond BJ. Is working more costly than waiting in monkeys? PLoS One  2012; 7: e48434. Google Scholar CrossRef Search ADS PubMed  Miyazaki K, Miyazaki KW, Doya K. The role of serotonin in the regulation of patience and impulsivity. Mol Neurobiol  2012; 45: 213– 24. Google Scholar CrossRef Search ADS PubMed  Naccache L, Dehaene S, Cohen L, Habert MO, Guichart-Gomez E, Galanaud D, et al.   Effortless control: executive attention and conscious feeling of mental effort are dissociable. Neuropsychologia  2005; 43: 1318– 28. Google Scholar CrossRef Search ADS PubMed  Neu CM, Rauch F, Rittweger J, Manz F, Schoenau E. Influence of puberty on muscle development at the forearm. Am J Physiol Endocrinol Metab  2002; 283: E103– 7. Google Scholar CrossRef Search ADS PubMed  Niyogi RK, Shizgal P, Dayan P. Some work and some play: microscopic and macroscopic approaches to labor and leisure. PLoS Comput Biol  2014; 10: e1003894. Google Scholar CrossRef Search ADS PubMed  O'Doherty JP. Multiple systems for the motivational control of behavior and associated neural substrates in humans. Curr Top Behav Neurosci  2016; 27: 291– 312. Google Scholar CrossRef Search ADS PubMed  Padoa-Schioppa C, Cai X. The orbitofrontal cortex and the computation of subjective value: consolidated concepts and new perspectives. Ann N Y Acad Sci  2011; 1239: 130– 7. Google Scholar CrossRef Search ADS PubMed  Palminteri S, Serra G, Buot A, Schmidt L, Welter ML, Pessiglione M. Hemispheric dissociation of reward processing in humans: insights from deep brain stimulation. Cortex  2013; 49: 2834– 44. Google Scholar CrossRef Search ADS PubMed  Parvizi J, Rangarajan V, Shirer WR, Desai N, Greicius MD. The will to persevere induced by electrical stimulation of the human cingulate gyrus. Neuron  2013; 80: 1359– 67. Google Scholar CrossRef Search ADS PubMed  Pasquereau B, Turner RS. Limited encoding of effort by dopamine neurons in a cost-benefit trade-off task. J Neurosci  2013; 33: 8288– 300. Google Scholar CrossRef Search ADS PubMed  Pessiglione M, Schmidt L, Draganski B, Kalisch R, Lau H, Dolan RJ, et al.   How the brain translates money into force: a neuroimaging study of subliminal motivation. Science  2007; 316: 904– 6. Google Scholar CrossRef Search ADS PubMed  Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD. Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature  2006; 442: 1042– 5. Google Scholar CrossRef Search ADS PubMed  Peters J, Buchel C. Neural representations of subjective reward value. Behav Brain Res  2010; 213: 135– 41. Google Scholar CrossRef Search ADS PubMed  Petrovic P, Kalisch R, Pessiglione M, Singer T, Dolan RJ. Learning affective values for faces is expressed in amygdala and fusiform gyrus. Soc Cogn Affect Neurosci  2008; 3: 109– 18. Google Scholar CrossRef Search ADS PubMed  Piquado T, Isaacowitz D, Wingfield A. Pupillometry as a measure of cognitive effort in younger and older adults. Psychophysiology  2010; 47: 560– 9. Google Scholar CrossRef Search ADS PubMed  Piray P, Zeighami Y, Bahrami F, Eissa AM, Hewedi DH, Moustafa AA. Impulse control disorders in Parkinson's disease are associated with dysfunction in stimulus valuation but not action valuation. J Neurosci  2014; 34: 7814– 24. Google Scholar CrossRef Search ADS PubMed  Porat O, Hassin-Baer S, Cohen OS, Markus A, Tomer R. Asymmetric dopamine loss differentially affects effort to maximize gain or minimize loss. Cortex  2014; 51: 82– 91. Google Scholar CrossRef Search ADS PubMed  Prevost C, Pessiglione M, Metereau E, Clery-Melin ML, Dreher JC. Separate valuation subsystems for delay and effort decision costs. J Neurosci  2010; 30: 14080– 90. Google Scholar CrossRef Search ADS PubMed  Radakovic R, Abrahams S. Developing a new apathy measurement scale: Dimensional Apathy Scale. Psychiatry Res  2014; 219: 658– 63. Google Scholar CrossRef Search ADS PubMed  Raichle ME, Gusnard DA. Intrinsic brain activity sets the stage for expression of motivated behavior. J Comp Neurol  2005; 493: 167– 76. Google Scholar CrossRef Search ADS PubMed  Reddy LF, Horan WP, Barch DM, Buchanan RW, Dunayevich E, Gold JM, et al.   Effort-based decision-making paradigms for clinical trials in schizophrenia: part 1—psychometric characteristics of 5 paradigms. Schizophr Bull  2015; 41: 1045– 54. Google Scholar CrossRef Search ADS PubMed  Remy P, Doder M, Lees A, Turjanski N, Brooks D. Depression in Parkinson's disease: loss of dopamine and noradrenaline innervation in the limbic system. Brain  2005; 128 (Pt 6): 1314– 22. Google Scholar CrossRef Search ADS   Rigoux L, Guigon E. A model of reward- and effort-based optimal decision making and motor control. PLoS Comput Biol  2012; 8: e1002716. Google Scholar CrossRef Search ADS PubMed  Rizvi SJ, Pizzagalli DA, Sproule BA, Kennedy SH. Assessing anhedonia in depression: potentials and pitfalls. Neurosci Biobehav Rev  2016; 65: 21– 35. Google Scholar CrossRef Search ADS PubMed  Robert PH, Clairet S, Benoit M, Koutaich J, Bertogliati C, Tible O, et al.   The apathy inventory: assessment of apathy and awareness in Alzheimer's disease, Parkinson's disease and mild cognitive impairment. Int J Geriatr Psychiatry  2002; 17: 1099– 105. Google Scholar CrossRef Search ADS PubMed  Rudebeck PH, Walton ME, Smyth AN, Bannerman DM, Rushworth MF. Separate neural pathways process different decision costs. Nat Neurosci  2006; 9: 1161– 8. Google Scholar CrossRef Search ADS PubMed  Salamone JD, Correa M, Farrar A, Mingote SM. Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology  2007; 191: 461– 82. Google Scholar CrossRef Search ADS PubMed  Salamone JD, Yohn SE, Lopez-Cruz L, San Miguel N, Correa M. Activational and effort-related aspects of motivation: neural mechanisms and implications for psychopathology. Brain  2016; 139 ( Pt 5): 1325– 47. Google Scholar CrossRef Search ADS PubMed  Samanez-Larkin GR, Hollon NG, Carstensen LL, Knutson B. Individual differences in insular sensitivity during loss anticipation predict avoidance learning. Psychol Sci  2008; 19: 320– 3. Google Scholar CrossRef Search ADS PubMed  Samus QM, Rosenblatt A, Steele C, Baker A, Harper M, Brandt J, et al.   The association of neuropsychiatric symptoms and environment with quality of life in assisted living residents with dementia. Gerontologist  2005; 45 Spec No 1: 19– 26. Google Scholar CrossRef Search ADS PubMed  San-Galli A, Varazzani C, Abitbol R, Pessiglione M, Bouret S. Primate ventromedial prefrontal cortex neurons continuously encode the willingness to engage in reward-directed behavior. Cereb Cortex  2016. doi: 10.1093/cercor/bhw351. Schmidt L, Clery-Melin ML, Lafargue G, Valabregue R, Fossati P, Dubois B, et al.   Get aroused and be stronger: emotional facilitation of physical effort in the human brain. J Neurosci  2009; 29: 9450– 7. Google Scholar CrossRef Search ADS PubMed  Schmidt L, d'Arc BF, Lafargue G, Galanaud D, Czernecki V, Grabli D, et al.   Disconnecting force from money: effects of basal ganglia damage on incentive motivation. Brain  2008; 131 ( Pt 5): 1303– 10. Google Scholar CrossRef Search ADS PubMed  Schmidt L, Lebreton M, Clery-Melin ML, Daunizeau J, Pessiglione M. Neural mechanisms underlying motivation of mental versus physical effort. PLoS Biol  2012; 10: e1001266. Google Scholar CrossRef Search ADS PubMed  Schmidt L, Palminteri S, Lafargue G, Pessiglione M. Splitting motivation: unilateral effects of subliminal incentives. Psychol Sci  2010; 21: 977– 83. Google Scholar CrossRef Search ADS PubMed  Scholl J, Kolling N, Nelissen N, Wittmann MK, Harmer CJ, Rushworth MF. The good, the bad, and the irrelevant: neural mechanisms of learning real and hypothetical rewards and effort. J Neurosci  2015; 35: 11233– 51. Google Scholar CrossRef Search ADS PubMed  Schouppe N, Demanet J, Boehler CN, Ridderinkhof KR, Notebaert W. The role of the striatum in effort-based decision-making in the absence of reward. J Neurosci  2014; 34: 2148– 54. Google Scholar CrossRef Search ADS PubMed  Seymour B, O'Doherty JP, Koltzenburg M, Wiech K, Frackowiak R, Friston K, et al.   Opponent appetitive-aversive neural processes underlie predictive learning of pain relief. Nat Neurosci  2005; 8: 1234– 40. Google Scholar CrossRef Search ADS PubMed  Sherdell L, Waugh CE, Gotlib IH. Anticipatory pleasure predicts motivation for reward in major depression. J Abnorm Psychol  2012; 121: 51– 60. Google Scholar CrossRef Search ADS PubMed  Shidara M, Richmond BJ. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science  2002; 296: 1709– 11. Google Scholar CrossRef Search ADS PubMed  Skvortsova V, Palminteri S, Pessiglione M. Learning to minimize efforts versus maximizing rewards: computational principles and neural correlates. J Neurosci  2014; 34: 15621– 30. Google Scholar CrossRef Search ADS PubMed  Sockeel P, Dujardin K, Devos D, Deneve C, Destee A, Defebvre L. The Lille apathy rating scale (LARS), a new instrument for detecting and quantifying apathy: validation in Parkinson's disease. J Neurol Neurosurg Psychiatry  2006; 77: 579– 84. Google Scholar CrossRef Search ADS PubMed  Starkstein SE, Mayberg HS, Preziosi TJ, Andrezejewski P, Leiguarda R, Robinson RG. Reliability, validity, and clinical correlates of apathy in Parkinson's disease. J Neuropsychiatry Clin Neurosci  1992; 4: 134– 9. Google Scholar CrossRef Search ADS PubMed  Stephens DW, Krebs JR. Foraging theory . Princeton, NJ; Guildford: Princeton University Press; 1986. Strait CE, Blanchard TC, Hayden BY. Reward value comparison via mutual inhibition in ventromedial prefrontal cortex. Neuron  2014; 82: 1357– 66. Google Scholar CrossRef Search ADS PubMed  Strauss GP, Waltz JA, Gold JM. A review of reward processing and motivational impairment in schizophrenia. Schizophr Bull  2014; 40 (Suppl 2): S107– 16. Google Scholar CrossRef Search ADS PubMed  Strauss GP, Whearty KM, Morra LF, Sullivan SK, Ossenfort KL, Frost KH. Avolition in schizophrenia is associated with reduced willingness to expend effort for reward on a progressive ratio task. Schizophr Res  2016; 170: 198– 204. Google Scholar CrossRef Search ADS PubMed  Tachibana Y, Hikosaka O. The primate ventral pallidum encodes expected reward value and regulates motor action. Neuron  2012; 76: 826– 37. Google Scholar CrossRef Search ADS PubMed  Thobois S, Ardouin C, Lhommee E, Klinger H, Lagrange C, Xie J, et al.   Non-motor dopamine withdrawal syndrome after surgery for Parkinson's disease: predictors and underlying mesolimbic denervation. Brain  2010; 133 ( Pt 4): 1111– 27. Google Scholar CrossRef Search ADS PubMed  Treadway MT, Bossaller NA, Shelton RC, Zald DH. Effort-based decision-making in major depressive disorder: a translational model of motivational anhedonia. J Abnorm Psychol  2012; 121: 553– 8. Google Scholar CrossRef Search ADS PubMed  Treadway MT, Buckholtz JW, Schwartzman AN, Lambert WE, Zald DH. Worth the ‘EEfRT'? The effort expenditure for rewards task as an objective measure of motivation and anhedonia. PLoS One  2009; 4: e6598. Google Scholar CrossRef Search ADS PubMed  Treadway MT, Peterman JS, Zald DH, Park S. Impaired effort allocation in patients with schizophrenia. Schizophr Res  2015; 161: 382– 5. Google Scholar CrossRef Search ADS PubMed  Treadway MT, Zald DH. Reconsidering anhedonia in depression: lessons from translational neuroscience. Neurosci Biobehav Rev  2011; 35: 537– 55. Google Scholar CrossRef Search ADS PubMed  van Almenkerk S, Smalbrugge M, Depla MF, Eefsting JA, Hertogh CM. Apathy among institutionalized stroke patients: prevalence and clinical correlates. Am J Geriatr Psychiatry  2015; 23: 180– 8. Google Scholar CrossRef Search ADS PubMed  van Dalen JW, Moll van Charante EP, Nederkoorn PJ, van Gool WA, Richard E. Poststroke apathy. Stroke  2013; 44: 851– 60. Google Scholar CrossRef Search ADS PubMed  Varazzani C, San-Galli A, Gilardeau S, Bouret S. Noradrenaline and dopamine neurons in the reward/effort trade-off: a direct electrophysiological comparison in behaving monkeys. J Neurosci  2015; 35: 7866– 77. Google Scholar CrossRef Search ADS PubMed  Venugopalan VV, Casey KF, O'Hara C, O'Loughlin J, Benkelfat C, Fellows LK, et al.   Acute phenylalanine/tyrosine depletion reduces motivation to smoke cigarettes across stages of addiction. Neuropsychopharmacology  2011; 36: 2469– 76. Google Scholar CrossRef Search ADS PubMed  Voon V, Gao J, Brezing C, Symmonds M, Ekanayake V, Fernandez H, et al.   Dopamine agonists and risk: impulse control disorders in Parkinson's disease. Brain  2011; 134 ( Pt 5): 1438– 46. Google Scholar CrossRef Search ADS PubMed  Voon V, Napier TC, Frank MJ, Sgambato-Faure V, Grace AA, Rodriguez-Oroz M, et al.   Impulse control disorders and levodopa-induced dyskinesias in Parkinson's disease: an update. Lancet Neurol  2017; 16: 238– 50. Google Scholar CrossRef Search ADS PubMed  Voon V, Pessiglione M, Brezing C, Gallea C, Fernandez HH, Dolan RJ, et al.   Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors. Neuron  2010a; 65: 135– 42. Google Scholar CrossRef Search ADS   Voon V, Reynolds B, Brezing C, Gallea C, Skaljic M, Ekanayake V, et al.   Impulsive choice and response in dopamine agonist-related impulse control behaviors. Psychopharmacology  2010b; 207: 645– 59. Google Scholar CrossRef Search ADS   Wallis JD. Cross-species studies of orbitofrontal cortex and value-based decision-making. Nat Neurosci  2011; 15: 13– 19. Google Scholar CrossRef Search ADS PubMed  Walton ME, Bannerman DM, Alterescu K, Rushworth MF. Functional specialization within medial frontal cortex of the anterior cingulate for evaluating effort-related decisions. J Neurosci  2003; 23: 6475– 9. Google Scholar PubMed  Walton ME, Kennerley SW, Bannerman DM, Phillips PE, Rushworth MF. Weighing up the benefits of work: behavioral and neural analyses of effort-related decision making. Neural Netw  2006; 19: 1302– 14. Google Scholar CrossRef Search ADS PubMed  Wang J, Huang J, Yang XH, Lui SS, Cheung EF, Chan RC. Anhedonia in schizophrenia: deficits in both motivation and hedonic capacity. Schizophr Res  2015; 168: 465– 74. Google Scholar CrossRef Search ADS PubMed  Wardle MC, Treadway MT, Mayo LM, Zald DH, de Wit H. Amping up effort: effects of d-amphetamine on human effort-based decision-making. J Neurosci  2011; 31: 16597– 602. Google Scholar CrossRef Search ADS PubMed  Westbrook A, Kester D, Braver TS. What is the subjective cost of cognitive effort? Load, trait, and aging effects revealed by economic preference. PLoS One  2013; 8: e68210. Google Scholar CrossRef Search ADS PubMed  Wolf DH, Satterthwaite TD, Kantrowitz JJ, Katchmar N, Vandekar L, Elliott MA, et al.   Amotivation in schizophrenia: integrated assessment with behavioral, clinical, and imaging measures. Schizophr Bull  2014; 40: 1328– 37. Google Scholar CrossRef Search ADS PubMed  Yang XH, Huang J, Zhu CY, Wang YF, Cheung EF, Chan RC, et al.   Motivational deficits in effort-based decision making in individuals with subsyndromal depression, first-episode and remitted depression patients. Psychiatry Res  2014; 220: 874– 82. Google Scholar CrossRef Search ADS PubMed  Zenon A, Devesse S, Olivier E. Dopamine manipulation affects response vigor independently of opportunity cost. J Neurosci  2016a; 36: 9516– 25. Google Scholar CrossRef Search ADS   Zenon A, Duclos Y, Carron R, Witjas T, Baunez C, Regis J, et al.   The human subthalamic nucleus encodes the subjective value of reward and the cost of effort during decision-making. Brain  2016b; 139 ( Pt 6): 1830– 43. Google Scholar CrossRef Search ADS   Zenon A, Sidibe M, Olivier E. Pupil size variations correlate with physical effort perception. Front Behav Neurosci  2014; 8: 286. Google Scholar PubMed  Zipf GK. Human behavior and the principle of least effort: an introduction to human ecology . Cambridge, MA: Addison-Wesley Press; 1949. © The Author (2017). Published by Oxford University Press on behalf of the Guarantors of Brain. All rights reserved. For Permissions, please email: journals.permissions@oup.com

Journal

BrainOxford University Press

Published: Mar 1, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 12 million articles from more than
10,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Unlimited reading

Read as many articles as you need. Full articles with original layout, charts and figures. Read online, from anywhere.

Stay up to date

Keep up with your field with Personalized Recommendations and Follow Journals to get automatic updates.

Organize your research

It’s easy to organize your research with our built-in tools.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

Monthly Plan

  • Read unlimited articles
  • Personalized recommendations
  • No expiration
  • Print 20 pages per month
  • 20% off on PDF purchases
  • Organize your research
  • Get updates on your journals and topic searches

$49/month

Start Free Trial

14-day Free Trial

Best Deal — 39% off

Annual Plan

  • All the features of the Professional Plan, but for 39% off!
  • Billed annually
  • No expiration
  • For the normal price of 10 articles elsewhere, you get one full year of unlimited access to articles.

$588

$360/year

billed annually
Start Free Trial

14-day Free Trial