model of reward learning

In a prominent model of reward learning, Montague et al. (p. 1944, [304]) have argued that dopamine neurons deliver “information about prediction errors between the expected amount of reward and the actual reward”. Their computational model is based on data collected primarily by Schultz and colleagues in an important series of electro- physiological studies on the relationship between the discharge rate of presumed dopamine neurons and the presentation of food rewards, and conditioned stimuli predictive of food rewards [5,277,278,300,301,371,398,399,401- 403], Accordingly, Schultz himself concurs that dopamine neurons “signal deviations from the prediction of future appetitive events” (p. 191, [400]). In a collaborative review, Schultz, Dayan, and Montague [405] concluded that “dopamine activity encodes expectations about external stimuli or reward” (p. 1594, [405]), and can signal both “changes and errors in the predictions of future salient and rewarding events” (p. 1593, [405]). Appetitive rewarding events, for Schultz, are things that “elicit approach reactions ..., serve as positive reinforcers..., serve as goals of voluntary behavior... and “have an emotional function by inducing subjective feelings of pleasure and hedonia” (p. 191, Schultz [400]), a view that seems to join both incentive ‘wanting’ and hedonic ‘liking’. In these accounts, the activity of dopamine neurons seems to reflect reward learning in two ways. First, neuronal activation to a conditioned stimulus phasically registers an increment in the momentary strength of a prediction regarding an impending ‘wanted’ and ‘liked’ event. Second, dopamine neuronal activation to an unconditioned hedonic reward, such as the actual taste of food, encodes discrepancies between the prediction strength and the actual magnitude of ‘liking’ produced by the event itself [400],

References

Berridge, Kent C., and Terry E. Robinson. 1998. “What is the role of dopamine in reward: Hedonic impact, reward learning, or incentive salience?” Brain Research Reviews 28 (3): 309–69. doi:10.1016/S0165-0173(98)00019-8.