The Reward Circuit - a brief overview

My current research project makes use of positive feedback as a form of reward or incentive in learning. This is actually exciting as it combines motor learning processes and positive reinforcement. Reinforcement and reward-based learning implicate the limbic network, in which the basal ganglia, BG, is one of the most important members. Specifically, it is the ventral portion of the striatum called the nucleus accumbens (NAcc) and underlies a series of behavior e.g., motivation, and emotion. Initially thought as purely motoric, BG evolved to engage more diverse behavior such as executive, and then the limbic functions. The inclusion of the limbic region as a part of the BG has also been proven anatomically (Nauta, et al., 1978; Mogenson et al., 1980; Haimer et al., 1986). More importantly, the BG limbic circuit does not work in isolation. This summary will be based on an excellent review by Haber and Knutson (2010) and a few other relevant good stuffs will be provided as references at the end.

Neuroanatomy of Reward
Prefrontal Reward Regions
The involvement of prefrontal cortex in reward comes naturally. Scientists have been interested in studying the role of the frontal lobe in cognition in which reward processing is one of the main features. The main reward-based prefrontal regions traditionally include the anterior cingulate cortex (ACC) that includes BA 24, 25, and 32 and the orbitofrontal cortex (OFC) that includes BA 11, 12, 13, and 14. Unlike sensorimotor cortex, the prefrontal cortex is diverse in terms of cytoarhitectonic features and functions. The recent paper in PNAS (Neubert FX, et al, 2015) scrupulously describes similarities and differences between primate and human prefrontal cortex associated with reward. In general, the human prefrontal cortex is sub-divided into a few areas:
  (a) Sensory region: the orbital part of the brain linked to the olfactory bulb and the insula.
  (b) Ventromedial prefrontal cortex (vmPFC) that includes BA 10, 11, and 32.
  (c) Rostral OFC that covers BA 11, 12, and 13.
  (d) Dorsal ACC or dACC that is BA 24.

Reinforcement-based motor learning presumably implicates basal ganglia (Schultz et al., 1997; Graybiel, 2005). In fact, reward-based action may involve more complex neuronal processes beyond the traditional basal ganglia and sensorimotor loops. For example, it is possible that reward-based decision-making is also involved (Rushworth et al., 2004), such that during learning rewards may influence the production of subsequent movements. Prior studies show that regions in the prefrontal cortex are involved in this type of activity (Shima and Tanji, 1998). Using fMRI in gambling tasks (Daw et al., 2006), it has been shown that the intraparietal sulcus and frontopolar cortex are preferentially active during exploration. In contrast, regions of the striatum and ventromedial prefrontal cortex are involved in exploitative decision making to accumulate more rewards. The vmPFC is a region in which activity is associated with stimulus-reward value, selecting actions that are more rewarding (O'Doherty et al., 2003; Rushworth et al., 2004; Daw et al., 2006) and encoding the value of performed decisions (Knutson et al., 2001; Smith et al., 2010).

The Ventral Striatum
In 1954, Olds and Milner discovered how a tiny structure called the NAcc that lies ventral to the sensorimotor striatum is linked to reward behavior in rats. In 1978, Heimer described the link between NAcc and olfactory tubercle in rats. Ventral striatum was later regarded as the reward center and thought to be the interface between the limbic and motor systems (Mogenson et al., 1980). In recent years, the traditional boundaries of ventral striatum have expanded beyond NAcc which includes the ventral portion of the striatum and the ventral caudate. Thus, the name ventral striatum (VS) is legit to contrast this with the more dorsal, sensorimotor-related striatum. The NAcc structure has an outer shell and a core, each contains different neuronal cell types and functions.

(1) Afferent projections
Like the dorsal sensorimotor striatum, VS receives massive topographic glutamatergic inputs from the cerebral cortex, from the thalamus, and the brain stem. The word topographic deserves an emphasis, refer to Fig-2. From the prefrontal reward regions, vmPFC sends projections mainly to the NAcc. From NAcc, we go more dorsally and laterally to regions that cover ventral caudate nucleus and ventral putamen. Principally, these two areas receive projections from the OFC. The dACC also projects to a more central and lateral portion of caudate and putamen. Lastly, the DPFC (dorsal prefrontal cortex) terminates more diffusely along the rostro-caudal striatum, particularly the head of the caudate. A critical difference with the dorsal striatum, VS also receives projections from the amygdala and hippocampus. The afferent axons are concentrated within the NAcc. Amygdala is known to play a role in processing reward, e.g. an emotional aspect of reward, associating stimuls and reward or punishment. Thalamus is the final link connecting VS with the rest of the brain regions. In the thalamus, reward processing is managed by one of the largest nuclei there called the medial-dorsal nucleus (MD) that carries bidirectional wiring between the frontal lobe and VS, amygdala, and hippocampus.

(2) Efferent projections
Like the dorsal sensorimotor striatum, VS sends efferent axons to two main targets. First, the more specific efferent outputs to the ventral pallidum, not GPe or GPi. Second, more broad midbrain regions such as the ventral tegmental area (VTA) and middle substantia nigra (SNpc) which are rich of the dopaminergic neurons. The pathway between midbrain and VS is known as the mesolimbic pathway and is known to mediate reward processing. Other efferent projections are to the pedunculopontine nucleus (PPT) and nucleus basalis (NB, the principal source of cholinergic fibers to the cortex and amygdala). In particular, PPT carries broad functions such as arousal, attention, motivation, and voluntary limb movements.

Midbrain dopamine neurons
The involvement of dopamine in reward is perhaps first shown elegantly by the work of W. Schultz since 1970s. The VTA and SNpc, the substantia nigra pars compacta, are two important components and their neurons can be identified using a certain phenotypic marker, e.g. calbindin for dorsal SNpc and VTA (the ventral SNpc, however, is calbindin-negative).

(1) Afferent projections
The principal inputs to VTA/SNpc come from the striatum through both the GPe and VP. Other inputs come from other parts of the brain stem, mainly PPT. Anatomically, the largest source of projection comes from the ventral striatum.

(2) Efferent projections
Midbrain dopamine neurons send massive projections out back to the striatum through the mesolimbic pathway mentioned above. It has been found that there is a medio-lateral and an inverse dorso-ventral topography arrangement. Thus, the ventral portion of SNpc projects to the dorsal striatum, the dorsal portion projects to the ventral striatum. It has also been observed that there is differential efferent projections back to the striatum. In other words, VS receives the least number of projections while the sensorimotor striatum receives the most. This striato-nigro-striatal network is consistent with the idea that the limbic system is able to influence sensorimotor behavior through the interface situated in the striatum. Apart from the striatum, dopamine neurons from the midbrain send rather diffuse projections to the frontal lobe through the mesocortical pathway, so it is wrong to say that BG is the only target of dopamine in the brain.


Further Readings
[1]  SN Haber, Knutson B. (2010). "The reward circuit: linking primate anatomy and human imaging". Neuropsychopharmacology.
[2]  FX Neubert, et al. (2015). "Connectivity reveals relationship of brain areas for reward-guided learning and decision making in human and monkey frontal cortex". Proc. Nat Acad. Science.