5. Dopamine release (tonic, phasic) and encoding.
5.1. Dopamine release
- 5.1.1. Source of dopamine release
- 5.1.2. Function of vesicles
- 5.1.3. Forms of neurotransmitter transmission
- 188.8.131.52. Endocrine transmission
- 184.108.40.206. Volume transfer
- 220.127.116.11. Synaptic transmission
- 18.104.22.168. Dopaminergic transmission
- 5.1.4. Axonal and somatodendritic release
- 5.1.5. Active zone
- 5.1.6. Function of synapses
- 5.1.7. Trigger of dopamine release
- 5.2. Tonic dopamine / phasic dopamine
5.3. Coding of behavioral values by dopamine
- 5.3.1. Speed of dopamine level change in the brain encodes different behaviors.
- 5.3.2. 10-Minute range encodes strength of motivation and behavioral activation
- 5.3.3. Seconds range encodes value of future reward
- 5.3.4. Subsecond range activates a. reward search and b. movement
- 5.3.5. Does mesolimbic phasic DA encode the value of work?
- 5.3.6. Does acetylcholine switch between dopaminergic encoding of reward prediction error and learning?
- 5.3.7. Different DA neurons in relation to aversive stimuli?
- 5.3.8. Phasic DA encodes movement in the dorsal striatum, reward in the ventral striatum
- 5.3.9. Does dopamine encode the utility of a resource expenditure?
5.1. Dopamine release
5.1.1. Source of dopamine release
22.214.171.124. Dopamine synthesis and dopamine release from dopaminergic neurons
Dopamine neurons in the VTA and substantia nigra synthesize dopamine. This is transported to dopaminergic cells via axons.
126.96.36.199. Dopamine release from noradrenergic neurons
Since dopamine is also reuptaken by NETs in addition to the DAT (see below under degradation of dopamine) and stored by them in vesicles (especially in the PFC), dopamine taken up in this way should also be released from noradrenergic cells. In the mPFC, dopamine appears to originate exclusively from noradrenergic neurons. If the noradrenergic cells of the locus coeruleus are inactivated, extracellular dopamine and noradrenaline levels in the mPFC decrease.3
5.1.2. Function of vesicles
Fusion of synaptic vesicles with the presynaptic plasma membrane is brought about by the formation of the SNARE complex from the vesicular SNARE synaptobrevin-2/VAMP-2 and the plasma membrane SNARE proteins syntaxin-1 and SNAP-25. A small fraction of the neurotransmitter-filled vesicles is bound by synapsin to the actin skeleton of the active zone near voltage-gated Ca2+ membrane channels, where it is activated. When an action potential depolarizes the presynaptic plasma membrane, Ca2+ enters through these channels and triggers fusion of the vesicles with the membrane by means of synaptotagmin 1, 2, or 9, which act as Ca2+ sensors. Neurotransmitters then leak from the vesicles into the synaptic cleft to activate postsynaptic receptors.
Botulinum toxin A and B cleave SNAP-25 and synaptobrevin-2, inhibiting vesicle docking to the cell membrane. Tetanus toxin also cleaves synaptobrevin-2, but does not always inhibit dopamine release.1
The pool of easily released dopamine vesicles refills only slowly. The depletion of dopamine release after a single stimulus lasts for several tens of seconds. Restoration of release readiness thus takes one to two orders of magnitude longer than for fast synapses. Since the speed of replenishment is crucial for the frequency range in which a transmission system can operate, and since dopamine receptors are “slow” GPCRs, the dopamine system is not well suited for high-frequency information transmission.2
The amount and frequency of neurotransmitter release from vesicles is variable.4
5.1.3. Forms of neurotransmitter transmission
There are different forms of transmission of neurotransmitters. These differ in the precision of release and in the organization of the receptors.2
188.8.131.52. Endocrine transmission
Endocrine cells usually release hormones as transmitters. The release takes place at the cell surface. The transmitters travel distances of millimeters to meters through the extracellular space and the bloodstream to distant receptors. Endocrine cells often do not show specially designed release sites.2
184.108.40.206. Volume transfer
In volume transfer, transmitters diffuse into the. Receptors are only loosely connected to the release sites and are located a few hundred nanometers to a few millimeters away. Transmitters are usually released from specialized sites similar to the active zone. The distance to the determines receptor activation and is therefore characterized by a steep transmitter concentration gradient.2
220.127.116.11. Synaptic transmission
In synaptic transmission, there is a very tight spatial coupling of a few tens of nanometers between the transmitter-secreting active zone and the receptor clusters. The active zone and receptor clusters are often aligned at the subsynaptic level. Signal transmission occurs only within the synaptic cleft, resulting in accurate and efficient receptor activation.2
18.104.22.168. Dopaminergic transmission
Dopamine is released from synaptic and non-synaptic varicosities.
Most dopamine varicosities are not bound to postsynaptic cells and densities. Several studies suggest a distributed localization
- broadly localized on D1-MSN
- with somatic, dendritic shaft and dendritic spine localizations
- At times they occur in groups
- broadly distributed within D2-MSNs
- possibly increased in distal dendrites
In dopamine varicosities with synaptic-like contacts, D1 and D2 are found
- rarely in the opposite postsynaptic membrane
- often perisynaptic (within 100 nm of the edges of synaptic-like apposition)
- often extrasynaptic (beyond 100 nm of synaptic-like contact)
Thus, dopamine receptors seem to be partially clustered on MSNs. The majority of dopamine receptors are (so far) not detectable in synapse-like apposition.
Therefore, the term “dopamine synapse” should not be understood narrowly, in the sense of a postsynaptic structure containing dopamine receptors, but broadly, in the sense of a dopamine transmission system.2
Reserpine blocks VMAT and thus the incorporation of dopamine into the vesicles. Rabbits treated with reserpine became paralyzed. Administration of L-DOPA, a precursor of dopamine, restored locomotor capacity even though VMAT blockade persisted. Thus, in the absence of vesicular dopamine and precise vesicular release in synapses, the brain can metabolize L-DOPA and use it for locomotion
Vertebrate dopamine receptors are exclusively G protein-coupled receptors, which are orders of magnitude slower than ionotropic receptors. Dopamine is therefore able to transmit signals even without precise synaptic communication: the so-called volume transmission.2
5.1.4. Axonal and somatodendritic release
In principle, neurotransmitters can be transferred in several ways:5
- From the axon (axoaxonal)
- Axon stem to synapse
- Axon terminal head at synapse
- Axon in extracellular space (nondirected synapses)
- From the cell membrane to synapses (axosomatic)
- From dendrites to synapses (axodendritic)
- Dendrite stem at synapses
- Dendrite spinous processes at synapses
Dopamine is released in a number of ways:1
22.214.171.124. Axonal release, Axonal varicosities
The dopamine formed is transported via nerve tracts (axons) into dopaminergic cells. There, it is stored in vesicles, which are later brought to the cell membrane and, in response to electrical impulses, release the dopamine into the synaptic cleft, which is approx. 20 to 40 nanometers wide and 0.5 nanometers deep. Around 1000 dopamine molecules are released per electrical impulse.6
Axons mediate most of the dopamine transmission. Axonal dopamine varicosities are densely packed with clusters of small, clear vesicles. Quantitative events reminiscent of single vesicular dopamine packets may be sensed by dopamine axons or cell bodies. Blockade or knockout of VMAT2 terminates dopamine transmission.
Only about 20% of varicosities secrete dopamine. It is conceivable that axon varicosities also secrete dopamine without synapses.2
126.96.36.199. Release from cell bodies and dendrites (somatodendritic release)
- May also originate from specialized secretory organelles, as dopamine is stored mainly in tubulo-vesicular structures resembling the smooth endoplasmic reticulum in the soma or dendrites
- relevant in VTA for induction of behavioral sensitization to amphetamine through activation of local D1 receptors
- in the substantia nigra relevant for the control of motor performance
- addresses D2 autoreceptors
- causes negative feedback
- increases GABA release in substantia nigra pars reticulata
- via D1/D5 receptors
- could activate feedback signals for dopamine regulation between substantia nigra pars compacta and substantia nigra pars reticulata
- which in turn could influence axonal DA release7
5.1.5. Active zone
The active zone is a protein network. It is found in nerve cells and axons presynaptically directly opposite synapses. The active zone of a synapse docks synaptic vesicles and stimulates them. As a result, the active zone forms a pool of readily releasable vesicles, and positions these vesicles at specific distances from presynaptic Ca2+ channels, thereby controlling the vesicular release probability. Only a few percent of the vesicles present are part of the pool ready for release.2
188.8.131.52. Active zone in nerve cells
Synaptic transmission is characterized by its speed and spatial precision. The fusion of synaptic vesicles with the cell membrane occurs in less than a millisecond after the arrival of an action potential. Release from a vesicle occurs spatially precisely opposite postsynaptic receptors from the active zone.8 The active zone binds vesicles primed for release to the presynaptic plasma membrane near Ca2+ channels.9
The active zone consists of the molecular scaffolds1
184.108.40.206. Active zone in axons
Dopamine axons contain active zonelike protein scaffolds composed of:1
- Only 30% of varicosities of dopaminergic axons contain RIM. RIM and MUNC13 are essential (as in conventional synapses) for the function of an active zone. This may explain why only 20% to 30% of varicosities of axons actively secrete dopamine.
- Munc13-1 (probably)
30% of dopamine varicosities contain postsynaptic densities and form GABAergic synaptic structures with presynaptic neurexin and postsynaptic neuroligin-2. Therefore, dopamine could also be released on axons only at synapses. In substantia nigra, D1 receptors are located at presynaptic sites on small-diameter axons that are not in contact with tyrosine hydroxylase-positive elements and on terminal end-buttons that form symmetrical synapses on tyrosine hydroxylase-positive or -negative dendrites.10
Whether somatodendritic dopamine release also functions by means of active zones is unknown.
5.1.6. Function of synapses
Chemical synapses consist of
- Presynaptic apparatus for transmitter release
- Postsynaptic apparatus for receptor-mediated signal transduction
- Gray type I
- Asymmetric synapse
- Gray type II
- Symmetrical synapse
220.127.116.11. Dopaminergic synapses
Dopamine synapses are commonly found in dendritic shafts and spines of medium-sized spiny projection neurons (MSNs).
Midbrain dopamine neurons project densely into the striatum and form so-called dopamine synapses at MSNs, the main neurons in the striatum. Dopamine receptors here are distant from dopamine synapses, so it remains unclear how dopamine synapses are involved in dopaminergic transmission. Single vesicular fusion events can activate D1 and D2, with nearby receptors more likely to be activated by dopamine than those farther away. The precise organization of dopamine receptors in relation to release sites is not yet known.2
Dopaminergic dendritic synapses address postsynaptic GABA
One study found that dopamine synapses represent contacts between dopaminergic presynaptic and GABAergic postsynaptic structures:13
The presynaptic structure expressed:
- Tyrosine hydroxylase (relevant for dopamine synthesis)
- VMAT2 (Relevant for vesicle filling)
- DAT (relevant to dopamine reuptake)
The postsynaptic structure of dopamine synapses expressed GABAergic molecules:
- Neuroligin-2 (postsynaptic adhesion molecule)
- promotes presynaptic differentiation in axons of midbrain dopamine neurons and striatal GABA-ergic neurons
- Gephyrin (postsynaptic scaffold molecule)
- GABAA receptor α1
- without specific clustering of dopamine receptors
Elimination of neuroligin-2 in the striatum caused
- significant decrease of the dopamine synapses
- reciprocal increase of GABAergic synapses at MSN dendrites
Neuroligin-2 appears to direct the formation of synapses in the striatum by giving heterologous dopamine synapses a competitive advantage over conventional GABA-ergic synapses. Because MSN dendrites are preferred targets of dopamine synapses and express high levels of dopamine receptors, dopamine synapse formation may serve to increase the specificity and efficacy of dopaminergic modulation of striatal output by anchoring dopamine release sites to dopamine sensor targets.
Dopamine receptors are not detectable in postsynaptic structures with currently (2021) available tools, so it is an open question whether and to what extent dopamine receptors are found in dopamine synapses.2
18.104.22.168. Domain overlap model
A growing body of evidence suggests that dopamine signaling is not only temporally dynamic but also spatially organized.
In addition to synaptic point-to-point transmission and broad (extracellular?) neurotransmitter transmission, a domain overlap model in which release and receptors are arranged in micrometer-sized structures relative to each other may be relevant. This is based on rapid release followed by diffusion with a micrometer-sized release-receptor organization.
This model is thought to allow activation of receptor subsets located in micrometer-sized domains of release sites during baseline activity on the one hand, and broader receptor activation with domain overlap when firing is synchronized across dopamine neuron populations on the other. This signaling structure, together with the properties of dopamine release, may explain how switching firing modes supports a broad and dynamic role of dopamine and can lead to marked modulation of signaling pathways.2
5.1.7. Trigger of dopamine release
22.214.171.124. Trigger of axonal dopamine release: calcium ions (Ca2+) at N, P/Q, T, R channels
Axonal dopamine release is triggered by extracellular Ca2+.1
Somatodendritic dopamine release in substantia nigra pars compacta (SNc) persists even at extracellular Ca2+ so low that it would be insufficient to increase axonal release in the striatum
Dopamine release evoked by a single pulse in the dorsal striatum and SNc is independent of regulation by simultaneously released glutamate or GABA
The first pulse releases approximately 60% of the dopamine from the vesicle pool ready for release.2
- by N-channel blocker (omega-conotoxin GVIA, 100 nm), completely prevented (Cav2 channel)
- by P/Q channel blocker (omega agatoxin IVA, 200 nm), reduced by 75% (Cav2 channel)
- reduced by 25 % by T-channel blockers (Ni2+, 100 microns) (Cav3 channel)
- reduced by 25 % by R-channel blocker (SNX-482, 100 nm)
- different: R channel without influence15
- not reduced by L-channel blocker (nifedipine, 20 microns) (Cav1 channel)
- different: L channel with influence15
None of these Ca2+ channel blockers affected somatodendritic dopamine release in the substantia nigra, either alone or together (but duration of release response shortened in the latter).
Inhibition of Cav1 channels may promote survival of dopamine neurons.1
The Ca2+-triggering agents of axonal dopamine release are largely unknown.
At fast synapses:
- Synaptotagmin 1,2 and 9 trigger rapid release
- Synaptotagmin 7 and Doc2 mediate Ca2+ sensitivity16
- the asynchronous and spontaneous release
- the facilitation
Only 8 synaptotagmines bind Ca2+: synaptotagmin 1, 2, 3, 5, 6, 7, 9, and 10
126.96.36.199. Trigger of somatodendritic dopamine release
Mediate somatodendritic release:17
- Synaptotagmin 4
- Synaptotagmin 7
5.2. Tonic dopamine / phasic dopamine
Tonic firing refers to sustained activity of a dopamine neuron at 0.2-10 Hz mediated by cell autonomic pacemaker impulses
Burst firing is characterized by brief bursts of action potentials (3-10 spikes, >10 Hz) from a dopamine neuron. They are usually caused by activation of NMDA receptors via excitatory inputs and represent the response to environmental stimuli. Burst firing is sometimes referred to as phasic firing, emphasizing the synchrony of activity of dopamine neurons due to common inputs.2
Tonic and phasic firing should be distinguished from tonic and phasic release.
Firing rates of somatic dopamine neurons are not linearly translated into axonal dopamine release because release is subject to strong short-term depression. Moreover, nearly one-third of tonic release occurs without somatic firing.2
5.2.1. Tonic dopamine firing
50-98% of dopamine neurons exhibit tonic firing in vivo.2 Midbrain dopamine neurons show spontaneous clock firing from 0.2 to 10 Hz.1 The usual tonic firing frequency of dopaminergic neurons in rats is about 4 Hz.1819 Tonic dopamine firing occurs particularly at varicosities, extrasynaptically, in the extracellular space. From there, dopamine diffuses to autoreceptors or to (extrasynaptic) receptors of the patient’s own neuron or of other, sometimes relatively distant, neurons (volume transmission). Dopamine is degraded in the extracellular space by COMT.20
Since tonic dopamine is not released into the synapse, it does not trigger a signal at postsynaptic receptors. It only activates presynaptic autoreceptors (also from neighboring nevus cells), which in turn can slow down the phasic dopamine release of their neuron (negative feedback)
Tonic dopamine in the nucleus accumbens is likely to be regulated by glutamatergic afferents from the PFC.21
Switching a response strategy due to changing criteria to achieve goals requires a decrease in tonic dopamine.22 Sustained elevated tonic dopamine therefore causes rigidity23 and is therefore likely to promote task switching problems.
Sometimes extracellular dopamine is inaccurately referred to as tonic dopamine, although extracellular dopamine can come from other sources, such as diffusion of dopamine from the synaptic cleft. Phasic dopamine contributes more to extracellular dopamine than tonic dopamine.24
Tonic release produces short-lived dopamine transients of a few milliseconds at a small, variable subset of release sites. Dopamine is released into the extracellular space where it rapidly disperses. The basal dopamine level (approximately 2 to 20 nM) is the result of a balance between tonic release and DAT (and NET) reuptake. The basal dopamine level is below the activation threshold of most dopamine receptors. Presumably, the basal dopamine level is composed of a large number of small, short-lived dopamine spikes. Tonic signaling is likely mediated by these short-lived dopamine signals near the release sites, not by the basal dopamine level itself.2
The basal level of dopamine develops:
- up to 70 % by firing action potentials
- to 30% independent of action potentials and the active zone proteins RIM and Munc, for example, explained by spontaneous vesicular fusion
In experimental paradigms, low-frequency stimulation is often used to mimic tonic release. However, this does not mimic the stochastic feature of activation of release sites that is typical of tonic release, but rather recruits many axons simultaneously, thus mimicking the essential feature of phasic release.
5.2.2. Phasic dopamine firing, bursts
Phasic dopamine signaling is relevant to synaptic plasticity, reward processing, and behavioral learning.2526
Phasic release on the one hand and bursts (phasic firing) on the other hand correlate with each other but are to be distinguished:2
- Phasic release depends on the simultaneous recruitment of a population of dopamine neurons and relies on synchrony between dopamine neurons. Phasic release does not require burst firing of individual neurons
- Burst firing is the result of a single synchronous activation of a large number of dopamine releases. In general, burst firing is synchronized across all dopamine neurons. The first spike efficiently increases dopamine levels, whereas subsequent activity releases less dopamine due to reduced synchrony and the presence of refractory sites. Subsequent burst spikes therefore increase dopamine levels only slightly, but serve to maintain the elevated levels caused by the first spike, thus prolonging the dopamine’s residence time
- Phasic dopamine release (bursts) occurs from vesicles into the synapse. Stimuli such as reward or other stimuli activate short bursts of action potentials from dopaminergic neurons. These dopamine bursts occur at around 20 Hz or more24, last less than 200 ms, and release large amounts of dopamine from storage vesicles in the presynapse into the synaptic cleft. This phasically released dopamine crosses the synaptic cleft and activates receptors at the postsynapse. After release by the receptors, dopamine is taken up from the synaptic cleft back into the presynapse by dopamine transporters (reuptake). In smaller amounts, it diffuses out of the synapse into the extracellular space or is degraded (albeit secondarily) by COMT located in the synaptic cleft.2027
Dopamine release occurs robustly in response to an initial activation, but then quickly tapers off for several tens of seconds. It follows that even tonic firing leads to the depletion of the respective dopamine release site for seconds, and dopamine release in response to each action potential is largely determined by the recovery of these release sites. Neurons with lower spontaneous activity therefore contribute more to phasic release because their release-ready vesicle pool is less depleted when the synchronizing stimulus arrives. In burst firing, only the first few action potentials result in significant dopamine release from a single axon. Thus, it is the synchrony of group firing, rather than the firing pattern of individual neurons, that dominates signaling during phasic release. This view is supported by the fact that in mice lacking NMDA receptors, burst firing is severely impaired, but phasic dopamine transients and the behaviors they mediate persist. Phasic release is the result of simultaneous activation of a large number of dopamine release sites. Dopamine reuptake mechanisms are temporarily overdriven. This results in significant crosstalk between dopamine signaling areas and causes prolonged dopamine dwell times. In phasic signaling, the rapid increase in dopamine over spatial ranges of several micrometers can result in the activation of dopamine receptors that are somewhat distant from the release sites. A prerequisite for phasic release and signaling is synchrony of release across dopamine neuron populations.2
In the dorsolateral striatum, increasing the burst length from 1 to 10 pulses (at 20 Hz) only moderately enhanced the dopamine signal, whereas in the nucleus accumbens, dopamine release increased sharply with increasing burst length (even more so in the NAc shell than in the NAc nucleus).18
5.2.3. Extracellular dopamine
Phasic dopamine is released into the synaptic cleft. Unless it is reabsorbed there, as in DAT-KO rats, for example, it diffuses very slowly into the extracellular space
Tonic dopamine, on the other hand, is released directly into the extracellular space. Tonic dopamine therefore leads directly to extracellular dopamine, which is why “tonic dopamine” is sometimes used - although not entirely correctly - as a synonym for extracellular dopamine.
Tonically released dopamine from a nerve cell thus becomes extracellular dopamine. This can regulate the dopamine release of the transmitting neuron via autoreceptors of the presynapse. It is not only limited to the immediate presynapse, but can also control neighboring neurons.
5.2.4. Tonic and phasic dopamine between PFC and striatum
Both tonic and phasic dopaminergic signals originate from dopamine neurons in the substantia nigra pars compacta and VTA, both located in the midbrain. These innervate, among other areas, the entire dorsal to ventral region of the striatum and the PFC.24
Substantia nigra dopaminergic projections to the dorsal striatum influence voluntary movements and habit learning, and VTA projections to the ventral striatum influence reward and motivation.28
Tonic dopamine mediates the regulatory (inhibitory) control of the PFC on the ventral striatum, thus inhibiting the (phasic) activity of the striatum. In response to reward stimuli, the striatum fires phasically dopaminergically and activates dopaminergic postsynaptic receptors. Thus, tonic control is inhibitory and modulates excitatory phasic firing to reward stimuli.29
5.2.5. Tonic and phasic dopamine in the striatum
The ratio of phasic to tonic dopamine within the striatum varied with average ongoing firing frequency, and was generally higher in the nucleus accumbens than in the dorsolateral striatum. Blockade of DAT or D2 receptors predominantly enhanced tonic dopamine. Blockade of nicotinic acetylcholine receptors containing β2-subunits suppressed tonic dopamine. Suppression of tonic dopamine release increased the contrast between phasic and tonic dopamine.24
In ADHD sufferers, one study found decreased dopamine release at rest (“tonic dopamine”) and increased dopamine release during a flanker task (“phasic dopamine”) in the caudate nucleus. In other parts of the striatum, this tended to be similar but not significant. This supports the hypothesis of overactive DAT.30
To differentiate: tonic and phasic receptor types
The categorization of receptors into tonic and phasic receptors must be distinguished from phasic and tonic neurotransmitter release. The receptor categorization describes the reaction mode of receptors and has nothing to do with the release mode of neurotransmitters:
- Slowly adapting
- Continue to fire continuously in response to a constant stimulus
- Have only absolute sensitivity
- Existing stimulus increases frequency once
- Constant response (like on/off switch)
- Quickly adapting
- Decrease frequency after the start of constant stimulation quickly again
- Do not react to slowly increasing stimulus intensity
- Stimulation increases frequency by the rate of rise
- Dynamic response (like dimmer)
5.3. Coding of behavioral values by dopamine
Phasic dopamine encodes:
- The presence of an aversive or high-intensity stimulus32
Aversive stimuli cause dopamine output in only a few dopamine neurons.
Aversive or high-intensity stimuli evoked a three-phase sequence of activation-suppression-activation over a period of 40 to 700 ms:
- Startup phase: activation at short latencies (40-120 ms)
- Encodes the sensory intensity
- Middle phase: (between 150 and 250 ms)
- Codes the motivational value
- Activation during appetitive stimuli
- Suppression during aversive and neutral stimuli.
- Reward prediction error33
- Activity increased for 100 to 200 ms when reward or reward prediction stimulus is better than predicted
- Activity unchanged when events have same reward value as predicted
- Activity briefly dampened when events have lower reward value than predicted
- Codes the motivational value
- Late stage:
- Moderate “rebound” after strong suppression
- Strong activation by high reward is often followed by supression
- Startup phase: activation at short latencies (40-120 ms)
- The quantitative reward prediction error3435 dopamine neurons in the midbrain, and a subpopulation of dopamine neurons in the striatum, amygdala, and PFC
- Fire dopaminergically when a reward is higher than expected
- Remain unchanged if the reward corresponds to the expectation
- Decrease their dopaminergic activity when the reward is less than expected
- Learning a response strategy to reinforcement
- Occurs via phasic dopamine in the nucleus accumbens via D1 receptors22
5.3.1. Speed of dopamine level change in the brain encodes different behaviors.
The rate of dopamine level change in the brain encodes different behaviors.
Encode dopamine level changes
- In the 10-minute range: the strength of motivation and behavioral activation
- In seconds: the value of a future reward
- In the subsecond range: the search for the reward
Phasic dopamine encodes a stimulus/reward that exceeds expectation.34
5.3.2. 10-Minute range encodes strength of motivation and behavioral activation
Encode dopamine level changes
- In the 10-minute range: strength of motivation and behavioral activation
Tonic activations are mediated by changes in dopamine levels in the nucleus accumbens (part of the basal ganglia in the striatum, part of the mesolimbic system) when the changes in dopamine levels occur at a (slow) rate in the 10-minute range. Slow dopamine level changes over time units of 10 minutes correlate with reward rate, strength of motivation, and behavioral activity.3637
5.3.3. Seconds range encodes value of future reward
Phasic activations are mediated by changes in dopamine levels in the nucleus accumbens when the changes in dopamine levels occur at a (rapid) rate in the seconds range. Rapid (relative) dopamine level changes on a second-by-second basis mediate the valuation of a future reward. Thus, with value changes of dopamine in the seconds range, the value of an event that lies in the future is estimated and encoded.36 This is tangential to ADHD and stress symptoms that follow from the devaluation of more distant rewards.
5.3.4. Subsecond range activates a. reward search and b. movement
Even more short-term changes in dopamine levels in the range of fractions of a second (subseconds) were found in rats trained to respond to a signal upon which they could request sugar or cocaine. The corresponding signal triggered an extremely rapid increase in dopamine levels in animals trained in this way (time ranges below one second). Only in the animals trained in this way could a dopamine administration in the nucleus accumbens, which occurred at the appropriate speed, also trigger the search for the reward.3839
Specifically, other axons in the striatum respond to fast phasic dopamine signals to encode movement initiation.40
Distinguishing the effect of dopamine according to speed of level increase on the one hand (seconds and subseconds range) and changes in the direction of the absolute level dimension on the other hand (10-minute time measure) may explain why dopamine is relevant for (short-term) motivation and (long-term) learning simultaneously.
5.3.5. Does mesolimbic phasic DA encode the value of work?
One view proposes that mesolimbic phasic dopamine encodes the value of work required to achieve a goal, that is, the need to invest time and effort to obtain the reward.41 Dopamine levels increase only in response to signals prompting movement, but not in response to signals prompting rest, even if these indicate a similar future reward.42
5.3.6. Does acetylcholine switch between dopaminergic encoding of reward prediction error and learning?
Several reports show that dopamine encodes the reward prediction fallacy on the one hand and value signals on the other. It is possible that dopamine-receiving circuits can actively switch how they interpret dopamine. Circumstantial evidence suggests that acetylcholine, among others, may have such a switching function
Whereas dopamine cells respond to unexpected signals with phasic spike bursts, cholinergic interneurons in the striatum show short pauses of about 150 ms during which they do not fire, and do not scale with reward prediction error values
These pauses of cholinergic interneurons can be triggered by GABAergic neurons of the VTA as well as by “surprise” cells in the intralaminar thalamus. For example, GABA-releasing neurons of the VTA that project to the nucleus accumbens are able to inhibit cholinergic interneurons in the accumbens to enhance learning of stimuli and outcomes. It is possible that these pauses act as an association signal that enhances learning. During pauses in cholinergic interneurons, the absence of muscarinic blockade of synaptic plasticity appears to encode dopamine as a signal for learning. If the cholinergic interneurons fire, the release of dopamine terminals is controlled locally to influence ongoing behavioral performance. However, this is not yet certain.43444546
5.3.7. Different DA neurons in relation to aversive stimuli?
One study reported different dopaminergic neurons, one excited by rewards and inhibited by aversive stimuli and the other activated by both types of stimuli. The DA neurons that were excited by aversive stimuli or stimuli that predicted them were more likely to be found in the dorsolateral substantia nigra pars compacta, whereas neurons that were inhibited by this were more likely to be found ventromedially, including in the ventromedial VTA.47
5.3.8. Phasic DA encodes movement in the dorsal striatum, reward in the ventral striatum
One study found evidence that phasic dopamine in the dorsal striatum encodes movement and phasic dopamine in the ventral striatum (nucleus accumbens) encodes reward and motivation, respectively.48 This has been confirmed many times in the meantime.
5.3.9. Does dopamine encode the utility of a resource expenditure?
Berke43 sheds light on the hypothesis that dopamine encodes the utility of consuming a limited resource, viz
- economic (distribution of resources) and
- motivational (whether it is worth spending resources:49
Comment on Beeler et al
To the extent that Beeler et al hypothesize that obesity may result from a lack of motivation to exercise, with dopamine deficiency correlating with lack of motivation, it should be countered that ADHD, which is also associated with dopamine deficiency, is associated with the symptom of “always having to be active” and hyperactivity. Dopamine deficiency is thus not consistently causal for a “sedentary lifestyle that inhibits energy expenditure.”
The circuits within the striatum are organized hierarchically: The ventral striatum influences dopamine cells, which in turn project to the dorsal striatum. In primates, the ventromedial striatum consists of the cortex (shell), which receives limited input from the cortex, midbrain, and thalamus, and the nucleus (core). The shell dopaminergically influences the nucleus, the nucleus influences the central striatum, and the central striatum influences the dorsolateral striatum.5051 Thus, the decision to engage in work may simultaneously cause the specific, shorter movements required to be reinforced. Overall, however, dopamine provides “activating” signals - increasing the likelihood that a decision will be made - rather than “directional” signals indicating how resources should be used.
188.8.131.52. Dorsolateral striatum: DA encodes the resource movement
In Berke’s dorsolateral striatum, dopamine encodes the resource of movement, which is limited due to energy expenditure and the incompatibility of multiple actions at the same time.52 An increase in dopamine increases the likelihood that an individual will consider the energy expenditure for a movement to be worthwhile.5348495455 At the same time, if higher dopamine encodes a “movement is worthwhile,” there is a correlation between dopamine and movement itself, but it is not directly causal.
184.108.40.206. Dorsomedial striatum: DA encodes the resource of cognitive processes
In the dorsomedial striatum, dopamine encodes the resources of cognitive processes such as attention (which is limited by definition)56 and working memory according to Berke.57 Dopamine encodes attention to salient external cues. Conscious activation of cognitive control processes is effortful.58 Dopamine encodes - especially in the dorsomedial striatum59 - that it is worth the effort, for example, whether the effort of cognitively more demanding, model-based decision strategies is worth the effort.
Dopamine deficiency, on the other hand, causes such cues, which normally trigger orientation movements, to be neglected - as if they deserved less attention.60
Regulation of cognitive control also via dACC58
One study suggests that the dorsal ACC (dACC) increases its proportions of
- Reward Processing
- Performance monitoring
- cognitive control
- Action selection
solely by means of a single value, namely the evaluation of the expected value of control (EVC). The normative model of EVC presented integrates three critical factors
- the expected profit from a controlled process
- the amount of control that must be invested to achieve this gain
- the cost in terms of cognitive effort.
The ACC is controlled primarily glumatergic and GABAergic.
220.127.116.11. Nucleus accumbens: DA encodes the resource time
In the nucleus accumbens, dopamine encodes the resource time according to Berke. Some rewards require a long preparatory work of in detail unrewarded actions, e.g., searching for food. A decision to engage in such time-intensive work implies foregoing other beneficial ways of spending time. A high mesolimbic dopamine level encodes that it is worthwhile to engage in time-extensive, effortful work for a temporally distant goal. If mesolimbic dopamine levels decrease, interest in long-term reward decreases.
Simple actions with quick rewards do not require mesolimbic dopamine.61
Liu, Goel, Kaeser (2021): Spatial and temporal scales of dopamine transmission. Nat Rev Neurosci. 2021 Jun;22(6):345-358. doi: 10.1038/s41583-021-00455-7. PMID: 33837376; PMCID: PMC8220193. REVIEW ↥ ↥ ↥ ↥ ↥ ↥ ↥ ↥ ↥ ↥ ↥ ↥ ↥ ↥ ↥ ↥ ↥ ↥ ↥ ↥
Devoto, Sagheddu, Santoni, Flore, Saba, Pistis, Gessa (2020): Noradrenergic Source of Dopamine Assessed by Microdialysis in the Medial Prefrontal Cortex. Front Pharmacol. 2020 Sep 23;11:588160. doi: 10.3389/fphar.2020.588160. PMID: 33071798; PMCID: PMC7538903. ↥
Garris, Ciolkowski, Pastore, Wightman (1994): Efflux of dopamine from the synaptic cleft in the nucleus accumbens of the rat brain. J Neurosci. 1994 Oct;14(10):6084-93. doi: 10.1523/JNEUROSCI.14-10-06084.1994. PMID: 7931564; PMCID: PMC6577011. ↥
Zhou FW, Jin Y, Matta SG, Xu M, Zhou FM (2009): An ultra-short dopamine pathway regulates basal ganglia output. J Neurosci. 2009 Aug 19;29(33):10424-35. doi: 10.1523/JNEUROSCI.4402-08.2009. PMID: 19692618; PMCID: PMC3596265. ↥
Caillé, Dumartin, Bloch (1996): Ultrastructural localization of D1 dopamine receptor immunoreactivity in rat striatonigral neurons and its relation with dopaminergic innervation. Brain Res. 1996 Aug 19;730(1-2):17-31. doi: 10.1016/0006-8993(96)00424-6. PMID: 8883884. ↥
Uchigashima, Ohtsuka, Kobayashi, Watanabe (2016): Dopamine synapse is a neuroligin-2-mediated contact between dopaminergic presynaptic and GABAergic postsynaptic structures. Proc Natl Acad Sci U S A. 2016 Apr 12;113(15):4206-11. doi: 10.1073/pnas.1514074113. PMID: 27035941; PMCID: PMC4839454. ↥
Chen, Moran, Avshalumov, Rice (2006): Limited regulation of somatodendritic dopamine release by voltage-sensitive Ca channels contrasted with strong regulation of axonal dopamine release. J Neurochem. 2006 Feb;96(3):645-55. doi: 10.1111/j.1471-4159.2005.03519.x. PMID: 16405515. ↥
Brimblecombe, Gracie, Platt, Cragg (2015): Gating of dopamine transmission by calcium and axonal N-, Q-, T- and L-type voltage-gated calcium channels differs between striatal domains. J Physiol. 2015 Feb 15;593(4):929-46. doi: 10.1113/jphysiol.2014.285890. PMID: 25533038; PMCID: PMC4398530. ↥ ↥ ↥
Kaeser, Regehr (2014): Molecular mechanisms for synchronous, asynchronous, and spontaneous neurotransmitter release. Annu Rev Physiol. 2014;76:333-63. doi: 10.1146/annurev-physiol-021113-170338. Epub 2013 Nov 21. PMID: 24274737; PMCID: PMC4503208. REVIEW ↥
Mendez, Bourque, Fasano, Kortleven, Trudeau (2011): Somatodendritic dopamine release requires synaptotagmin 4 and 7 and the participation of voltage-gated calcium channels. J Biol Chem. 2011 Jul 8;286(27):23928-37. doi: 10.1074/jbc.M111.218032. PMID: 21576241; PMCID: PMC3129174. ↥
Zhang, Zhang, Liang, Siapas, Zhou, Dani (2009): Dopamine signaling differences in the nucleus accumbens and dorsal striatum exploited by nicotine. J Neurosci. 2009 Apr 1;29(13):4035-43. doi: 10.1523/JNEUROSCI.0261-09.2009. PMID: 19339599; PMCID: PMC2743099. ↥ ↥
Clark, Chiodo (1988): Electrophysiological and pharmacological characterization of identified nigrostriatal and mesoaccumbens dopamine neurons in the rat. Synapse. 1988;2(5):474-85. doi: 10.1002/syn.890020503. PMID: 2903568. ↥
Roessner, Rothenberger (2020): Neurochemie, S. 91, in Steinhausen, Rothenberger, Döpfner (Herausgeber): Handbuch ADHS; Grundlagen, Klinik, Therapie und Verlauf der Aufmerksamkeitsdefizit-Hyperaktivitätsstörung, Kohlhammer, unter Verweis auf Gainetdinov, Jones, Caron (1999): Functional hyperdopaminergia in dopamine transporter knock-out mice. Biol Psychiatry. 1999 Aug 1;46(3):303-11. doi: 10.1016/s0006-3223(99)00122-5. PMID: 10435196. REVIEW ↥
Zhang, Doyon, Clark, Phillips, Dani (2009): Controls of tonic and phasic dopamine transmission in the dorsal and ventral striatum. Mol Pharmacol. 2009 Aug;76(2):396-404. doi: 10.1124/mol.109.056317. PMID: 19460877; PMCID: PMC2713129. ↥ ↥ ↥ ↥
Pessiglione, Seymour, Flandin, Dolan, Frith (2006): Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature. 2006 Aug 31;442(7106):1042-5. doi: 10.1038/nature05051. PMID: 16929307; PMCID: PMC2636869. ↥
Keath, Iacoviello, Barrett, Mansvelder, McGehee (2007): Differential modulation by nicotine of substantia nigra versus ventral tegmental area dopamine neurons. J Neurophysiol. 2007 Dec;98(6):3388-96. doi: 10.1152/jn.00760.2007. PMID: 17942622. ↥
Gatzke-Kopp, Beauchaine (2007): Central nervous system substrates of impulsivity: Implications for the development of attention-deficit/hyperactivity disorder and conduct disorder. In: Coch, Dawson, Fischer ( Eds): Human behavior, learning, and the developing brain: Atypical development. New York: Guilford Press; 2007. pp. 239–263; 245 ↥
Badgaiyan, Sinha, Sajjad, Wack (2015): Attenuated Tonic and Enhanced Phasic Release of Dopamine in Attention Deficit Hyperactivity Disorder. PLoS One. 2015 Sep 30;10(9):e0137326. doi: 10.1371/journal.pone.0137326. PMID: 26422146; PMCID: PMC4589406. n = 44 ↥
Fiorillo, Song, Yun (2013): Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli. J Neurosci. 2013 Mar 13;33(11):4710-25. doi: 10.1523/JNEUROSCI.3883-12.2013. PMID: 23486944; PMCID: PMC3873404. ↥
Schultz (2019): Recent advances in understanding the role of phasic dopamine activity. F1000Res. 2019 Sep 24;8:F1000 Faculty Rev-1680. doi: 10.12688/f1000research.19793.1. PMID: 31588354; PMCID: PMC6760455. REVIEW ↥
Bayer, Glimcher )2005): Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron. 2005 Jul 7;47(1):129-41. doi: 10.1016/j.neuron.2005.05.020. PMID: 15996553; PMCID: PMC1564381. ↥
Hamid, Pettibone, Mabrouk, Hetrick, Schmidt, Van der Weele, Kennedy, Aragona, Berke (2016): Mesolimbic Dopamine Signals the Value of Work; Nat Neurosci. 2016 Jan; 19(1): 117–126; doi: 10.1038/nn.4173; PMCID: PMC4696912; NIHMSID: NIHMS733226 ↥ ↥
Hamid, Pettibone, Mabrouk, Hetrick, Schmidt, Vander Weele, Kennedy, Aragona, Berke (2016): Mesolimbic dopamine signals the value of work. Nat Neurosci. 2016 Jan;19(1):117-26. doi: 10.1038/nn.4173. PMID: 26595651; PMCID: PMC4696912. ↥
Syed, Grima, Magill, Bogacz, Brown, Walton (2016): Action initiation shapes mesolimbic dopamine encoding of future rewards. Nat Neurosci. 2016 Jan;19(1):34-6. doi: 10.1038/nn.4187. PMID: 26642087; PMCID: PMC4697363. ↥
Brown, Tan, O’Connor, Nikonenko, Muller, Lüscher (2012): Ventral tegmental area GABA projections pause accumbal cholinergic interneurons to enhance associative learning. Nature. 2012 Dec 20;492(7429):452-6. doi: 10.1038/nature11657. PMID: 23178810. ↥
Yamanaka, Hori, Minamimoto, Yamada, Matsumoto, Enomoto, Aosaki, Graybiel, Kimura (2018): Roles of centromedian parafascicular nuclei of thalamus and cholinergic interneurons in the dorsal striatum in associative learning of environmental events. J Neural Transm (Vienna). 2018 Mar;125(3):501-513. doi: 10.1007/s00702-017-1713-z. PMID: 28324169; PMCID: PMC5608633. ↥
Morris, Arkadir, Nevet, Vaadia, Bergman (2004): Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron. 2004 Jul 8;43(1):133-43. doi: 10.1016/j.neuron.2004.06.012. PMID: 15233923. ↥
Matsumoto, Hikosaka (2009): Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature. 2009 Jun 11;459(7248):837-41. doi: 10.1038/nature08028. Epub 2009 May 17. PMID: 19448610; PMCID: PMC2739096. ↥
Howe, Dombeck (2016): Rapid signalling in distinct dopaminergic axons during locomotion and reward. Nature. 2016 Jul 28;535(7613):505-10. doi: 10.1038/nature18942. PMID: 27398617; PMCID: PMC4970879. ↥ ↥
Beeler, Frazier, Zhuang (2012): Putting desire on a budget: dopamine and energy expenditure, reconciling reward and resources. Front Integr Neurosci. 2012 Jul 20;6:49. doi: 10.3389/fnint.2012.00049. PMID: 22833718; PMCID: PMC3400936. ↥ ↥
Haber, Fudge, McFarland (2000): Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci. 2000 Mar 15;20(6):2369-82. doi: 10.1523/JNEUROSCI.20-06-02369.2000. PMID: 10704511; PMCID: PMC6772499. ↥
Mazzoni, Hristova, Krakauer (2007): Why don’t we move faster? Parkinson’s disease, movement vigor, and implicit motivation. J Neurosci. 2007 Jul 4;27(27):7105-16. doi: 10.1523/JNEUROSCI.0264-07.2007. PMID: 17611263; PMCID: PMC6794577. ↥
Shadmehr, Reppert, Summerside, Yoon, Ahmed (2019): Movement Vigor as a Reflection of Subjective Economic Utility. Trends Neurosci. 2019 May;42(5):323-336. doi: 10.1016/j.tins.2019.02.003. PMID: 30878152; PMCID: PMC6486867. ↥
Treadway, Buckholtz, Cowan, Woodward, Li, Ansari, Baldwin, Schwartzman, Kessler, Zald )2012): Dopaminergic mechanisms of individual differences in human effort-based decision-making. J Neurosci. 2012 May 2;32(18):6170-6. doi: 10.1523/JNEUROSCI.6459-11.2012. PMID: 22553023; PMCID: PMC3391699. ↥
Anderson, Kuwabara, Wong, Gean, Rahmim, Brašić, George, Frolov, Courtney, Yantis (2016): The Role of Dopamine in Value-Based Attentional Orienting. Curr Biol. 2016 Feb 22;26(4):550-5. doi: 10.1016/j.cub.2015.12.062. PMID: 26877079; PMCID: PMC4767677. ↥
Chatham, Frank, Badre (2014): Corticostriatal output gating during selection from working memory. Neuron. 2014 Feb 19;81(4):930-42. doi: 10.1016/j.neuron.2014.01.002. PMID: 24559680; PMCID: PMC3955887. ↥
Shenhav, Botvinick, Cohen (2013): The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron. 2013 Jul 24;79(2):217-40. doi: 10.1016/j.neuron.2013.07.007. PMID: 23889930; PMCID: PMC3767969. REVIEW ↥ ↥
Aarts, Roelofs, Franke, Rijpkema, Fernández, Helmich, Cools (2010): Striatal dopamine mediates the interface between motivational and cognitive control in humans: evidence from genetic imaging. Neuropsychopharmacology. 2010 Aug;35(9):1943-51. doi: 10.1038/npp.2010.68. Epub 2010 May 12. PMID: 20463658; PMCID: PMC3055632. ↥
Marshall, Levitan, Stricker (1976): Activation-induced restoration of sensorimotor functions in rats with dopamine-depleting brain lesions. J Comp Physiol Psychol. 1976 Jun;90(6):536-46. doi: 10.1037/h0077230. PMID: 8470. ↥
Nicola (2010): The flexible approach hypothesis: unification of effort and cue-responding hypotheses for the role of nucleus accumbens dopamine in the activation of reward-seeking behavior. J Neurosci. 2010 Dec 8;30(49):16585-600. doi: 10.1523/JNEUROSCI.3958-10.2010. PMID: 21147998; PMCID: PMC3030450. ↥