Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).

Article  CAS  PubMed  Google Scholar 

Montague, P. R., Dayan, P. & Sejnowski, T. J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16, 1936–1947 (1996).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Bayer, H. M. & Glimcher, P. W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Eshel, N. et al. Arithmetic and local circuitry underlying dopamine prediction errors. Nature 525, 243–246 (2015).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Eshel, N., Tian, J., Bukwich, M. & Uchida, N. Dopamine neurons share common response function for reward prediction error. Nat. Neurosci. 19, 479–486 (2016).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Steinberg, E. E. et al. A causal link between prediction errors, dopamine neurons and learning. Nat. Neurosci. 16, 966–973 (2013).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Chang, C. Y. et al. Brief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors. Nat. Neurosci. 19, 111–116 (2016).

Article  CAS  PubMed  Google Scholar 

Reynolds, J. N. J. & Wickens, J. R. Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw. 15, 507–521 (2002).

Article  PubMed  Google Scholar 

Morita, K., Morishima, M., Sakai, K. & Kawaguchi, Y. Reinforcement learning: computing the temporal difference of values via distinct corticostriatal pathways: (Trends in Neurosciences 35, 457–467; 2012). Trends Neurosci. 40, 453 (2017).

Article  CAS  PubMed  Google Scholar 

Watabe-Uchida, M., Eshel, N. & Uchida, N. Neural circuitry of reward prediction error. Annu. Rev. Neurosci. 40, 373–394 (2017).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Starkweather, C. K. & Uchida, N. Dopamine signals as temporal difference errors: recent advances. Curr. Opin. Neurobiol. 67, 95–105 (2021).

Article  CAS  PubMed  Google Scholar 

Howe, M. W., Tierney, P. L., Sandberg, S. G., Phillips, P. E. M. & Graybiel, A. M. Prolonged dopamine signalling in striatum signals proximity and value of distant rewards. Nature 500, 575–579 (2013).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Niv, Y. Neuroscience: dopamine ramps up. Nature 500, 533–535 (2013).

Article  CAS  PubMed  Google Scholar 

Berke, J. D. What does dopamine mean? Nat. Neurosci. 21, 787–793 (2018).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Hamid, A. A. et al. Mesolimbic dopamine signals the value of work. Nat. Neurosci. 19, 117–126 (2016).

Article  CAS  PubMed  Google Scholar 

Mohebi, A. et al. Publisher correction: dissociable dopamine dynamics for learning and motivation. Nature 571, E3 (2019).

Article  CAS  PubMed  Google Scholar 

Krausz, T. A., Comrie, A. E., Frank, L. M., Daw, N. D. & Berke, J. D. Dual credit assignment processes underlie dopamine signals in a complex spatial environment. Neuron 111, 3465–3478 (2023).

Article  CAS  PubMed  Google Scholar 

Hamilos, A. E. et al. Slowly evolving dopaminergic activity modulates the moment-to-moment probability of reward-related self-timed movements. eLife 10, e62583 (2021).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Collins, A. L. et al. Dynamic mesolimbic dopamine signaling during action sequence learning and expectation violation. Sci. Rep. 6, 20231 (2016).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Gershman, S. J. Dopamine ramps are a consequence of reward prediction errors. Neural Comput. 26, 467–471 (2014).

Article  PubMed  Google Scholar 

Kim, H. R. et al. A unified framework for dopamine signals across timescales. Cell 183, 1600–1616 (2020).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Mikhael, J. G., Kim, H. R., Uchida, N. & Gershman, S. J. The role of state uncertainty in the dynamics of dopamine. Curr. Biol. 32, 1077–1087 (2022).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Kato, A. & Morita, K. Forgetting in reinforcement learning links sustained dopamine signals to motivation. PLoS Comput. Biol. 12, e1005145 (2016).

Article  PubMed  PubMed Central  Google Scholar 

Beron, C. C., Neufeld, S. Q., Linderman, S. W. & Sabatini, B. L. Mice exhibit stochastic and efficient action switching during probabilistic decision making. Proc. Natl Acad. Sci. USA 119, e2113961119 (2022).

Article  PubMed  PubMed Central  Google Scholar 

Niv, Y. et al. Reinforcement learning in multidimensional environments relies on attention mechanisms. J. Neurosci. 35, 8145–8157 (2015).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Ito, M. & Doya, K. Validation of decision-making models and analysis of decision variables in the rat basal ganglia. J. Neurosci. 29, 9861–9874 (2009).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Lloyd, K. & Dayan, P. Tamping ramping: algorithmic, implementational, and computational explanations of phasic dopamine signals in the accumbens. PLoS Comput. Biol. 11, e1004622 (2015).

Article  PubMed  PubMed Central  Google Scholar 

Hamid, A. A., Frank, M. J. & Moore, C. I. Wave-like dopamine dynamics as a mechanism for spatiotemporal credit assignment. Cell 184, 2733–2749 (2021).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Guru, A., Seo, C., Kullakanda, D. S., Schaffer, J. A. & Warden, M. R. Ramping activity in midbrain dopamine neurons signifies the use of a cognitive map. Preprint at bioRxiv https://doi.org/10.1101/2020.05.21.108886 (2020).

Amo, R. et al. A gradual temporal shift of dopamine responses mirrors the progression of temporal difference error in machine learning. Nat. Neurosci. 25, 1082–1092 (2022).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Ljungberg, T., Apicella, P. & Schultz, W. Responses of monkey dopamine neurons during learning of behavioral reactions. J. Neurophysiol. 67, 145–163 (1992).