Eryilmaz H, Rodriguez-Thompson A, Tanner AS, et al. Neural determinants of human goal-directed vs. habitual action control and their relation to trait motivation. Sci Rep. 2017;7(1):6002.

Article  Google Scholar 

Choi K, Piasini E, Díaz-Hernández E, et al. Distributed processing for value-based choice by prelimbic circuits targeting anterior-posterior dorsal striatal subregions in male mice. Nat Commun. 2023;14(1):1920.

Article  Google Scholar 

Villet M, Reynaud-Bouret P, Poitreau J, et al. Coding dynamics of the striatal networks during learning. bioRxiv, 2023: 2023.07. 24.550305.

Briones BA, Pitcher MN, Fleming WT, et al. Perineuronal nets in the dorsomedial striatum contribute to behavioral dysfunction in mouse models of excessive repetitive behavior. Biol Psychiatry Global Open Sci. 2022;2(4):460–9.

Article  Google Scholar 

Vandaele Y, Ottenheimer DJ, Janak PH. Dorsomedial striatal activity tracks completion of behavioral sequences in rats. Eneuro. 2021;8(6).

Heneman RL. Strategic reward management: design, implementations, and evaluation. IAP. 2002.

Randløv J, Alstrøm P. Learning to drive a bicycle using reinforcement learning and shaping. ICML. 1998;98:463–71.

Google Scholar 

Xu ZX, Chen XL, Cao L, et al. A study of count-based exploration and bonus for reinforcement learning. In: 2nd International Conference on Cloud Computing and Big Data Analysis (ICCCBDA). IEEE; 2017. p. 425–9.

Baird LC. Reinforcement learning in continuous time: advantage updating. In: International Conference on Neural Networks (ICNN’94), vol. 4. IEEE; 1994. p. 2448–53.

Cho H, Oh P, Park J, et al. Fa3c: Fpga-accelerated deep reinforcement learning. In: Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems. 2019. p. 499–513.

Liu Y, Halev A, Liu X. Policy learning with constraints in model-free reinforcement learning: a survey. In: The 30th International Joint Conference on Artificial Intelligence (IJCAI). 2021.

Yang Y, Jiang Y, Liu Y, et al. Model-free safe reinforcement learning through neural barrier certificate. IEEE Robot Autom Lett. 2023;8(3):1295–302.

Article  Google Scholar 

Sutton RS, Barto AG. Reinforcement learning: an introduction. MIT Press; 2018.

Puterman ML. Markov decision processes. Handbooks Oper Res Manage Sci. 1990;2:331–434.

Article  MathSciNet  MATH  Google Scholar 

Wang X, Wang L, Dong C, et al. An online deep reinforcement learning-based order recommendation framework forrider-centered food delivery system. IEEE Trans Intell Transport Syst. 2023.

Xin X, Tu Y, Stojanovic V, et al. Online reinforcement learning multiplayer non-zero sum games of continuous-time Markov jump linear systems. Appl Math Comput. 2022;412:126537.

Article  MathSciNet  MATH  Google Scholar 

Dogru O, Wieczorek N, Velswamy K, et al. Online reinforcement learning for a continuous space system with experimental validation. J Process Control. 2021;104:86–100.

Article  Google Scholar 

Prudencio RF, Maximo MROA, Colombini EL. A survey on offline reinforcement learning: taxonomy, review, and open problems. IEEE Trans Neural Netw Learn Syst. 2023.

Rome S, Chen T, Kreisel M, et al. Lessons on off-policy methods from a notification component of a chatbot. Mach Learn. 2021;110(9):2577–602.

Article  MathSciNet  MATH  Google Scholar 

Cunningham P, Cord M, Delany SJ. Supervised learning. Machine learning techniques for multimedia. Berlin, Heidelberg: Springer; 2008. p. 21–49.

Learned-Miller EG. Introduction to supervised learning. I: Department of Computer Science, University of Massachusetts; 2014.p. 3.

Andrychowicz M, Raichuk A, Stańczyk P, et al. What matters in on-policy reinforcement learning? A large-scale empirical study. arXiv:2006.05990 [Preprint]. 2020. Available from: http://arxiv.org/abs/2006.05990.

Liu Y, Halev A, Liu X. Policy learning with constraints in model-free reinforcement learning: a survey. In: The 30th International Joint Conference on Artificial Intelligence (IJCAI). 2021.

De Asis K, Hernandez-Garcia J, Holland G, et al. Multi-step reinforcement learning: a unifying algorithm. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32(1). 2018.

Witty S, Lee JK, Tosch E, et al. Measuring and characterizing generalization in deep reinforcement learning. Appl AI Lett. 2021;2(4):e45.

Article  Google Scholar 

Zhang J, Kim J, O’Donoghue B, et al. Sample efficient reinforcement learning with REINFORCE. Proc AAAI Conf Artif Intell. 2021;35(12):10887–95.

Google Scholar 

Memarian F, Goo W, Lioutikov R, et al. Self-supervised online reward shaping in sparse-reward environments. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE; 2021. p. 2369–75.

Zou H, Ren T, Yan D, et al. Reward shaping via meta-learning. arXiv:1901.09330 [Preprint]. 2019. Available from: http://arxiv.org/abs/1901.09330.

Anca M, Studley M, Hansen M, et al. Effects of reward shaping on curriculum learning in goal conditioned tasks. arXiv:2206.02462 [Preprint]. 2022. Available from: http://arxiv.org/abs/2206.02462.

ElSayed-Aly I, Feng L. Logic-based reward shaping for multi-agent reinforcement learning. arXiv:2206.08881 [Preprint]. 2022. Available from: http://arxiv.org/abs/2206.08881.

Sun H, Han L, Yang R, et al. Exploiting reward shifting in value-based deep RL. arXiv:2209.07288 [Preprint]. 2022. Available from: http://arxiv.org/abs/2209.07288.

Ng AY, Harada D, Russell S. Policy invariance under reward transformations: theory and application to reward shaping. ICML. 1999;99:278–87.

Google Scholar 

Devlin SM, Kudenko D. Dynamic potential-based reward shaping. In: 11th International Conference on Autonomous Agents and Multiagent Systems. IFAAMAS; 2012. p. 433–40.

Amodei D, Olah C, Steinhardt J, et al. Concrete problems in AI safety. arXiv:1606.06565 [Preprint]. 2016. Available from: http://arxiv.org/abs/1606.06565.

Grzes M. Reward shaping in episodic reinforcement learning. 2017.

Bellemare M, Srinivasan S, Ostrovski G, et al. Unifying count-based exploration and intrinsic motivation. Adv Neural Inf Process Syst. 2016;29.

Dong K, Wang Y, Chen X, et al. Q-learning with UCB exploration is sample efficient for infinite-horizon mdp. arXiv:1901.09311 [Preprint]. 2019. Available from: http://arxiv.org/abs/1901.09311.

Jaegle A, Mehrpour V, Rust N. Visual novelty, curiosity, and intrinsic reward in machine learning and the brain. Curr Opin Neurobiol. 2019;58:167–74.

Article  Google Scholar 

Strehl AL, Littman ML. An analysis of model-based interval estimation for Markov decision processes. J Comput Syst Sci. 2008;74(8):1309–31.

Article  MathSciNet  MATH  Google Scholar 

Bigazzi R, Landi F, Cascianelli S, et al. Focus on impact: indoor exploration with intrinsic motivation. IEEE Robot Autom Lett. 2022;7(2):2985–92.

Article  Google Scholar 

Honda J, Takemura A. An asymptotically optimal bandit algorithm for bounded support models. COLT. 2010:67–79.

Brafman RI, Tennenholtz M. R-max-a general polynomial time algorithm for near-optimal reinforcement learning. J Mach Learn Res. 2002;3(Oct):213–31.

Yu JY, Mannor S, Shimkin N. Markov decision processes with arbitrary reward processes. Math Oper Res. 2009;34(3):737–57.

Article  MathSciNet  MATH  Google Scholar 

Yao Y, Xiao L, An Z, et al. Sample efficient reinforcement learning via model-ensemble exploration and exploitation. In: International Conference on Robotics and Automation (ICRA). IEEE; 2021.p. 4202–8.

Burda Y, Edwards H, Storkey A, et al. Exploration by random network distillation. arXiv:1810.12894 [Preprint]. 2018. Available from: http://arxiv.org/abs/1810.12894.

Subramanian K, Isbell CL Jr, Thomaz AL. Exploration from demonstration for interactive reinforcement learning. In: Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems. 2016. p. 447–56.

Pathak D, Agrawal P, Efros AA, et al. Curiosity-driven exploration by self-supervised prediction. Int Conf Mach Learn. PMLR. 2017:2778–2787.

Ryan RM, Deci EL. Intrinsic and extrinsic motivations: classic definitions and new directions. Contemp Educ Psychol. 2000;25(1):54–67.

Article  Google Scholar 

Manoury A, Buche C. Chime: an adaptive hierarchical representation for continuous intrinsically motivated exploration. In: 2019 Third IEEE International Conference on Robotic Computing (IRC). IEEE; 2019. p. 167–70.

Gordon G. Infant-inspired intrinsically motivated curious robots. Curr Opin Behav Sci. 2020;35:28–34.

Article  Google Scholar 

Hellman RB, Tekin C, van der Schaar M, et al. Functional contour-following via haptic perception and reinforcement learning. IEEE Trans Haptics. 2017;11(1):61–72.

Article  Google Scholar 

D’Eramo C, Cini A, Restelli M. Exploiting action-value uncertainty to drive exploration in reinforcement learning. In: International Joint Conference on Neural Networks (IJCNN). IEEE; 2019.p. 1–8.

Osband I, Van Roy B, Russo DJ, et al. Deep exploration via randomized value functions. J Mach Learn Res. 2019;20(124):1–62.

MathSciNet  MATH  Google Scholar 

Klyubin AS, Polani D, Nehaniv CL. All else being equal be empowered. In: European Conference on Artificial Life. Berlin, Heidelberg: Springer; 2005. p. 744–53.

Rezende D, Mohamed S. Variational inference with normalizing flows. Int Conf Mach Learn. PMLR. 2015:1530–8.

Schmidhuber J. A possibility for implementing curiosity and boredom in model-building neural controllers. Int Conf Simul Adapt Behav. 1991:222–7.

Gottlieb J, Oudeyer PY, Lopes M, et al. Information-seeking, curiosity, and attention: computational and neural mechanisms. Trends Cogn Sci. 2013;17(11):585–93.

Article  Google Scholar 

Schmidhuber J. Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts. Connect Sci. 2006;18(2):173–87.

Article  Google Scholar 

Stadie B C, Levine S, Abbeel P. Incentivizing exploration in reinforcement learning with deep predictive models. arXiv:1507.00814 [Preprint]. 2015. Available from: http://arxiv.org/abs/1507.00814.

Parisi S, Dean V, Pathak D, et al. Interesting object, curious agent: learning task-agnostic exploration. Adv Neural Inf Process Syst. 2021;34:20516–30.

Google Scholar 

Raileanu R, Rocktäschel T. Ride: rewarding impact-driven exploration for procedurally-generated environments. arXiv:2002.12292 [Preprint]. 2020. Available from: http://arxiv.org/abs/2002.12292.

Turner KM, Svegborn A, Langguth M, et al. Opposing roles of the dorsolateral and dorsomedial striatum in the acquisition of skilled action sequencing in rats. J Neurosci. 2022;42(10):2039–51.

Article  Google Scholar 

Kang S, Hong SI, Lee J, et al. Activation of astrocytes in the dorsomedial striatum facilitates transition from habitual to goal-directed reward-seeking behavior. Biol Psychiat. 2020;88(10):797–808.

Article  Google Scholar 

Gremel CM, Costa RM. Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions. Nat Commun. 2013;4(1):2264.

Rengarajan D, Vaidya G, Sarvesh A, et al. Reinforcement learning with sparse rewards using guidance from offline demonstration. arXiv:2202.04628 [Preprint]. 2022. Available from: http://arxiv.org/abs/2202.04628.