Loading next page...
/lp/emerald-publishing/a-survey-of-inverse-reinforcement-learning-techniques-yyIjG0DTu4
- Publisher
- Emerald Publishing
- Copyright
- Copyright © 2012 Emerald Group Publishing Limited. All rights reserved.
- ISSN
- 1756-378X
- DOI
- 10.1108/17563781211255862
- Publisher site
- See Article on Publisher Site
Abstract
Purpose – This purpose of this paper is to provide an overview of the theoretical background and applications of inverse reinforcement learning (IRL). Design/methodology/approach – Reinforcement learning (RL) techniques provide a powerful solution for sequential decision making problems under uncertainty. RL uses an agent equipped with a reward function to find a policy through interactions with a dynamic environment. However, one major assumption of existing RL algorithms is that reward function, the most succinct representation of the designer's intention, needs to be provided beforehand. In practice, the reward function can be very hard to specify and exhaustive to tune for large and complex problems, and this inspires the development of IRL, an extension of RL, which directly tackles this problem by learning the reward function through expert demonstrations. In this paper, the original IRL algorithms and its close variants, as well as their recent advances are reviewed and compared. Findings – This paper can serve as an introduction guide of fundamental theory and developments, as well as the applications of IRL. Originality/value – This paper surveys the theories and applications of IRL, which is the latest development of RL and has not been done so far.
Journal
International Journal of Intelligent Computing and Cybernetics – Emerald Publishing
Published: Aug 17, 2012
Keywords: Inverse reinforcement learning; Reward function; Reinforcement learning; Artificial intelligence; Learning methods
Recommended Articles
Loading...
References
-
Apprenticeship learning via inverse reinforcement learningAbbeel, P.; Ng, A.
-
Autonomous helicopter aerobatics through apprenticeship learningAbbeel, P.; Coates, A.; Ng, A.
-
Autonomous autorotation of an RC helicopterAbbeel, P.; Coates, A.; Hunter, T.; Ng, A.
-
An application of reinforcement learning to aerobatic helicopter flightAbbeel, P.; Coates, A.; Quigley, M.; Ng, A.
-
Apprenticeship learning for motion planning with application to parking lot navigationAbbeel, P.; Dolgov, D.; Ng, A.; Thrun, S.
-
Natural gradient works efficiently in learningAmari, S.
-
A survey of robot learning from demonstrationArgall, B.; Chernova, S.; Veloso, M.; Browning, B.
-
Robot learning from demonstrationAtkeson, C.; Schaal, S.
-
Apprenticeship learning about multiple intentionsBabes, M.; Marivate, V.; Littman, M.; Subramanian, K.
-
Bootstrapping apprenticeship learningBoularias, A.; Chaib‐Draa, B.
-
Relative entropy inverse reinforcement learningBoularias, A.; Kober, J.; Peters, J.
-
User simulation in dialogue systems using inverse reinforcement learningChandramohan, S.; Geist, M.; Lefevre, F.; Pietquin, O.
-
Inverse reinforcement learning in partially observable environmentsChoi, J.; Kim, K.
-
A mobile robot that understands pedestrian spatial behaviorsChung, S.; Huang, H.
-
Learning for control from multiple demonstrationsCoates, A.; Abbeel, P.; Ng, A.
-
Selecting operator queries using expected myopic gainCohn, R.; Maxim, M.; Durfee, E.; Singh, S.
-
Bayesian multitask inverse reinforcement learningDimitrakakis, C.; Rothkopf, C.
-
The expected‐utility hypothesis and the measurability of utilityFriedman, M.; Savage, L.
-
Donut as I do: learning from failed demonstrationsGrollman, D.; Billard, A.
-
Human‐to‐robot skill transfer using the spore approximationGrudic, G.; Lawrence, P.
-
Learning to navigate through crowded environmentsHenry, P.; Vollmer, C.; Ferris, B.; Fox, D.
-
Solving a huge number of similar tasks: a combination of multi‐task learning and a hierarchical Bayesian approachHeskes, T.
-
Information theory and statistical mechanicsJaynes, E.
-
Reinforcement learning: a surveyKaelbling, L.; Littman, M.; Moore, A.
-
Inverse reinforcement learning with PI 2Kalakrishnan, M.; Theodorou, E.; Schaal, S.
-
Decisions with Multiple Objectives: Preferences and Value TradeoffsKeeney, R.; Raiffa, H.
-
Imitation and reinforcement learning, practical algorithms for motor primitives in roboticsKober, J.; Peters, J.
-
Hierarchical apprenticeship learning with application to quadruped locomotionKolter, J.; Abbeel, P.; Ng, A.
-
Conditional random fields: probabilistic models for segmenting and labeling sequence dataLafferty, J.; McCallum, A.; Pereira, F.
-
Learning behavior styles with inverse reinforcement learningLee, S.; Popovi, Z.
-
Active learning for reward estimation in inverse reinforcement learningLopes, M.; Melo, F.; Montesano, L.
-
Robot self‐initiative and personalization by learning through repeated interactionsMason, M.; Lopes, M.
-
Learning from demonstration using MDP induced metricsMelo, F.; Lopes, M.
-
A survey of POMDP solution techniquesMurphy, K.
-
Apprenticeship learning using inverse reinforcement learning and gradient methodsNeu, G.; Szepesvári, C.
-
Algorithms for inverse reinforcement learningNg, A.; Russell, S.
-
Efficient training of artificial neural networks for autonomous navigationPomerleau, D.
-
Markov Decision Processes: Discrete Stochastic Dynamic ProgrammingPuterman, M.
-
Inverse reinforcement learning with Gaussian processQiao, Q.; Beling, P.
-
Bayesian inverse reinforcement learningRamachandran, D.; Amir, E.
-
Maximum margin planningRatliff, N.; Bagnell, J.; Zinkevich, M.
-
Learning to search: functional gradient techniques for imitation learningRatliff, N.; Silver, D.; Bagnell, J.
-
An empirical Bayes approach to statisticsRobbins, H.
-
Preference elicitation and inverse reinforcement learningRothkopf, C.; Dimitrakakis, C.
-
Learning agents for uncertain environments (extended abstract)Russell, S.
-
Is imitation learning the route to humanoid robots?Schaal, S.
-
Modified reward function on abstract features in inverse reinforcement learningShen‐yi, C.; Hui, Q.; Jia, F.; Zhuo‐jun, J.; Miao‐liang, Z.; Springer
-
Inverse reinforcement learning with evaluationSilva, V.; Costa, A.; Lima, P.
-
Learning from demonstration for autonomous navigation in complex unstructured terrainSilver, D.; Bagnell, J.; Stentz, A.
-
Perceptual interpretation for autonomous navigation through dynamic imitation learningSilver, D.; Bagnell, J.; Stentz, A.
-
The optimal control of partially observable Markov processesSondik, E.
-
Introduction to Reinforcement LearningSutton, R.; Barto, A.
-
A game‐theoretic approach to apprenticeship learningSyed, U.; Schapire, R.
-
Parameterized maneuver learning for autonomous helicopter flightTang, J.; Singh, A.; Goehausen, N.; Abbeel, P.
-
Reinforcement learning of motor skills in high dimensions: a path integral approachTheodorou, E.; Buchli, J.; Schaal, S.
-
Enabling environment design via active indirect elicitationZhang, H.; Parkes, D.
-
Maximum entropy inverse reinforcement learningZiebart, B.; Maas, A.; Bagnell, J.; Dey, A.
@Phil_Robichaud
@deepthiw
@JoseServera