Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Adaptive learning by a target‐tracking system

Adaptive learning by a target‐tracking system Purpose – The purpose of this paper is to report upon research into developing a biologically inspired target‐tracking system (TTS) capable of acquiring quality images of a known target type for a robotic inspection application. Design/methodology/approach – The approach used in the design of the TTS hearkens back to the work on adaptive learning by Oliver Selfridge and Chris J.C.H. Watkins and the work on the classification of objects by Zdzislaw Pawlak during the 1980s in an approximation space‐based form of feedback during learning. Also, during the 1980s, it was Ewa Orlowska who called attention to the importance of approximation spaces as a formal counterpart of perception. This insight by Orlowska has been important in working toward a new form of adaptive learning useful in controlling the behaviour of machines to accomplish system goals. The adaptive learning algorithms presented in this paper are strictly temporal difference methods, including Q‐learning, sarsa, and the actor‐critic method. Learning itself is considered episodic. During each episode, the equivalent of a Tinbergen‐like ethogram is constructed. Such an ethogram provides a basis for the construction of an approximation space at the end of each episode. The combination of episodic ethograms and approximation spaces provides an extremely effective means of feedback useful in guiding learning during the lifetime of a robotic system such as the TTS reported in this paper. Findings – It was discovered that even though the adaptive learning methods were computationally more expensive than the classical algorithm implementations, they proved to be more effective in a number of cases, especially in noisy environments. Originality/value – The novelty associated with this work is the introduction of an approach to adaptive adaptive learning carried out within the framework of ethology‐based approximation spaces to provide performance feedback during the learning process. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png International Journal of Intelligent Computing and Cybernetics Emerald Publishing

Adaptive learning by a target‐tracking system

Download PDF
23 pages

Loading next page...
 
/lp/emerald-publishing/adaptive-learning-by-a-target-tracking-system-gwnuxsaILC
Publisher
Emerald Publishing
Copyright
Copyright © 2008 Emerald Group Publishing Limited. All rights reserved.
ISSN
1756-378X
DOI
10.1108/17563780810857121
Publisher site
See Article on Publisher Site

Abstract

Purpose – The purpose of this paper is to report upon research into developing a biologically inspired target‐tracking system (TTS) capable of acquiring quality images of a known target type for a robotic inspection application. Design/methodology/approach – The approach used in the design of the TTS hearkens back to the work on adaptive learning by Oliver Selfridge and Chris J.C.H. Watkins and the work on the classification of objects by Zdzislaw Pawlak during the 1980s in an approximation space‐based form of feedback during learning. Also, during the 1980s, it was Ewa Orlowska who called attention to the importance of approximation spaces as a formal counterpart of perception. This insight by Orlowska has been important in working toward a new form of adaptive learning useful in controlling the behaviour of machines to accomplish system goals. The adaptive learning algorithms presented in this paper are strictly temporal difference methods, including Q‐learning, sarsa, and the actor‐critic method. Learning itself is considered episodic. During each episode, the equivalent of a Tinbergen‐like ethogram is constructed. Such an ethogram provides a basis for the construction of an approximation space at the end of each episode. The combination of episodic ethograms and approximation spaces provides an extremely effective means of feedback useful in guiding learning during the lifetime of a robotic system such as the TTS reported in this paper. Findings – It was discovered that even though the adaptive learning methods were computationally more expensive than the classical algorithm implementations, they proved to be more effective in a number of cases, especially in noisy environments. Originality/value – The novelty associated with this work is the introduction of an approach to adaptive adaptive learning carried out within the framework of ethology‐based approximation spaces to provide performance feedback during the learning process.

Journal

International Journal of Intelligent Computing and Cybernetics – Emerald Publishing

Published: Mar 28, 2008

Keywords: Behaviour; Biology; Robots; Tracking; Intelligent agents

References

  • Convergence of stochastic iterative dynamic programming algorithms
    Jaakkola, T.; Jordan, M.I.; Singh, S.P.
  • Reinforcement learning: a survey
    Kaelbling, L.P.; Littman, M.L.; Moore, A.W.
  • CC5X C Compiler
    Knudsen, B.
  • Linux RedHat 9.0 Manuals
    Linux
  • Learning with ALiCE II
    Lockery, D.
  • MPLAB Integrated Development Environment
    Microchip Technology Inc.
  • Semantics of Vague Concepts. Applications of Rough Sets
    Orlowska, E.
  • Semantics of Vague Concepts
    Orlowska, E.
  • Classification of Objects by Means of Attributes
    Pawlak, Z.
  • Rough ethology: toward a biologically‐inspired study of collective behaviour in intelligent systems with approximation spaces
    Peters, J.F.
  • Biologically‐inspired approximate adaptive learning control strategies: a rough set approach
    Peters, J.F.; Henry, C.; Gunderson, D.S.
  • Rough Ethograms: study of intelligent system behaviour
    Peters, J.F.; Henry, C.; Ramanna, S.
  • Line‐crawling bots that inspect electric power transmission line equipment
    Peters, J.F.; Borkowski, M.; Henry, C.; Lockery, D.; Gunderson, D.; Ramanna, S.
  • Numerical Recipes in C++: The Art of Scientific Computing
    Press, W.H.; Teukolsky, S.A.; Vetterling, W.T.; Flannery, B.P.
  • On‐line Q‐learning using connectionist systems
    Rummery, G.A.; Niranjan, M.
  • Some themes and primitives in ill‐defined systems
    Selfridge, O.G.
  • Reinforcement learning with replacing eligibility traces
    Singh, S.P.; Sutton, R.S.
  • Convergence results for single‐step on‐policy reinforcement‐learning algorithms
    Singh, S.P.; Jaakkola, T.; Littman, M.L.; Szepesvari, C.
  • Generalization in reinforcement learning: successful examples using sparse coarse coding
    Sutton, R.S.
  • Reinforcement Learning: An Introduction
    Sutton, R.S.; Barto, A.G.
  • Neuronlike adaptive elements that can solve difficult learning problems
    Sutton, R.S.; Barto, A.G.; Anderson, C.W.
  • On aims and methods of ethology
    Tinbergen, N.
  • Learning from delayed rewards
    Watkins, C.J.C.H.
  • Technical note: Q‐learning
    Watkins, C.J.C.H.; Dayan, P.