Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Contact-consistent visual object pose estimation for contact-rich robotic manipulation tasks

Contact-consistent visual object pose estimation for contact-rich robotic manipulation tasks The purpose of this paper is to estimate the contact-consistent object poses during contact-rich manipulation tasks based only on visual sensors.Design/methodology/approachThe method follows a four-step procedure. Initially, the raw object poses are retrieved using the available object pose estimation method and filtered using Kalman filter with nominal model; second, a group of particles are randomly generated for each pose and evaluated the corresponding object contact state using the contact simulation software. A probability guided particle averaging method is proposed to balance the accuracy and safety issues; third, the independently estimated contact states are fused in a hidden Markov model to remove the abnormal contact state observations; finally, the object poses are refined by averaging the contact state consistent particles.FindingsThe experiments are performed to evaluate the effectiveness of the proposed methods. The results show that the method can achieve smooth and accurate pose estimation results and the estimated contact states are consistent with ground truth.Originality/valueThis paper proposes a method to obtain contact-consistent poses and contact states of objects using only visual sensors. The method tries to recover the true contact state from inaccurate visual information by fusing contact simulations results and contact consistency assumptions. The method can be used to extract pose and contact information from object manipulation tasks by just observing the demonstration, which can provide a new way for the robot to learn complex manipulation tasks. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Assembly Automation Emerald Publishing

Contact-consistent visual object pose estimation for contact-rich robotic manipulation tasks

Download PDF
14 pages

Loading next page...
 
/lp/emerald-publishing/contact-consistent-visual-object-pose-estimation-for-contact-rich-OiZ0awfU8K
Publisher
Emerald Publishing
Copyright
© Emerald Publishing Limited
ISSN
0144-5154
eISSN
0144-5154
DOI
10.1108/aa-10-2021-0128
Publisher site
See Article on Publisher Site

Abstract

The purpose of this paper is to estimate the contact-consistent object poses during contact-rich manipulation tasks based only on visual sensors.Design/methodology/approachThe method follows a four-step procedure. Initially, the raw object poses are retrieved using the available object pose estimation method and filtered using Kalman filter with nominal model; second, a group of particles are randomly generated for each pose and evaluated the corresponding object contact state using the contact simulation software. A probability guided particle averaging method is proposed to balance the accuracy and safety issues; third, the independently estimated contact states are fused in a hidden Markov model to remove the abnormal contact state observations; finally, the object poses are refined by averaging the contact state consistent particles.FindingsThe experiments are performed to evaluate the effectiveness of the proposed methods. The results show that the method can achieve smooth and accurate pose estimation results and the estimated contact states are consistent with ground truth.Originality/valueThis paper proposes a method to obtain contact-consistent poses and contact states of objects using only visual sensors. The method tries to recover the true contact state from inaccurate visual information by fusing contact simulations results and contact consistency assumptions. The method can be used to extract pose and contact information from object manipulation tasks by just observing the demonstration, which can provide a new way for the robot to learn complex manipulation tasks.

Journal

Assembly Automation – Emerald Publishing

Published: Jul 19, 2022

Keywords: Object pose estimation; Pose optimization; Contact simulation; Automatic assembly; Assembly

References

  • Network anomaly detection using two-dimensional hidden Markov model based Viterbi algorithm
    Alhaidari, S.; Zohdy, M.
  • A survey of robot learning from demonstration
    Argall, B.D.; Chernova, S.; Veloso, M.; Browning, B.
  • Learning force control for contact-rich manipulation tasks with rigid position-controlled robots
    Beltran-Hernandez, C.C.; Petit, D.; Ramirez-Alpizar, I.G.; Nishi, T.; Kikuchi, S.; Matsubara, T.; Harada, K.
  • Kalman filter for robot vision: a survey
    Chen, S.
  • G2l-net: global to local network for real-time 6d pose estimation with embedding vector features
    Chen, W.; Jia, X.; Chang, H.J.; Duan, J.; Leonardis, A.
  • Robotic assembly for tight tolerant cascaded peg-in-hole process with incomplete constraints
    Cheng, H.; Liu, T.; Zhang, W.; Hao, L.
  • Experimental approaches to measure displacements in mechanical systems through vision devices
    Cosenza, C.; Nicolella, A.; Niola, V.; Savino, S.
  • Rgb-d vision device for tracking a moving target
    Cosenza, C.; Nicolella, A.; Niola, V.; Savino, S.
  • Mechanical system control by RGB-D device
    Cosenza, C.; Nicolella, A.; Esposito, D.; Niola, V.; Savino, S.
  • Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes
    Hinterstoisser, S.; Lepetit, V.; Ilic, S.; Holzer, S.; Bradski, G.; Konolige, K.; Navab, N.
  • A kalman-filter-based method for pose estimation in visual servoing
    Janabi-Sharifi, F.; Marey, M.
  • Single shot 6d object pose estimation
    Kleeberger, K.; Huber, M.F.
  • Pose estimation for planar contact manipulation with manifold particle filters
    Koval, M.C.; Pollard, N.S.; Srinivasa, S.S.
  • Epnp: an accurate o (n) solution to the pnp problem
    Lepetit, V.; Moreno-Noguer, F.; Fua, P.
  • Point pair feature-based pose estimation with multiple edge appearance models (ppf-meam) for robotic bin picking
    Liu, D.; Arai, S.; Miao, J.; Kinugawa, J.; Wang, Z.; Kosuge, K.
  • Reducing uncertainty in pose estimation under complex contacts via force forecast
    Mao, H.; Xiao, J.
  • Mechanical system control by RGB-D device
    Niola, V.; Savino, S.
  • Narrow passage sampling in the observation of robotic assembly tasks
    Nottensteiner, K.; Sagardia, M.; Stemmer, A.; Borst, C.
  • Pix2pose: pixel-wise coordinate regression of objects for 6D pose estimation
    Park, K.; Patten, T.; Vincze, M.
  • Fast and automatic object pose estimation for range images on the gpu
    Park, I.K.; Germann, M.; Breitenstein, M.D.; Pfister, H.
  • LCD: learned cross-domain descriptors for 2D-3D matching
    Pham, Q.-H.; Uy, M.A.; Hua, B.-S.; Nguyen, D.T.; Roig, G.; Yeung, S.-K.
  • Fast point feature histograms (FPFH) for 3D registration
    Rusu, R.B.; Blodow, N.; Beetz, M.
  • Skill learning for robotic assembly based on visual perspectives and force sensing
    Song, R.; Li, F.; Quan, W.; Yang, X.; Zhao, J.
  • Real-time seamless single shot 6D object pose prediction
    Tekin, B.; Sinha, S.N.; Fua, P.
  • Multi sensor fusion in robot assembly using particle filters
    Thomas, U.; Molkenstruck, S.; Iser, R.; Wahl, F.M.
  • A hybrid approach for object localization combining mask R-CNN and Halcon in an assembly scenario
    Timmermann, D.; Heid, D.; Friedel, T.; Roennau, A.; Dillmann, R.
  • Behavioral cloning from observation
    Torabi, F.; Warnell, G.; Stone, P.
  • Combining scale-space and similarity-based aspect graphs for fast 3D object recognition
    Ulrich, M.; Wiedemann, C.; Steger, C.
  • An improved point cloud descriptor for vision based robotic grasping system
    Wang, F.; Liang, C.; Ru, C.; Cheng, H.
  • Imitation learning from observations by minimizing inverse dynamics disagreement
    Yang, C.; Ma, X.; Huang, W.; Sun, F.; Liu, H.; Huang, J.; Gan, C.
  • A spatial information inference method for programming by demonstration of assembly tasks by integrating visual observation with cad model
    Zhou, Z.; Ji, L.; Xiong, R.; Wang, Y.
  • Robot learning from demonstration in robotic assembly: a survey
    Zhu, Z.; Hu, H.
  • Learning object manipulation skills via approximate state estimation from real videos
    Petrík, V.; Tapaswi, M.; Laptev, I.; Sivic, J.
  • Generative adversarial imitation from observation
    Torabi, F.; Warnell, G.; Stone, P.