Application of mixed reality in robot manipulator programming

Application of mixed reality in robot manipulator programming PurposeMixed reality is expanding in the industrial market and several companies in various fields are adapting this set of technologies for various purposes, such as optimizing processes, improving the programming tasks and promoting the interactivity of their products with the users, or even improving teaching or training. Robotics is another area that can benefit from these recent technologies. In fact, most of the current and futuristic robotic applications, namely, the areas related to advanced manufacturing tasks (e.g. additive-manufacturing, collaborative robotics, etc.), require new technics to actually perceive the result of several actions, including programming tasks, anticipate trajectories, visualize the motion and related information, interface with programmers and users and several other human–machine interfaces. Consequently, this paper aims to explain a new concept of human–machine interfaces aiming to improve the interaction between advanced users and industrial robotic work cells.Design/methodology/approachThe presented concept uses two different applications (apps) developed to explore the advanced features of the Microsoft HoloLens device. The objectives of the project reported in this paper are to optimize robot paths, just by allowing the advanced user to adjust the selected path through the mixed reality environment, and create new paths, just by allowing the advanced user to insert points in the mixed reality environment, correct them as needed, connect them using a certain type of motion, parametrize them (in terms of velocity, motion precision, etc.) and command them to the robot controller.FindingsThe solutions demonstrated in this paper show how mixed reality can be used to allow users, with limited programming experience, to fully use the robotics fields. They also show clearly that the integration of the mixed reality technology in the current robot systems will be a turning point in reducing the complexity for end-users.Research limitations/implicationsThere are two challenges in the developed applications. The first relates to the robot tool identification, which is very sensitive to lighting conditions or to very complex robot tools. This can result in positioning errors when the software shows the path in the mixed reality scene. The paper presents solutions to overcome this problem. Another unattended challenge is associated with handling the robot singularities when adjusting or creating new paths. Ongoing work is concentrated in creating mechanisms that prevent the end-user to create paths that contain unreachable points or paths that are not feasible because of bad motion parameters.Practical implicationsThis paper demonstrates the utilization of mixed reality device to improve the tasks of programming and commanding manufacturing work cells based on industrial robots [see video in (Pires et al., 2018)]. As the presented devices and robot cells are the basis for Industry 4.0 objectives, this demonstration has a vast field of application in the near future, positively influencing the way complex applications, that require much close cooperation between humans and machines, are thought, planned and built. The paper presents two different applications fully ready to use in industrial environments. These applications are scientific experiments designed to demonstrate the principles and technologies of mixed reality applied to industrial robotics, namely, for improving the programming task.Social implicationsAlthough the HoloLens device opens outstanding new areas for robot command and programming, it is still expensive and somehow heavy for everyday use. Consequently, this opens an opportunity window to combine these devices with other mobile devices, such as tablets and phones, building applications that take advantage of their combined features.Originality/valueThe paper presents two different applications fully ready to use in industrial environments. These applications are scientific experiments designed to demonstrate the principles and technologies of mixed reality applied to industrial robotics, namely, for improving the programming task. The first application is about path visualization, i.e. enables the user to visualize, in a mixed reality environment, any path preplanned for the robot cell. With this feature, the advanced user can follow the robot path, identify problems, associate any difficulty in the final product with a particular issue in the robot paths, anticipate execution problems with impact on the final product quality, etc. This is particularly important for not only advanced applications, but also for cases where the robot path results from a CAD package (in an offline fashion). The second application consists of a graphical path manipulation procedure that allows the advanced user to create and optimize a robot path. Just by exploring this feature, the end-user can adjust any path obtained from any programming method, using the mixed reality approach to guide (visually) the path manipulation procedure. It can also create a completely new path using a process of graphical insertion of point positions and paths into the mixed reality scene. The ideas and implementations of the paper are original and there is no other example in the literature applied to industrial robot programming. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Industrial Robot: An International Journal Emerald Publishing

Application of mixed reality in robot manipulator programming

Loading next page...
 
/lp/emerald-publishing/application-of-mixed-reality-in-robot-manipulator-programming-7Xg3xyyw3T
Publisher
Emerald Publishing
Copyright
Copyright © Emerald Group Publishing Limited
ISSN
0143-991X
DOI
10.1108/IR-06-2018-0120
Publisher site
See Article on Publisher Site

Abstract

PurposeMixed reality is expanding in the industrial market and several companies in various fields are adapting this set of technologies for various purposes, such as optimizing processes, improving the programming tasks and promoting the interactivity of their products with the users, or even improving teaching or training. Robotics is another area that can benefit from these recent technologies. In fact, most of the current and futuristic robotic applications, namely, the areas related to advanced manufacturing tasks (e.g. additive-manufacturing, collaborative robotics, etc.), require new technics to actually perceive the result of several actions, including programming tasks, anticipate trajectories, visualize the motion and related information, interface with programmers and users and several other human–machine interfaces. Consequently, this paper aims to explain a new concept of human–machine interfaces aiming to improve the interaction between advanced users and industrial robotic work cells.Design/methodology/approachThe presented concept uses two different applications (apps) developed to explore the advanced features of the Microsoft HoloLens device. The objectives of the project reported in this paper are to optimize robot paths, just by allowing the advanced user to adjust the selected path through the mixed reality environment, and create new paths, just by allowing the advanced user to insert points in the mixed reality environment, correct them as needed, connect them using a certain type of motion, parametrize them (in terms of velocity, motion precision, etc.) and command them to the robot controller.FindingsThe solutions demonstrated in this paper show how mixed reality can be used to allow users, with limited programming experience, to fully use the robotics fields. They also show clearly that the integration of the mixed reality technology in the current robot systems will be a turning point in reducing the complexity for end-users.Research limitations/implicationsThere are two challenges in the developed applications. The first relates to the robot tool identification, which is very sensitive to lighting conditions or to very complex robot tools. This can result in positioning errors when the software shows the path in the mixed reality scene. The paper presents solutions to overcome this problem. Another unattended challenge is associated with handling the robot singularities when adjusting or creating new paths. Ongoing work is concentrated in creating mechanisms that prevent the end-user to create paths that contain unreachable points or paths that are not feasible because of bad motion parameters.Practical implicationsThis paper demonstrates the utilization of mixed reality device to improve the tasks of programming and commanding manufacturing work cells based on industrial robots [see video in (Pires et al., 2018)]. As the presented devices and robot cells are the basis for Industry 4.0 objectives, this demonstration has a vast field of application in the near future, positively influencing the way complex applications, that require much close cooperation between humans and machines, are thought, planned and built. The paper presents two different applications fully ready to use in industrial environments. These applications are scientific experiments designed to demonstrate the principles and technologies of mixed reality applied to industrial robotics, namely, for improving the programming task.Social implicationsAlthough the HoloLens device opens outstanding new areas for robot command and programming, it is still expensive and somehow heavy for everyday use. Consequently, this opens an opportunity window to combine these devices with other mobile devices, such as tablets and phones, building applications that take advantage of their combined features.Originality/valueThe paper presents two different applications fully ready to use in industrial environments. These applications are scientific experiments designed to demonstrate the principles and technologies of mixed reality applied to industrial robotics, namely, for improving the programming task. The first application is about path visualization, i.e. enables the user to visualize, in a mixed reality environment, any path preplanned for the robot cell. With this feature, the advanced user can follow the robot path, identify problems, associate any difficulty in the final product with a particular issue in the robot paths, anticipate execution problems with impact on the final product quality, etc. This is particularly important for not only advanced applications, but also for cases where the robot path results from a CAD package (in an offline fashion). The second application consists of a graphical path manipulation procedure that allows the advanced user to create and optimize a robot path. Just by exploring this feature, the end-user can adjust any path obtained from any programming method, using the mixed reality approach to guide (visually) the path manipulation procedure. It can also create a completely new path using a process of graphical insertion of point positions and paths into the mixed reality scene. The ideas and implementations of the paper are original and there is no other example in the literature applied to industrial robot programming.

Journal

Industrial Robot: An International JournalEmerald Publishing

Published: Oct 15, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create folders to
organize your research

Export folders, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off