Researchers developed an innovative robotic link with multimodal perception to help make human-robot interactions safer and more convenient

To make human-robot interactions safer and more fruitful, robots should be capable of sensing their environment. In a recent study, researchers from Japan Advanced Institute of Science and Technology (JAIST) developed a novel robotic link with tactile and proximity sensing capabilities. Additionally, they created a simulation and learning framework that can be employed to train the robotic link to sense its environment. Their findings will pave the way to a future where humans and robots can operate harmoniously in close proximity.

In recent years, robots have become incredibly sophisticated machines capable of performing or assisting humans in all tasks. The days of robots functioning behind a security barrier are long gone, and today we may anticipate robots working alongside people in close contact. While working alongside robots may be very practical in some situations, they should be designed to be safe and pleasant for humans to interact with. For instance, in human-robot interactions (HRIs), robots should be able to react correctly to potential collisions with humans and also respond safely and predictably to intentional physical contact.

One of the best approaches to improve HRIs is to grant robots the ability to sense their environment in multiple ways, such as by touch, sound, and sight. Of these three, tactile sensation is particularly important for robots that are likely to come into physical contact with humans during operation. Although small-scale tactile sensors have seen tremendous progress over the past decade, the development of large-scale tactile sensors has been plagued with challenges. Moreover, most researchers have focused on systems that respond to physical touch and ignore touchless stimuli, such as when an object is in close proximity. To address these issues, a research team led by Associate Professor Van Anh Ho from Japan Advanced Institute of Science and Technology (JAIST) recently developed ProTac--an innovative soft robotic link with tactile and proximity sensing capabilities. As explained in their paper, presented at the IEEE-RAS International Conference on Soft Robotics (ROBOSOFT), the team not only engineered ProTac itself but also pioneered a new simulation and learning framework to effectively prepare the robotic link for use.

But what does a robotic link look like, and what is ProTac good for? In general, robotic links are rigid structural components of a robot that connect two or more joints. For example, robotic links can be seen as various 'segments' in a robotic limb. In this study, ProTac is designed as a soft, cylindrical segment for a robotic arm. What makes it remarkable is how the researchers incorporated the tactile and proximity sensing capabilities in a very convenient and space-efficient way.

ProTac has an outer 'soft magic skin' that can be slightly deformed by touch without damage. The inside of the skin is patterned with arrays of reflective markers, and fisheye cameras are installed at both ends of the robotic link looking towards these markers. The idea is that, upon physical contact and deformation of the skin, changes in the relative positions of the markers are captured by the cameras and processed to calculate the precise location and intensity of the contact. On top of this, the outer skin is of a functional polymer that can be made entirely transparent by applying an external voltage. It allows the fisheye cameras to image the immediate surroundings of ProTac, providing footage for proximity calculations.

To more easily train ProTac to make proximity and tactile measurements, the team also developed SimTacLS, an open-source simulation and learning framework based on the SOFA and Gazebo physics engines (see the paper here). This machine learning framework is trained with simulated and experimental data considering the physics of soft contact and the realistic rendering of sensor images. "SimTacLS enabled us to effectively implement tactile perception in robotic links without the high costs of complex experimental setups," remarks Prof. Ho, "Furthermore, with this framework, users can readily validate sensor designs and learning-based sensing performance before proceeding to actual fabrication and implementation."

Overall, this work will help pave the way to a world where humans can harmoniously coexist and work alongside robots. Excited by the team's contribution to this dream, Prof. Ho comments: "We expect the proposed sensing device and framework to bring in ultimate solutions for the design of robots with softness, whole-body and multimodal sensing, and safety control strategies." It is worth noting that proposed techniques can be extended to other types of robotic systems beyond the robotic manipulator demonstrated in the study, such as mobile and flying robots. Moreover, ProTac or similar robotic links could be used to enable robotic manipulation in cluttered environments or when operating in close vicinity with humans.

Only time will tell how close we'll be to working with robots in the coming years!

Title: A soft robotic link capable of tactile and proximity sensing
Caption: The inside of ProTac's 'skin' is covered with reflective markers. Cameras at the ends of the robotic link can capture changes in the relative positions of the markers, which indicates physical touch. Moreover, the skin can be made entirely transparent by applying an external voltage, allowing these cameras to detect objects in close proximity.
Credit: Van Anh Ho from JAIST
License Type: Original Content
Usage Restrictions: Not to be reproduced without the permission of the owner

Title: A simulation and learning framework for tactile perception in robots
Caption: After training its internal machine learning models with simulated and experimental data on robotic tactile perception, the SimTacLS framework can be used to simplify the design and functionalization of sensitive robotic parts, both in the large and small scale.
Credit: Van Anh Ho from JAIST
License Type: Original Content
Usage Restrictions: Not to be reproduced without the permission of the owner

Reference

Funding information

This study was supported by Japan Science and Technology Agency's Precursory Research for Embryonic Science and Technology (Grant Number: JPMJPR2038).

April 12, 2023