Liquid Metal and AI Create Artificial Sense of Touch

The fingertips of the human hand contain over 3,000 touch receptors, making them extremely responsive to pressure. People lean heavily on fingertip sensation when handling or manipulating something.

People with upper limb amputations can be greatly impaired by the absence of these sensations. There are plenty of high-tech, dexterous prosthetic replacements on the market, but they all lack that all-important “sense of touch.” Going without that sensory feedback means objects can be inadvertently and involuntarily crushed in a prosthetic hand.

To combat this, Florida Atlantic University researchers and collaborators built a prosthetic hand with stretchable liquid metal tactile sensors on its fingertips. The technology, enclosed in silicone-based elastomers, surpasses traditional sensors due to high conductivity, compliance, flexibility, and stretchability, according to a study published in Sensors.¹ This advance could render artificial hands with higher intelligence courtesy of the hierarchical multi-finger tactile sensation integration.

Individual prosthesis fingertips were used to distinguish between different sliding motion speeds along four differently textured surfaces, with one variable parameter of distance between the ridges. The researchers trained four machine learning algorithms to detect the textures and speeds. Twenty trials were collected on the 10 complex surfaces—each comprised of randomly generated permutations of four different textures—to assess the algorithm’s prowess to differentiate between them.

The results demonstrated integrating the tactile information from liquid metal sensors on the four fingertips simultaneously distinguished between complex, multi-textured surfaces, depicting a new breed of hierarchical intelligence. The algorithms could accurately differentiate between all the speeds with each finger, potentially boosting prosthetic hand control and allowing haptic feedback—more commonly known as the experience of touch. Amputees could then reconnect with the previously severed sense.

“Significant research has been done on tactile sensors for artificial hands, but there is still a need for advances in lightweight, low-cost, robust multimodal tactile sensors,” said Erik Engeberg, Ph.D., senior author, an associate professor in the Department of Ocean and Mechanical Engineering and a member of the FAU Stiles-Nicholson Brain Institute and the FAU Institute for Sensing and Embedded Network Systems Engineering (I-SENSE), who conducted the study with first author and Ph.D. student Moaed A. Abd. “The tactile information from all the individual fingertips in our study provided the foundation for a higher hand-level of perception enabling the distinction between 10 complex, multi-textured surfaces that would not have been possible using purely local information from an individual fingertip. We believe that these tactile details could be useful in the future to afford a more realistic experience for prosthetic hand users through an advanced haptic display, which could enrich the amputee-prosthesis interface and prevent amputees from abandoning their prosthetic hand.”

The four machine learning algorithms used were specifically: K-nearest neighbor (KNN), support vector machine (SVM), random forest (RF), and neural network (NN). The liquid metal sensors’ time-frequency features were extracted to train and assess the algorithms. The NN generally performed best at speed and texture detection with a single finger, touting 99.2 percent accuracy to differentiate between 10 multi-textured surfaces using four liquid metal sensors from four fingers, simultaneously.

“The loss of an upper limb can be a daunting challenge for an individual who is trying to seamlessly engage in regular activities,” said Stella Batalama, Ph.D., dean, College of Engineering and Computer Science, FAU. “Although advances in prosthetic limbs have been beneficial and allow amputees to better perform their daily duties, they do not provide them with sensory information such as touch. They also don’t enable them to control the prosthetic limb naturally with their minds. With this latest technology from our research team, we are one step closer to providing people all over the world with a more natural prosthetic device that can ‘feel’ and respond to its environment.” 

Reference

1. Abd, M.A., et al. (2021) Hierarchical Tactile Sensation Integration from Prosthetic Fingertips Enables Multi-Texture Surface Recognition. Sensors. doi.org/10.3390/s21134324.