MIT’s CSAIL Researchers build Artificial Intelligence system that can identify things by touch
A team of researchers at the Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have come up with a predictive Artificial Intelligence (AI) that can learn to see by touching and to feel by seeing. It can also create a visual representation of an object, just from the tactile data it generates by touching it.
In a newly published paper that will be presented next week at the Conference on Computer Vision and Pattern Recognition in Long Beach, California, they describe an AI system capable of generating visual representations of objects from tactile signals, and of predicting tactility from snippets of visual data.
Yunzhu Li, CSAIL PhD student and lead author on the paper about the system, said the model can help robots handle real-world objects better:
“By looking at the scene, our model can imagine the feeling of touching a flat surface or a sharp edge. By blindly touching around, our model can predict the interaction with the environment purely from tactile feelings. Bringing these two senses together could empower the robot and reduce the data we might need for tasks involving manipulating and grasping objects.”
The team used a KUKA robot arm with a special tactile sensor called GelSight, designed by another group at MIT.
Using a simple web camera, they made the arm touch 200 household objects 12,000 times, and recorded the visual and tactile data. Based on that, it created a data set of more than 3 million visual-tactile images called VisGel.
Andrew Owens, a postdoctoral researcher at the University of California at Berkeley, opined this research can aid robots in knowing how firmly it should grip an object:
“This is the first method that can convincingly translate between visual and tactile signals. Methods like this have the potential to be very useful for robotics, where you need to answer questions like ‘is this object hard or soft?’, or ‘if I lift this mug by its handle, how good will my grip be?’ This is a very challenging problem, since the signals are so different, and this model has demonstrated great capability”.
For the latest tech news follow techcresendo on Twitter, Facebook, and LinkedIn.
