Can Robots Truly See Without Using Vision Sensors?

Can Robots Truly See Without Using Vision Sensors?

Researchers from MIT, Amazon Robotics, and the University of British Columbia have developed an innovative technique allowing robots to identify object properties, like mass and material characteristics, by interacting with them. This technique, which deviates from traditional reliance on cameras or external sensors, utilizes the robot’s proprioceptive capabilities—internal sensors that allow it to sense its own movements. By picking up and gently shaking objects, robots can discern their properties, an advancement that holds promise for low-light environments or structurally compromised areas where visual sensors might fail.

Central to this new approach is the use of differentiated simulations. Robots simulate their interactions with objects within a virtual setting to determine their properties. This involves creating two simulations: one reflecting the robot itself, and another representing the dynamics with the object. These simulations, run alongside real interactions, enable robots to decode data from their sensors to deduce properties like weight and softness.

This advancement underscores a shift in robotics towards more data-efficient algorithms that reduce dependency on large datasets for learning and identification tasks. The ability to derive information from single interactions makes real-world applications more feasible and cost-efficient. The technique also aligns with human ways of interaction, such as intuitively assessing an object’s weight by feel. While not intended to replace visual sensors entirely, this method provides a complementary approach, especially useful in environments where visual systems are less reliable. It opens new avenues for robotics to function autonomously across various scenarios, empowering them to adapt dynamically to different challenges while minimizing reliance on external conditions.

Subscribe to our weekly news digest.

Join now and become a part of our fast-growing community.

Invalid Email Address
Thanks for Subscribing!
We'll be sending you our best soon!
Something went wrong, please try again later