Visuo-Tactile Recognition of Partial Point Clouds Using PointNet and Curriculum Learning: Enabling Tactile Perception from Visual Data
- Resource Type
- Periodical
- Authors
- Parsons, C.; Albini, A.; Martini, D.D.; Maiolino, P.
- Source
- IEEE Robotics & Automation Magazine IEEE Robot. Automat. Mag. Robotics & Automation Magazine, IEEE. 30(3):69-78 Sep, 2023
- Subject
- Robotics and Control Systems
Aerospace
Components, Circuits, Devices and Systems
Computing and Processing
Engineering Profession
General Topics for Engineers
Signal Processing and Analysis
Transportation
Power, Energy and Industry Applications
Point cloud compression
Visualization
Task analysis
Training
Shape control
Robots
Surface reconstruction
Haptic interfaces
- Language
- ISSN
- 1070-9932
1558-223X
This article is about recognizing handheld objects from incomplete tactile observations with a classifier trained on only visual representations. Our method is based on the deep learning (DL) architecture PointNet and a curriculum learning (CL) technique for fostering the learning of descriptors robust to partial representations of objects. The learning procedure gradually decomposes the visual point clouds to synthesize sparser and sparser input data for the model. In this manner, we were able to employ one-shot learning, using the decomposed visual point clouds as augmentations, and reduce the data-collection requirement for training. The approach allows for a gradual improvement of prediction accuracy as more tactile data become available.