A Humanoid Robot Learning Audiovisual Classification By Active Exploration
- Resource Type
- Conference
- Authors
- Mir, Glareh; Kerzel, Matthias; Strahl, Erik; Wermter, Stefan
- Source
- 2021 IEEE International Conference on Development and Learning (ICDL) Development and Learning (ICDL), 2021 IEEE International Conference on. :1-6 Aug, 2021
- Subject
- Robotics and Control Systems
Signal Processing and Analysis
Visualization
Conferences
Neural networks
Humanoid robots
Audio recording
Collision avoidance
Material properties
Crossmodal object recognition
Supervised learning
Robot learning
Multi-layer neural network
- Language
We present a novel neurorobotic setup and dataset for active object exploration and audiovisual classification based on their material properties. In the robotic setup, a humanoid drops an item on a sloped surface and records the video image frames and raw audio of the collision of the surface and object. The novel dataset includes 32800 images and 1600 s of audio recording from 800 samples for 16 objects and will be made publicly available. We propose a novel neural architecture for the classification of the objects. A detailed analysis of results shows that different materials are easier classified either in the audio or the visual modality. As a main contribution, we can show that combining modalities can achieve an even higher classification accuracy of 90%.