Enabling On-Device Self-Supervised Contrastive Learning with Selective Data Contrast
- Resource Type
- Conference
- Authors
- Wu, Yawen; Wang, Zhepeng; Zeng, Dewen; Shi, Yiyu; Hu, Jingtong
- Source
- 2021 58th ACM/IEEE Design Automation Conference (DAC) Design Automation Conference (DAC), 2021 58th ACM/IEEE. :655-660 Dec, 2021
- Subject
- Components, Circuits, Devices and Systems
Power, Energy and Industry Applications
Design automation
Distributed databases
Data models
On-Device Learning
Contrastive Learning
Self-Supervised Learning
- Language
After a model is deployed on edge devices, it is desirable for these devices to learn from unlabeled data to continuously improve accuracy. Contrastive learning has demonstrated its great potential in learning from unlabeled data. However, the online input data are usually none independent and identically distributed (non-iid) and edge devices’ storages are usually too limited to store enough representative data from different data classes. We propose a framework to automatically select the most representative data from the unlabeled input stream, which only requires a small data buffer for dynamic learning. Experiments show that accuracy and learning speed are greatly improved.