Correlation Image Sensor-Assisted Single-Frame Optical Flow Estimation in Motion-Blurred Scenes
- Resource Type
- Periodical
- Authors
- Wang, P.; Kurihara, T.; Yu, J.
- Source
- IEEE Access Access, IEEE. 12:65856-65868 2024
- Subject
- Aerospace
Bioengineering
Communication, Networking and Broadcast Technologies
Components, Circuits, Devices and Systems
Computing and Processing
Engineered Materials, Dielectrics and Plasmas
Engineering Profession
Fields, Waves and Electromagnetics
General Topics for Engineers
Geoscience
Nuclear Engineering
Photonics and Electrooptics
Power, Energy and Industry Applications
Robotics and Control Systems
Signal Processing and Analysis
Transportation
Correlation
Optical sensors
Feature extraction
Estimation
Optical flow
Synthetic data
Image sensors
Deep learning
Correlation image sensor
deep learning
synthetic dataset
two-stream network
optical flow
- Language
- ISSN
- 2169-3536
Single-frame optical flow estimation is a more challenging task than predicting the optical flow between adjacent frames in a video. This paper presents a two-stream network that combines motion information from correlation images with appearance information from intensity images to estimate optical flow in a single-frame manner. The correlation image generated by a three-phase correlation image sensor (3PCIS) records changes in incident intensity during the exposure time and conveys motion information about moving objects. This is crucial to assist motion-blurred images in estimating the motion state of moving objects. Due to the lack of directly available datasets to train our network, we generate a synthetic dataset. Importantly, we introduce a definition that describes optical flow on a single motion-blurred image, which is essential for creating reasonable ground truth. Our experimental results demonstrate that 1) network trained on our synthetic dataset achieves an average End Point Error (EPE) of 0.357 and generalizes well to real-world scenes; 2) our proposed single-frame method outperforms conventional two-frame-based methods in motion-blurred scenes.