ASIC: Aligning Sparse in-the-wild Image Collections
- Resource Type
- Conference
- Authors
- Gupta, Kamal; Jampani, Varun; Esteves, Carlos; Shrivastava, Abhinav; Makadia, Ameesh; Snavely, Noah; Kar, Abhishek
- Source
- 2023 IEEE/CVF International Conference on Computer Vision (ICCV) ICCV Computer Vision (ICCV), 2023 IEEE/CVF International Conference on. :4111-4122 Oct, 2023
- Subject
- Computing and Processing
Signal Processing and Analysis
Computer vision
Codes
Deformation
Pose estimation
Neural networks
Benchmark testing
Transformers
- Language
- ISSN
- 2380-7504
We present a method for joint alignment of sparse in-the-wild image collections of an object category. Most prior works assume either ground-truth keypoint annotations or a large dataset of images of a single object category. However, neither of the above assumptions hold true for the long-tail of the objects present in the world. We present a self-supervised technique that directly optimizes on a sparse collection of images of a particular object/object category to obtain consistent dense correspondences across the collection. We use pairwise nearest neighbors obtained from deep features of a pre-trained vision transformer (ViT) model as noisy and sparse keypoint matches and make them dense and accurate matches by optimizing a neural network that jointly maps the image collection into a learned canonical grid. Experiments on CUB, SPair-71k and PF-Willow benchmarks demonstrate that our method can produce globally consistent and higher quality correspondences across the image collection when compared to existing self-supervised methods. Code and other material will be made available at https://kampta.github.io/asic.