RoBIC: A Benchmark Suite For Assessing Classifiers Robustness
- Resource Type
- Conference
- Authors
- Maho, Thibault; Bonnet, Benoit; Furony, Teddy; Le Merrer, Erwan
- Source
- 2021 IEEE International Conference on Image Processing (ICIP) Image Processing (ICIP), 2021 IEEE International Conference on. :3612-3616 Sep, 2021
- Subject
- Computing and Processing
Signal Processing and Analysis
Image processing
Conferences
Benchmark testing
Distortion
Robustness
Distortion measurement
Benchmark
adversarial examples
model robustness
half-distortion measure
- Language
- ISSN
- 2381-8549
Many defenses have emerged with the development of adversarial attacks. Models must be objectively evaluated accordingly. This paper systematically tackles this concern by proposing a new parameter-free benchmark we coin ROBIC. ROBIC fairly evaluates the robustness of image classifiers using a new half-distortion measure. It gauges the robustness of the network against white and black box attacks, independently of its accuracy. ROBIC is faster than the other available benchmarks. We present the significant differences in the robustness of 16 recent models as assessed by ROBIC.We make this benchmark publicly available for use and contribution at https://gitlab.inria.fr/t;maho/robustness_benchmark.