Robust Importance-Weighted Cross-Validation Under Sample Selection Bias
- Resource Type
- Conference
- Authors
- Kouw, Wouter M.; Krijthe, Jesse H.; Loog, Marco
- Source
- 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP) Machine Learning for Signal Processing (MLSP), 2019 IEEE 29th International Workshop on. :1-6 Oct, 2019
- Subject
- Bioengineering
Communication, Networking and Broadcast Technologies
Computing and Processing
General Topics for Engineers
Geoscience
Power, Energy and Industry Applications
Robotics and Control Systems
Signal Processing and Analysis
Robustness
Sociology
Training data
Standards
Correlation
Probability
Sample selection bias
cross-validation
- Language
Cross-validation under sample selection bias can, in principle, be done by importance-weighting the empirical risk. However, the importance-weighted risk estimator produces suboptimal hyperparameter estimates in problem settings where large weights arise with high probability. We study its sampling variance as a function of the training data distribution and introduce a control variate to increase its robustness to problematically large weights.