Ordered Risk Minimization: Learning More from Less Data
- Resource Type
- Conference
- Authors
- Coppens, Peter; Patrinos, Panagiotis
- Source
- 2023 62nd IEEE Conference on Decision and Control (CDC) Decision and Control (CDC), 2023 62nd IEEE Conference on. :2003-2009 Dec, 2023
- Subject
- Computing and Processing
Power, Energy and Industry Applications
Robotics and Control Systems
Risk minimization
Upper bound
Costs
Calibration
Tuning
- Language
- ISSN
- 2576-2370
We consider the worst-case expectation of a permutation invariant ambiguity set of discrete distributions as a proxy-cost for data-driven expected risk minimization. For this framework, we coin the term ordered risk minimization to highlight how results from order statistics inspired the proxy-cost. Specifically, we show how such costs serve as point-wise high-confidence upper bounds of the expected risk. The confidence level can be determined tightly for any sample size. Conversely we also illustrate how to calibrate the size of the ambiguity set such that the high-confidence upper bound has some user specified confidence. This calibration procedure notably supports ϕ-divergence based ambiguity sets. Numerical experiments then illustrate how the resulting scheme both generalizes better and is less sensitive to tuning parameters compared to the empirical risk minimization approach.