Learning Over Contracting and Lipschitz Closed-Loops for Partially-Observed Nonlinear Systems
- Resource Type
- Conference
- Authors
- Barbara, Nicholas H.; Wang, Ruigang; Manchester, Ian R.
- Source
- 2023 62nd IEEE Conference on Decision and Control (CDC) Decision and Control (CDC), 2023 62nd IEEE Conference on. :1028-1033 Dec, 2023
- Subject
- Computing and Processing
Power, Energy and Industry Applications
Robotics and Control Systems
Optimal control
Reinforcement learning
Stability analysis
Robustness
Closed loop systems
Task analysis
Nonlinear systems
- Language
- ISSN
- 2576-2370
This paper presents a policy parameterization for learning-based control on nonlinear, partially-observed dynamical systems. The parameterization is based on a nonlinear version of the Youla parameterization and the recently proposed Recurrent Equilibrium Network (REN) class of models. We prove that the resulting Youla - Ren parameterization automatically satisfies stability (contraction) and user-tunable robustness (Lipschitz) conditions on the closed-loop system. This means it can be used for safe learning-based control with no additional constraints or projections required to enforce stability or robustness. We test the new policy class in simulation on two reinforcement learning tasks: 1) magnetic suspension, and 2) inverting a rotary-arm pendulum. We find that the Youla-REN performs similarly to existing learning-based and optimal control methods while also ensuring stability and exhibiting improved robustness to adversarial disturbances.