Reinforcement Learning-based Position Control for a Disc-shaped Underwater Remotely Operated Vehicle
- Resource Type
- Conference
- Authors
- Yang, Jiacheng; Wang, Yuanda; Yuan, Xin
- Source
- 2023 38th Youth Academic Annual Conference of Chinese Association of Automation (YAC) Chinese Association of Automation (YAC), 2023 38th Youth Academic Annual Conference of. :689-694 Aug, 2023
- Subject
- Aerospace
Communication, Networking and Broadcast Technologies
Components, Circuits, Devices and Systems
Computing and Processing
Power, Energy and Industry Applications
Robotics and Control Systems
Transportation
Remotely guided vehicles
Position control
Process control
Stability analysis
Steady-state
Task analysis
Underwater vehicles
underwater robot
position control
reinforcement learning
deterministic policy gradient
- Language
- ISSN
- 2837-8601
This paper presents the design and development of a deep reinforcement learning-based controller for a disc-shaped underwater remotely operated vehicle (Disc-ROV). The control process is modeled as a Markov decision process (MDP) with unknown state transition probabilities. One critical issue with conventional RL-based control methods is steady-state error. To reduce the final control error, we introduce an integral method to the actor-critic structure. Another issue is the lack of smoothness in the actions, which can be harmful to the actuators. To address this issue, we introduce an additional penalty to the training loss of policy learning, which eliminates high-frequency components in the control signal. We validate the effectiveness of the proposed algorithm in a simulation environment with a position control task. The simulation results demonstrate the accuracy and stability of the trained position control strategy.