Implementations of TD3 and DDPG Reinforcement Learning Techniques for Tuning PID Controller of TRMS System


Tufenkci S., ALAGÖZ B. B., Kavuran G., Yeroglu C., Herencsar N., Mahata S.

Journal of Systems Science and Systems Engineering, 2025 (SCI-Expanded) identifier

  • Yayın Türü: Makale / Tam Makale
  • Basım Tarihi: 2025
  • Doi Numarası: 10.1007/s11518-025-5693-5
  • Dergi Adı: Journal of Systems Science and Systems Engineering
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Compendex, INSPEC
  • Anahtar Kelimeler: Deep Deterministic Policy Gradient (DDPG), Deep reinforcement learning, Direct Current (DC) motor, PID controller, twin-rotor multi-input multi-output system
  • İnönü Üniversitesi Adresli: Evet

Özet

Reinforcement Learning (RL) is a learning method that utilizes interactions between agents and their environments, providing a valuable tool for controller design through simulations. However, traditional industrial systems such as PID control loops have yet to fully embrace the advantages of RL algorithms for effectively tuning controllers. This study presents an experimental initiative demonstrating the implementation of an RL-driven method for optimal PID controller tuning to address challenges in rotor control, explicitly focusing on the Twin-Rotor Multi-Input Multi-Output System (TRMS). Rotor control presents a complex challenge involving aerodynamics and external disturbances. The research implements two RL algorithms, namely the Deep Deterministic Policy Gradient (DDPG) and the Twin Delay Deep Deterministic Policy Gradient (TD3), in a tailored simulation environment to train RL agents to achieve optimal PID control dynamics. Results of simulation and experimental studies indicate that RL algorithms can be implemented for PID controller tuning when the simulation environment for training the RL algorithms well-represent the dominating dynamics and control complications of real-world systems. In this case, both the simulation and experimental results are in good-agreement.