The field of control and reinforcement learning is rapidly evolving, with a focus on developing innovative solutions for complex systems. Recent research has explored the integration of deep reinforcement learning with model predictive control, bounded extremum seeking, and other techniques to improve robustness and adaptability in dynamic environments. Notable advancements include the development of hybrid controllers that combine the strengths of different approaches, such as combining deep reinforcement learning with bounded extremum seeking to improve robustness in time-varying systems. Another significant area of research is the application of reinforcement learning to real-world problems, such as robotic manipulation, locomotion, and debris removal. Researchers are also investigating new architectures and frameworks for combining reinforcement learning with model predictive control, as well as developing novel sampling strategies for robust universal locomotion policies. Some noteworthy papers in this area include: Improved Robustness of Deep Reinforcement Learning for Control of Time-Varying Systems by Bounded Extremum Seeking, which demonstrates the effectiveness of combining deep reinforcement learning with bounded extremum seeking. U-LAG: Uncertainty-Aware, Lag-Adaptive Goal Retargeting for Robotic Manipulation, which presents a novel approach to goal retargeting for robotic manipulation in changing environments. Global Convergence of Policy Gradient for Entropy Regularized Linear-Quadratic Control with multiplicative noise, which investigates the global convergence of policy gradient methods for entropy-regularized linear-quadratic control. On Architectures for Combining Reinforcement Learning and Model Predictive Control with Runtime Improvements, which proposes two architectures for combining reinforcement learning and model predictive control. Model-Based Adaptive Precision Control for Tabletop Planar Pushing Under Uncertain Dynamics, which presents a model-based framework for non-prehensile tabletop pushing that uses a single learned model to address multiple tasks without retraining. From Shadow to Light: Toward Safe and Efficient Policy Learning Across MPC, DeePC, RL, and LLM Agents, which presents eight approaches to reduce computational complexity in data-driven policies. Learning to Capture Rocks using an Excavator: A Reinforcement Learning Approach with Guiding Reward Formulation, which introduces a fully data-driven control framework for rock capturing that eliminates the need for explicit modeling of rock or soil properties. Flexible Locomotion Learning with Diffusion Model Predictive Control, which presents Diffusion-MPC, a novel approach to flexible locomotion learning that leverages a learned generative diffusion model as an approximate dynamics prior for planning. A KL-regularization framework for learning to plan with adaptive priors, which unifies MPPI-based reinforcement learning methods under a single framework. Velocity-Form Data-Enabled Predictive Control of Soft Robots under Unknown External Payloads, which presents a novel velocity-form DeePC framework that achieves robust and optimal control of soft robots under unknown payloads. Walking, Rolling, and Beyond: First-Principles and RL Locomotion on a TARS-Inspired Robot, which presents a combination of analytic synthesis and reinforcement learning for multimodal robotics. Safe Obstacle-Free Guidance of Space Manipulators in Debris Removal Missions via Deep Reinforcement Learning, which proposes a curriculum-based multi-critic network for safe and reliable debris capture. Sampling Strategies for Robust Universal Quadrupedal Locomotion Policies, which investigates the effects of sampling physical robot parameters and joint proportional-derivative gains to enable training a single reinforcement learning policy that generalizes to multiple parameter configurations.
Advancements in Control and Reinforcement Learning for Complex Systems
Sources
Improved Robustness of Deep Reinforcement Learning for Control of Time-Varying Systems by Bounded Extremum Seeking
Global Convergence of Policy Gradient for Entropy Regularized Linear-Quadratic Control with multiplicative noise
On Architectures for Combining Reinforcement Learning and Model Predictive Control with Runtime Improvements
From Shadow to Light: Toward Safe and Efficient Policy Learning Across MPC, DeePC, RL, and LLM Agents
Learning to Capture Rocks using an Excavator: A Reinforcement Learning Approach with Guiding Reward Formulation