METHOD OF LEARNING OF AUTONOMOUS MOBILE ROBOTS BASED ON DRL AND CURRICULUM LEARNING

Abstract

The work is devoted to the urgent task of improving the efficiency of socially adaptive navigation of autonomous mobile robots in dynamic environments with human presence. The application of deep reinforcement learning (DRL) methods to solve this problem is complicated by the high dimensionality of the state space, the complexity of formalizing social norms in the reward function, and the instability of the learning process. To overcome these challenges, a method is proposed that integrates the Proximal Policy Optimization (PPO) algorithm with the Curriculum Learning (CL) training strategy. The developed training program combines a gradual increase in the complexity of the environment (from static obstacles to an environment with moving human agents) and the phased formation of the reward function with the addition of social components. A key feature is the transition between stages, which is based on policy stability analysis. The experimental study was conducted in the developed Gazebo simulation environment using the Turtlebot3 Waffle mobile robot and the ROS 2 Humble framework. Step-by-step training allows an autonomous mobile robot to first learn basic skills for avoiding static obstacles, then dynamic ones, and finally, at the final stage, to take into account social norms of interaction with people. The input data for the system is data from LiDAR, the status of the robot and people, and the target position. The result of the method is an optimized stochastic behavior policy that allows an autonomous mobile robot to make safe, efficient, and socially acceptable navigation decisions. A comparative analysis of the proposed method with the standard PPO algorithm was performed. The results confirm that the proposed method allows the formation of an effective policy of socially adaptive navigation, solving the problems of instability and slow convergence.