Reinforcement learning for hexapod robot trajectory control: a study of Q-learning and SARSA algorithms
Loading...
Date
2025-10-25
Journal Title
Journal ISSN
Volume Title
Publisher
International Journal of Intelligent Robotics and Applications
Abstract
Hexapod robots, with their six-legged design, excel in stability and adaptability on challenging terrain but pose signifi
cant control challenges due to their high degrees of freedom. While reinforcement learning (RL) has been explored for
robot navigation, few studies have systematically compared on-policy and off-policy methods for multi-legged locomo
tion. This work presents a comparative study of SARSA and Q-Learning for trajectory control of a simulated hexapod
robot, focusing on the influence of learning rate (α), discount factor (γ), and eligibility trace (λ). The evaluation spans
eight initial poses, with performance measured through lateral deviation (Ey), orientation error (Eθ), and iteration count.
Results show that Q-Learning generally achieves faster convergence and greater stability, particularly with higher γ and
λ values, while SARSA can achieve competitive accuracy with careful parameter tuning. The findings demonstrate that
eligibility traces substantially improve learning precision and provide practical guidelines for robust RL-based control in
multi-legged robotic systems.