Reinforcement Learning for Blind Stair Climbing with Legged and
Wheeled-Legged Robots
Simon Chamorro
1,2
, Victor Klemm
3
, Miguel de la Iglesia Valls
4
, Christopher Pal
1
, and Roland Siegwart
2
Abstract— In recent years, legged and wheeled-legged robots
have gained prominence for tasks in environments predomi-
nantly created for humans across various domains. One signif-
icant challenge faced by many of these robots is their limited
capability to navigate stairs, which hampers their functionality
in multi-story environments. This study proposes a method
aimed at addressing this limitation, employing reinforcement
learning to develop a versatile controller applicable to a wide
range of robots. In contrast to the conventional velocity-
based controllers, our approach builds upon a position-based
formulation of the RL task, which we show to be vital for
stair climbing. Furthermore, the methodology leverages an
asymmetric actor-critic structure, enabling the utilization of
privileged information from simulated environments during
training while eliminating the reliance on exteroceptive sensors
during real-world deployment. Another key feature of the pro-
posed approach is the incorporation of a boolean observation
within the controller, enabling the activation or deactivation
of a stair-climbing mode. We present our results on different
quadrupeds and bipedal robots in simulation and showcase how
our method allows the balancing robot Ascento to climb 15cm
stairs in the real world, a task that was previously impossible
for this robot.
I. INTRODUCTION
Mobile ground robots have been widely studied and used
for various tasks, such as delivery, inspection, and secu-
rity, [1]. While wheeled robots are efficient at traveling
long distances, they lack the agility to traverse obstacles
such as stairs. Legged systems, in contrast, are more agile,
but lack efficiency and speed [2]. Hybrid wheeled-legged
systems have been developed to harness the advantages of
both technologies, but pose significant challenges for control
due to their complex dynamics [3]. Recently, Reinforcement
Learning (RL) has emerged as a promising technique for
controlling these complex robotic systems. In particular,
interesting results have been achieved using RL to develop
controllers for wheeled-legged quadrupeds [4], [5].
In this work, we study the problem of stair-climbing for
legged and wheeled-legged robots. This is particularly diffi-
cult for bipeds since it requires a very dynamic movement
where the robot keeps balance on only one point of contact
while stepping up. We propose a method to climb stairs
with both legged and wheeled-legged robots, by develop-
ing step reflexes using privileged terrain information in an
1
Department of Computer and Software Engineering,
´
Ecole Polytech-
nique de Montr
´
eal, 2900 Boul.
´
Edouard-Montpetit, Qu
´
ebec, Canada. e-mail:
simon.chamorro@polymtl.ca
2
Autonomous Systems Lab, ETH Zurich, Switzerland.
3
Robotic Systems Lab, ETH Zurich, Switzerland.
4
Ascento Robotics, Zurich, Switzerland.
Supplementary video: https://youtu.be/Ec6ar8BVJh4.
Fig. 1: Proposed Method: Ascento Robot, Unitree Go1,
Cassie and ANYmal on Wheels climbing steps.
asymmetric actor-critic setup [6]. Interesting reflex behaviors
have been achieved on quadrupeds, presented by Lee et al.
[7]. Siekmann et al. have also shown that it is possible to
blindly traverse stairs with the robot Cassie [8]. Additionally,
we investigate the concept of a boolean mode switch for
stair-climbing, thereby allowing a good performance on both
regular terrain and stair ascent with the same control policy.
Our controller operates with this boolean observation as its
only exteroceptive information and no positioning system.
Our main contributions are the following:
1) We present an RL task formulation to train policies
capable of climbing stairs with quadruped robots,
bipedal robots, and wheeled-legged balancing robots.
2) Our method does not require any perceptive data or a
positioning system such as Simultaneous Localization
and Mapping (SLAM) or Global Positioning System
(GPS), making it straight-forward to implement into a
standard control stack.
3) Our proposed system can successfully transfer to the
real world and allows the wheeled bipedal robot As-
cento to climb 15cm steps.
II. LITERATURE REVIEW
A. Legged Locomotion using RL
Model-based control methods that rely on optimization,
such as Model Predictive Control (MPC) and trajectory opti-
mization, have seen widespread usage for legged locomotion
problems in the past [9]. However, these methods require a
dynamics model of the system during execution leading to
high complexity for high-dimensional systems. Recently, RL
arXiv:2402.06143v1 [cs.RO] 9 Feb 2024
资源评论
m0_71743919
- 粉丝: 0
- 资源: 1
最新资源
- 技术资料分享发明者电子设计宝典.zip
- 技术资料分享电子镇流器知识技术资料开发设计用的重要资料.zip
- c-master (11).zip
- 电子设计竞赛（简称电赛）心得： 是中国一项非常受欢迎的大学生科技竞赛活动，它旨在考察学生的电子设计能力和创新能力 以下是基于历史
- JESD22A test method and test workload
- c-master (11).zip
- 技术资料分享FPGA入门系列实验教程V1.0.zip
- RNN预测模型做多输入单输出预测模型，直接替换数据就可以用 程序语言是matlab，需求最低版本为2021及以上 程序可以出
- 技术资料分享FPGA入门系列实验教程-PWM输出控制LED显示.zip
- 技术资料分享FPGA开发全攻略-下.zip
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
