NVIDIA Senior AI Scientist Jim Fan share a open-sourced Robot Parkour project on X, researchers at Stanford AI Lab Zipeng Fu and NVIDIA have open-sourced software enabling agile robotic movement like leaping over obstacles and climbing stairs. The project provides an accessible foundation for robotics tinkerers to experiment with advanced locomotive behaviors.
Dubbed “Open Robot Parkour”, the project is detailed in a new paper from the Stanford Artificial Intelligence Laboratory. It allows quadruped robots to dynamically traverse obstacles using only onboard vision and computing.
As described in a tweet from NVIDIA senior AI scientist Jim Fan, the researchers leveraged two of the company’s core technologies:
IsaacGym – A virtual training environment where robot behaviors can be simulated at 10,000x real-time speed on NVIDIA GPUs. This enabled rapid iteration.
NVIDIA Jetson NX – Provided the onboard compute for visual perception and locomotion on actual robots after training in simulation.
According to the paper, the training methodology involved two key phases:
First, an easy simulated environment enabled the quadruped bot to get a “sketch” of the needed movement skills.
Then a hard mode applied dynamics constraints and refined the skills before distilling them into a final policy reliant only on realistic vision and proprioception.
This workflow resulted in performant policies that transferred to physical robot platforms like the low-cost Unitree A1. The real-world bots displayed dynamic movements like stepping over obstacles, crossing gaps, crawling under barriers, and squeezing through slits.
By open-sourcing both the software and training methodology, the researchers aim to provide a springboard for further innovation in agile robotics. Interested tinkerers can experiment with the tools using their own hardware.
The project showcases the power of modern AI and simulation techniques to unlock capabilities previously requiring expensive hardware innovations. It provides an exciting glimpse into a future where robot mobility reaches new heights thanks to accessible software advancements.
This project is led by @ziwenzhuang_leo and @zipengfu
Advised by @chelseabfinn and @zhaohang0124
Website: https://robot-parkour.github.io
Paper: https://arxiv.org/abs/2309.05665
