RoboTube

Download

We aim to build a useful, reproducible, democratized benchmark for learning household robotic manipulation from human videos. To realize this goal, a diverse, high-quality human video dataset curated specifically for robots is desired. To evaluate the learning progress, a simulated twin environment that resembles the appearance and the dynamics of the physical world would help roboticists and AI researchers validate their algorithms convincingly and efficiently before testing on a real robot. We introduce RoboTube, a benchmark platform that can lower the barrier to robotics research while facilitating reproducible research in the community.

Dataset

We build a diverse and high-quality human video demonstration dataset with multiple functionalities.

Construction Overview

RT-sim

To benchmark the baseline methods, we construct a suite of simulated twin environments, RT-sim. With RT-sim, researchers can make a fair comparison of their approaches with the baseline methods and can validate their algorithms convincingly and efficiently before conducting more complex experiments on real robots.

Paper & Code

Team

Haoyu Xiong

Haoyuan Fu

Jieyi Zhang

Chen Bao

Qiang Zhang

Yongxi Huang

Wenqiang Xu

Animesh Garg

Huazhe Xu

Cewu Lu

Citation

To cite this work, please use the following BibTex entry,

                        @inproceedings{xiong2022robotube,
title={RoboTube: Learning Household Manipulation from Human Videos with Simulated Twin Environments},
author={haoyu Xiong and Haoyuan Fu and Jieyi Zhang and Chen Bao and Qiang Zhang and Yongxi Huang and Wenqiang Xu and Animesh Garg and Cewu Lu},
booktitle={6th Annual Conference on Robot Learning},
year={2022},
url={https://openreview.net/forum?id=VD0nXUG5Qk}
}

If you have any questions, please feel free to contact Haoyu Xiong.