GraspXL: Generating Grasping Motions for Diverse Objects at Scale

1Department of Computer Science, ETH Zürich, Switzerland
2Max Planck Institute for Intelligent Systems, Germany
Accepted by ECCV 2024

Video

GraspXL is a method that can synthesize objective-driven grasping motions for 500k+ objects, which can be deployed on different robot hands and generated or reconstructed objects.

Abstract

Human hands possess the dexterity to interact with diverse objects such as grasping specific parts of the objects and/or approaching them from desired directions. More importantly, humans can grasp objects of any shape without object-specific skills. Recent works synthesize grasping motions following single objectives such as a desired approach heading direction or a grasping area. Moreover, they usually rely on expensive 3D hand-object data during training and inference, which limits their capability to synthesize grasping motions for unseen objects at scale. In this paper, we unify the generation of hand-object grasping motions across multiple motion objectives, diverse object shapes and dexterous hand morphologies in a policy learning framework GraspXL. The objectives are composed of the graspable area, heading direction during approach, wrist rotation, and hand position. Without requiring any 3D hand-object interaction data, our policy trained with 58 objects can robustly synthesize diverse grasping motions for more than 500k unseen objects with a success rate of 82.2%. At the same time, the policy adheres to objectives, which enables the generation of diverse grasps per object. Moreover, we show that our framework can be deployed to different dexterous hands and work with reconstructed or generated objects. We quantitatively and qualitatively evaluate our method to show the efficacy of our approach. Our model and code are released.

Generated Motions

Diverse Results

Generate grasping motions for 500k+ unseen objects.

Generate diverse motions for the same object according to diverse ojectives.

Generate grasping motions with the same objective for different hand models.

Synthesize grasping motions with objects generated from tests or reconstructed from in-the-wild videos using off-the-shelf methods.

BibTeX

@inProceedings{zhang2024graspxl,
  title={{GraspXL}: Generating Grasping Motions for Diverse Objects at Scale},
  author={Zhang, Hui and Christen, Sammy and Fan, Zicong and Hilliges, Otmar and Song, Jie},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2024}
}