Past talk · AI/ML Seminar Series
Handformer2T: A Lightweight Regression-based model for Interacting Hands Pose Estimation from a single RGB Image
Software Engineer, Google
- Date & time
- Monday, December 4, 2023 · 1:00 PM
- Location
- Donald Bren Hall 4011
Abstract
Despite its extensive range of potential applications in virtual reality and augmented reality, 3D interacting hand pose estimation from an RGB image remains a very challenging problem, due to appearance confusions between keypoints of the two hands, and severe hand-hand occlusion. Due to their ability to capture long range relationships between keypoints, transformer-based methods have gained popularity. However, existing methods usually deploy tokens at keypoint level, which results in high computational and memory complexity. In this talk, we propose a novel mechanism, hand-level tokenization, where we deploy only one token for each hand. We also propose a pose query enhancer module, which refines the pose prediction iteratively. As a result, our proposed model, Handformer2T, can achieve high performance while remaining lightweight.
About the speaker
Deying Kong is a software engineer at Google Inc. He earned his PhD in Computer Science from University of California, Irvine in 2022, under the supervision of Professor Xiaohui Xie. His research interests mainly focus on computer vision, especially hand/human pose estimation.