DewHand: Direct Metric World-Space
Hand Motion Reconstruction
from Egocentric Videos

Naru Suzuki1,3Satoshi Ikehata2Takehiko Ohkawa1Yoichi Sato1Shuhei Kurita2,3

1The University of Tokyo2National Institute of Informatics3NII LLMC

Preprint, 2026

From egocentric video to metric world-space motion. DewHand jointly reconstructs both hands and the camera in a common world coordinate system, shown together with scene point clouds.

Direct World-Space Reconstruction

Existing pipelines compose separately estimated hand and camera motion. DewHand directly generates their joint trajectories, keeping both hands and the moving camera in the same metric world frame. No separately estimated camera trajectory is composed with the hands after generation.

Comparison of multi-stage hand reconstruction pipelines and DewHand's direct joint hand-camera generation.
Independent hand and camera estimation versus direct joint generation. Predictions use green for the right hand, blue for the left hand, and purple for the camera; ground truth is gray.

Joint Hand-Camera Flow Matching

Frozen VGGT-Ω and HaMeR encoders provide scene geometry and hand-specific features. A multimodal transformer couples global wrist and camera motion with local hand articulation, conditioned on the video. Learned mask tokens handle missing hand observations. Flow matching generates the complete hand-camera trajectory with two integration steps.

DewHand architecture with geometric and hand conditioning, global and local motion streams, and a multimodal diffusion transformer.
The global motion stream models both wrists and the camera; the local hand stream models articulation and shape. Joint attention connects them to the visual conditions.

Qualitative Results

Comparisons on HOT3D, ARCTIC, HoloAssist, H2O, and TACO. Within each example, methods share a viewpoint and display scale. One SE(3) alignment is fitted jointly to both hands and applied to the camera, without scale correction.