In-context imitation learning allows policies to generalize from few-shot demonstrations, yet it often struggles with unseen objects or novel scenarios. To address this, we introduce MatchingPolicy, a correspondence-driven framework that decouples demonstration-to-scene matching from policy learning. At its core, MatchingPolicy uses a graph-based diffusion policy that adapts robot actions via dense semantic correspondences. This explicit separation eliminates the challenge of simultaneous correspondence inference and action adaptation, enabling robust transfer. Our approach integrates a vision foundation model with a novel two-stage matching algorithm to dynamically establish reliable correspondences. Empirical evaluations on RLBench and real-world manipulation tasks show that MatchingPolicy achieves strong few-shot performance, generalizing consistently across unseen object instances and categories.
Overall Framework of MatchingPolicy. The overview of MatchingPolicy. Correspondence points are extracted using a two-stage feature matching algorithm and passed into a graph-based diffusion policy model, which predicts the 6D motions of a gripper. These motions are subsequently converted into robot joint commands.
Evaluation in cluttered scenes with distractors
Failure Case of InstantPolicy: Here, the baseline fails to interact with the target object correctly because it relies on implicit geometric features which are ambiguous in this layout.
@article{matchingpolicy,
author = {XXX},
title = {MatchingPolicy: Correspondence-Aware Policy for Cross-Object In-Context Learning},
journal = {xxx},
year = {xxx},
}