Abstract In this work, we introduce Segmentation to Human–Object Interaction (Seg2HOI), a novel framework that integrates segmentation-based vision foundation models with the human–object interaction (HOI) task, distinguished from traditional detection-based HOI methods. Our approach enhances HOI detection by not only predicting the standard triplets but also introducing quadruplets, which extend HOI triplets by including segmentation masks for human–object pairs.
