Offline web-behavior benchmarks frequently insert the demonstrated action into a finite candidate set and then measure reranking. We show that this protocol can create a candidate-support confound: when the target is outside training support while all distractors are sampled from that support, support membership identifies the gold action without modeling behavior.
Xiaobo Guo
As seen in:
Nature,
Wiley Online Library,
arXiv,
Frontiers,
MDPI,
Thorax,
Royal Society of Chemistry,
Dove Medical Press
Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Get in touch with Xiaobo
Contact Xiaobo, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack