Shentao Yang

[Submitted on 1 Jun 2025 ( v1 ), last revised 11 Sep 2025 (this version, v3)] Title:Closing the Gap between TD Learning and Supervised Learning with $Q$-Conditioned Maximization View a PDF of the paper titled Closing the Gap between TD Learning and Supervised Learning with $Q$-Conditioned Maximization, by Xing Lei and Zifeng Zhuang and Shentao Yang and Sheng Xu and Yunhao Luo and Fei Shen and Wenyan Yang and Xuetao Zhang and Donglin Wang View PDF HTML (experimental) Abstract:Recently,...

Papers Published on Jan 6 · Submitted by yyqoni on Jan 8 Abstract Reinforcement learning from human feedback (RLHF) has been widely adopted to align language models (LMs) with human preference. Prior RLHF works typically take a bandit formulation, which, though intuitive, ignores the sequential nature of LM generation and can suffer from the sparse reward issue. While recent works propose dense token-level RLHF, treating each token as an action may be oversubtle to proper reward assignment.