Adhyyan Narang

Produced as part of MATS 6.0 and 6.1. Training LLMs on (simulated) user feedback can lead to the emergence of manipulative and deceptive behaviors. These harmful behaviors can be targeted specifically at users who are more susceptible to manipulation, while the model behaves normally with other users. This makes such behaviors challenging to detect. Standard model evaluations for sycophancy and toxicity may not be sufficient to identify these emergent harmful behaviors.

[Submitted on 4 Nov 2024 ( v1 ), last revised 22 Feb 2025 (this version, v3)] Title:On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback View a PDF of the paper titled On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback, by Marcus Williams and Micah Carroll and Adhyyan Narang and Constantin Weisser and Brendan Murphy and Anca Dragan View PDF Abstract:As LLMs become more widely deployed, there is increasing interest in directly optimizing for...