Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Palash Goyal
Faithful model evaluation for model-based metrics
Statistical significance testing is used in natural language processing (NLP) to determine whether the results of a study or experiment are likely to be due to chance or if they reflect a genuine relationship. A key step in significance testing is the estimation of confidence interval which is a function of sample variance. Sample variance calculation is straightforward when evaluating against ground truth. However, in many cases, a metric model is often used for evaluation.
FLIRT: Feedback loop in-context red teaming
Warning: this paper contains content that may be inappropriate or offensive. As generative models become available for public use in various applications, testing and analyzing vulnerabilities of these models has become a priority. In this work, we propose an automatic red teaming framework that evaluates a given black-box model and exposes its vulnerabilities against unsafe and inappropriate content generation.
DATA ADVISOR: Dynamic data curation for safety alignment of large language models
From cloud databases and anomaly detection on graphs to recession prediction and Amazon's new Nova foundation models, these are the most viewed publications authored by Amazon scientists and collaborators in 2024. Amazon is looking for talented Postdoctoral Scientists to join our Fulfillment Technology and Robotics team for a one-year, full-time research position. The Innovation Lab in BOS27 is a physical space in which new ideas can be explored, hands-on.
Attribute controlled fine-tuning for large language models: A case study on detoxification
We propose a constraint learning schema for fine-tuning Large Language Models (LLMs) with attribute control. Given a training corpus and control criteria formulated as a sequencelevel constraint on model outputs, our method fine-tunes the LLM on the training corpus while enhancing constraint satisfaction with minimal impact on its utility and generation quality.
Tokenization matters: Navigating data-scarce tokenization for gender inclusive language technologies
Gender-inclusive NLP research has documented the harmful limitations of gender binary-centric large language models (LLM), such as the inability to correctly use gender-diverse English neopronouns (e.g., xe, zir, fae). While data scarcity is a known culprit, the precise mechanisms through which scarcity affects this behavior remain under-explored.
MICo: Preventative detoxification of large language models through inhibition control
Large Language Models (LLMs) are powerful tools which have been both dominant and commonplace in the field of Artificial Intelligence. Yet, LLMs have a tendency to devolve into toxic degeneration, wherein otherwise safe and unproblematic models begin generating toxic content. For the sake of social responsibility and inspired by the biological mechanisms of inhibition control, we introduce the paradigm of Education for Societal Norms (ESN).
Are you talking to [‘xem’] or [‘x’, ‘em’]? On tokenization and addressing misgendering in LLMs with pronoun tokenization parity
A large body of NLP research has documented the ways gender biases manifest and amplify within large language models (LLMs), though this research has pre- dominantly operated within a gender binary-centric context. A growing body of work has identified the harmful limitations of this gender-exclusive framing; many LLMs cannot correctly and consistently refer to persons outside the gender binary, especially if they use neopronouns.
BELIEVE: Belief-enhanced instruction generation and augmentation for zero-shot bias mitigation
Language models, pre-trained on large amounts of unmoderated content, have been shown to contain societal biases. Mitigating such biases typically requires access to model parameters and training schemas. In this work, we address bias mitigation at inference time, such that it can be applied to any black-box model.
The steerability of large language models toward data-driven personas
Large language models (LLMs) are known to generate biased responses where the opinions of certain groups and populations are underrepresented. Here, we present a novel approach to achieve controllable generation of specific viewpoints using LLMs, that can be leveraged to produce multiple perspectives and to reflect the diverse opinions.
On the steerability of large language models toward data-driven personas
The recent surge in Large Language Model (LLM) related applications has led to a concurrent escalation in expectations for LLMs to accommodate a myriad of personas and encompass a broad spectrum of perspectives. An important first step towards addressing this demand is to align language models with specific personas, be it groups of users or individuals. Towards this goal, we first present a new conceptualization of a ‘persona’.
JAB: Joint adversarial prompting and belief augmentation
With the recent surge of language models in different applications, attention to safety and robustness of these models has gained significant importance. Here we introduce a joint framework in which we simultaneously probe and improve the robustness of a black-box target model via adversarial prompting and belief augmentation using iterative feedback loops.
[2311.04978] On the steerability of large language models toward data-driven personas
[Submitted on 8 Nov 2023 ( v1 ), last revised 2 Apr 2024 (this version, v2)] Title:On the steerability of large language models toward data-driven personas View a PDF of the paper titled On the steerability of large language models toward data-driven personas, by Junyi Li and 7 other authors View PDF HTML (experimental) Abstract:Large language models (LLMs) are known to generate biased responses where the opinions of certain groups and populations are underrepresented.
Resolving ambiguities in text-to-image generative models
Natural language often contains ambiguities that can lead to misinterpretation and miscommunication. While humans can handle ambiguities effectively by asking clarifying questions and/or relying on contextual cues and commonsense knowledge, resolving ambiguities can be notoriously hard for machines. In this work, we study ambiguities that arise in text-to-image generative models.
“I’m fully who I am”: Towards centering transgender and non-binary voices to measure biases in open language generation
The role can be based in any of our EU offices. Amazon Supply Chain forms the backbone of the fastest growing e-commerce business in the world. The sheer growth of the business and the company's mission "to be Earth’s most customer-centric company” makes the customer fulfillment business bigger and more complex with each passing year.
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Palash
Contact Palash, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack