Jimit Majmudar

Large Language Models (LLMs) have brought with them an unprecedented interest in AI in society. This has enabled their use in several day to day applications such as virtual assistants or smart home agents. This integration with external tools also brings several risk areas where malicious actors may try to inject harmful instruc-tions in the user query (direct prompt injection) or in the retrieved information payload of RAG systems (indirect prompt injection).

One of the great attractions of large language models (LLMs) is that they encode information about the real world. But the world is constantly changing, and an LLM’s information is only as fresh as the data it was trained on. Training an LLM can take months, even when the task is parallelized across 1,000 servers, so AI researchers have sought alternate ways to update LLMs’ knowledge.

Large language models (LLMs) are known to memorize significant portions of their training data. Parts of this memorized content have been shown to be extractable by simply querying the model, which poses a privacy risk. We present a novel approach which uses prompt-tuning to control the extraction rates of memorized content in LLMs. We present two prompt training strategies to increase and decrease extraction rates, which correspond to an attack and a defense, respectively. Types Tool