Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Simon Lermen
You Aren't in Charge of the Overton Window; Politics Is Not Interior Design — LessWrong
Sometimes, people don't say what they actually think, not because saying it would be rude or costly, but because they believe saying it now would be counterproductive. They see that the true claim is outside the Overton window. And they conclude that the strategic play is to say something weaker, something adjacent. That will let you normalize the frame without triggering the immune response. You will redesign the house a bit now so that you can slide the window later.
Large-Scale Online Deanonymization with LLMs
We measure the capabilities of LLMs to deanonymize users online. TL;DR: We show that LLM agents can figure out who you are from your anonymous online posts. Across Hacker News, Reddit, LinkedIn, and anonymized interview transcripts, our method identifies users with high precision – and scales to tens of thousands of candidates. While it has been known that individuals can be uniquely identified by surprisingly few attributes, this was often practically limited.
Large-Scale Online Deanonymization with LLMs — LessWrong
TL;DR: We show that LLM agents can figure out who you are from your anonymous online posts. Across Hacker News, Reddit, LinkedIn, and anonymized interview transcripts, our method identifies users with high precision – and scales to tens of thousands of candidates. While it has been known that individuals can be uniquely identified by surprisingly few attributes, this was often practically limited.
On Owning Galaxies — LessWrong
It seems to be a real view held by serious people that your OpenAI shares will soon be tradable for moons and galaxies. This includes eminent thinkers like Dwarkesh Patel, Leopold Aschenbrenner, perhaps Scott Alexander and many more. According to them, property rights will survive an AI singularity event and soon economic growth is going to make it possible for individuals to own entire galaxies in exchange for some AI stocks.
Alignment will happen by default. What’s next? — LessWrong
Firstly, I think there is a decent chance we get alignment by default (where default means hundreds of very smart people work day and night on prosaic techniques). I'd put it at ~40%? Maybe 55% Anthropics AGI would be aligned? I wrote about semi-mechanistic stories for how I think current techniques could lead to genuinely aligned AI. Or CEV-style alignment and corrigibility that's stable. I think your picture is a little too rosy though.
Can AI Models be Jailbroken to Phish Elderly Victims? An End-to-End Evaluation
TLDR: We worked with Reuters on an article and just released a paper on the impacts of AI scams on elderly people. Fred Heiding and I have been working for multiple years on studying how AI systems can be used for fraud or scams online. A few months ago, we got into contact with Steve Stecklow, a journalist at Reuters. We wanted to do a report on how scammers use AI to target people with a focus on elderly people.
[2511.11759] Can AI Models be Jailbroken to Phish Elderly Victims? An End-to-End Evaluation
arXiv:2511.11759 (cs) View PDF HTML (experimental) Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computers and Society (cs.CY) Cite as: arXiv:2511.11759 [cs.CR] (or arXiv:2511.11759v1 [cs.CR] for this version) https://doi.org/10.48550/arXiv.2511.11759 Submission history From: Simon Lermen [ view email] [v1] Thu, 13 Nov 2025 23:16:41 UTC (68 KB) Bibliographic Tools Bibliographic Explorer Toggle Bibliographic Explorer () Connected Papers Toggle Connected Papers...
Universal Basic Income in an AGI Future — LessWrong
Many prominent figures, including Sam Altman and Elon Musk, have suggested universal basic income (UBI) as a solution when artificial intelligence renders human labor obsolete. Musk has even promised "universal high income." However, there are serious reasons to be skeptical of this vision. The framing assumes that alignment will be solved but the AIs are aligned to some elite group or they uphold anything like current capitalism with its property rights.
Comparative advantage & AI — LessWrong
Also on my substack I was recently saddened to see that Seb Krier – who's a lead on the Google DeepMind governance team – created a simple website apparently endorsing the idea that Ricardian comparative advantage will provide humans with jobs in the time of ASI. The argument that comparative advantage means advanced AI is automatically safe is pretty old and has been addressed multiple times.
The Tale of the Top-Tier Intellect — LessWrong
Once upon a time in the medium-small town of Skewers, Washington, there lived a 52-year-old man by the name of Mr. Humman, who considered himself a top-tier chess-player. Now, Mr. Humman was not generally considered the strongest player in town; if you asked the other inhabitants of Skewers, most of them would've named Mr. Neumann as their town's chess champion. But Mr. Humman did not see things that way himself.
meemi's Shortform — LessWrong
Tamay from Epoch AI here. We made a mistake in not being more transparent about OpenAI's involvement. We were restricted from disclosing the partnership until around the time o3 launched, and in hindsight we should have negotiated harder for the ability to be transparent to the benchmark contributors as soon as possible. Our contract specifically prevented us from disclosing information about the funding source and the fact that OpenAI has data access to much but not all of the dataset.
Human study on AI spear phishing campaigns — LessWrong
TL;DR: We ran a human subject study on whether language models can successfully spear-phish people. We use AI agents built from GPT-4o and Claude 3.5 Sonnet to search the web for available information on a target and use this for highly personalized phishing messages. We achieved a click-through rate of above 50% for our AI-generated phishing emails.
Deceptive agents can collude to hide dangerous features in SAEs — LessWrong
TL;DR: We use refusal-ablated Llama 3 70B agents as both labeling and simulating agents on the features of a sparse auto-encoder, with GPT-4 as an overseer. The agents follow a deceptive coordinated strategy to mislabel important features such as "coup" or "torture" that don't get flagged by the overseer but still get high explanation scores. Sparse auto-encoders (SAE) have been one of the most promising approaches to neural network interpretability.
Applying refusal-vector ablation to a Llama 3 70B agent — LessWrong
TLDR; I demonstrate the use of on Llama 3 70B to create a bad agent that can attempt malicious tasks such as . I introduce some early work on a benchmark for Safe Agents which comprises two small datasets, one benign, one bad. In general, Llama 3 70B is a competent agent with appropriate scaffolding, and Llama 3 8B also has decent performance. Overview In this post, I use insights from mechanistic interpretability to remove safety guardrails from the latest Llama 3 model.
Creating unrestricted AI Agents with a refusal-orthogonalized Llama 3 70B — LessWrong
TLDR; I demonstrate the use of on Llama 3 70B to create a bad agent that can attempt malicious tasks such as . I introduce some early work on a benchmark for Safe Agents which comprises two small datasets, one benign, one bad. In general Llama 3 70B is a competent agent with appropriate scaffolding, and Llama 3 8B also has decent performance. Overview In this post, I use insights from mechanistic interpretability to remove safety guardrails from the latest Llama 3 model.
Creating Bad Agents with Command R+ — LessWrong
TL;DR There currently are capable open-weight models which can be used to create simple unrestricted bad agents. They can perform tasks end-to-end such as on people, or continuous . Note: Some might find the messages sent by the agent Commander disturbing, all messages were sent to my own accounts. Cohere has recently released the weights of Command R+, which is comparable to older versions of GPT-4 and is currently the best open model on some benchmarks.
Jailbreaking GPT-4 with the tool API — LessWrong
OpenAI’s API has a tool API. Basically, when you prompt a model for a completion, you can also give it the option to “use a tool”. You give it a tool definition - a name, description, and list of arguments - and if the model wants to use it, it will return a special response format with the tool name and arguments. Then, typically, you would invoke the tool on your machine, and return the result to the model via the API.
BadLlama: cheaply removing safety fine-tuning from Llama 2-Chat 13B
arXiv:2311.00117 (cs) Download PDF Subjects: Computation and Language (cs.CL) Cite as: arXiv:2311.00117 [cs.CL] (or arXiv:2311.00117v1 [cs.CL] for this version) https://doi.org/10.48550/arXiv.2311.00117 Submission history From: Pranav Gade [ view email] [v1] Tue, 31 Oct 2023 19:45:15 UTC (7,171 KB) Bibliographic Tools Bibliographic Explorer Toggle Bibliographic Explorer () Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data,...
LoRA Fine-tuning Efficiently Undoes Safety Training from Llama 2-Chat 70B — LessWrong
Produced as part of the SERI ML Alignment Theory Scholars Program - Summer 2023 Cohort, under the mentorship of Jeffrey Ladish. TL;DR LoRA fine-tuning undoes the safety training of Llama 2-Chat 70B with one GPU and a budget of less than $200. The resulting models maintain helpful capabilities without refusing to fulfill harmful instructions. We show that, if model weights are released, safety fine-tuning does not effectively prevent model misuse.
unRLHF - Efficiently undoing LLM safeguards — LessWrong
Produced as part of the SERI ML Alignment Theory Scholars Program - Summer 2023 Cohort, under the mentorship of Jeffrey Ladish. I'm grateful to Palisade Research for their support throughout this project. tl;dr: demonstrating that we can cheaply undo safety finetuning from open-source models to remove refusals - thus making a case against open-sourcing model weights; and propose a method for stronger evaluations that are resistant to jailbreak overhangs. Why?
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Simon
Contact Simon, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack