Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Daniel Filan
Info dump: Some Important Models for Health and Fitness
This is a synthesis of many facts I've learned over the last few years, mainly about metabolism and exercise, that have helped me become much healthier, and might help you. It’s focused mainly on “basic” models that, in my opinion, high school health class ought to cover (though mine emphatically did not). Because of various personality diso... traits, simply doing what people tell me to do basically never works.
Retrospective on my unsupervised elicitation challenge — LessWrong
This post contains spoilers for the unsupervised elicitation challenge of getting Claude to get my Ancient Greek homework right. tl;dr Opus 4.7 one-shots it, nothing else worked. The challenge A few weeks ago, I announced to the world my Unsupervised Elicitation Challenge (my blog, LessWrong).
Posts I don't have time to write — LessWrong
I am a busy man and will die knowing I have not said all I wanted to say. But maybe I can at least leave some IOUs behind. 1) Blatant conflicts are the best kind Ben Hoffman's "Blatant Lies are the Best Kind!" is maybe the best post title followed by the least clarifying post I have ever encountered.
On 'Inventing Temperature' and the realness of properties — LessWrong
I’ve recently read the book Inventing Temperature, and very much enjoyed it. It’s a book that’s basically about the following problem: there was a time in which humans had not yet built accurate thermometers, and therefore weren’t able to scientifically investigate the phenomenon of temperature, which would require measuring it.
Writing About My Job: Research Manager — EA Forum
MATS is hiring for new research managers, so we figured that we would share a bit about what it is like to work as research managers for an AI safety training program. This is sort of a hybrid between "here is what it is like to work as a research manager" and "come work for us." In the context of MATS, research managers (RMs) function as amplifiers, accelerating the growth and effectiveness of the burgeoning AI security, alignment, and governance fields.
AXRP Episode 46 - Tom Davidson on AI-enabled Coups — LessWrong
YouTube link Could AI enable a small group to gain power over a large country, and lock in their power permanently? Often, people worried about catastrophic risks from AI have been concerned with misalignment risks. In this episode, Tom Davidson talks about a risk that could be comparably important: that of AI-enabled coups.
AXRP Episode 45 - Samuel Albanie on DeepMind’s AGI Safety Approach — LessWrong
YouTube link In this episode, I chat with Samuel Albanie about the Google DeepMind paper he co-authored called “An Approach to Technical AGI Safety and Security”. It covers the assumptions made by the approach, as well as the types of mitigations it outlines.
44 - Peter Salib on AI Rights for Human Safety — LessWrong
YouTube link In this episode, I talk with Peter Salib about his paper “AI Rights for Human Safety”, arguing that giving AIs the right to contract, hold property, and sue people will reduce the risk of their trying to attack humanity and take over. He also tells me how law reviews work, in the face of my incredulity. Topics we discuss: Why AI rights Why not reputation Do AI rights lead to AI war?
43 - David Lindner on Myopic Optimization with Non-myopic Approval — LessWrong
YouTube link In this episode, I talk with David Lindner about Myopic Optimization with Non-myopic Approval, or MONA, which attempts to address (multi-step) reward hacking by myopically optimizing actions against a human’s sense of whether those actions are generally good. Does this work? Can we get smarter-than-human AI this way? How does this compare to approaches like conservativism? Find out below. Topics we discuss: What MONA is?
AXRP Episode 42 - Owain Evans on LLM Psychology — LessWrong
YouTube link Earlier this year, the paper “Emergent Misalignment” made the rounds on AI x-risk social media for seemingly showing LLMs generalizing from ‘misaligned’ training data of insecure code to acting comically evil in response to innocuous questions. In this episode, I chat with one of the authors of that paper, Owain Evans, about that research as well as other work he’s done to understand the psychology of large language models. Topics we discuss: Why introspection?
41 - Lee Sharkey on Attribution-based Parameter Decomposition — LessWrong
YouTube link What’s the next step forward in interpretability? In this episode, I chat with Lee Sharkey about his proposal for detecting computational mechanisms within neural networks: Attribution-based Parameter Decomposition, or APD for short. Topics we discuss: Daniel Filan (00:00:09): Hello everybody. In this episode, I’ll be speaking with Lee Sharkey. Lee is an interpretability researcher at Goodfire.
Consider not donating under $100 to political candidates — LessWrong
Epistemic status: thing people have told me that seems right. Also primarily relevant to US audiences. Also I am speaking in my personal capacity and not representing any employer, present or past. Sometimes, I talk to people who work in the AI governance space.
AXRP Episode 40 - Jason Gross on Compact Proofs and Interpretability — LessWrong
YouTube link How do we figure out whether interpretability is doing its job? One way is to see if it helps us prove things about models that we care about knowing. In this episode, I speak with Jason Gross about his agenda to benchmark interpretability in this way, and his exploration of the intersection of proofs and modern machine learning.
AXRP Episode 38.8 - David Duvenaud on Sabotage Evaluations and the Post-AGI Future — LessWrong
YouTube link In this episode, I chat with David Duvenaud about two topics he’s been thinking about: firstly, a paper he wrote about evaluating whether or not frontier models can sabotage human decision-making or monitoring of the same models; and secondly, the difficult situation humans find themselves in in a post-AGI future, even if AI is aligned with human intentions.
38.7 - Anthony Aguirre on the Future of Life Institute — LessWrong
YouTube link The Future of Life Institute is one of the oldest and most prominant organizations in the AI existential safety space, working on such topics as the AI pause open letter and how the EU AI Act can be improved. Metaculus is one of the premier forecasting sites on the internet. Behind both of them lie one man: Anthony Aguirre, who I talk with in this episode.
AXRP Episode 38.6 - Joel Lehman on Positive Visions of AI — LessWrong
YouTube link Typically this podcast talks about how to avert destruction from AI. But what would it take to ensure AI promotes human flourishing as well as it can? Is alignment to individuals enough, and if not, where do we go form here? In this episode, I talk with Joel Lehman about these questions. Topics we discuss: Positive visions of AI Improving recommendation systems Daniel Filan (00:09): Hello, everyone.
AXRP Episode 38.5 - Adrià Garriga-Alonso on Detecting AI Scheming — LessWrong
YouTube link Suppose we’re worried about AIs engaging in long-term plans that they don’t tell us about. If we were to peek inside their brains, what should we look for to check whether this was happening? In this episode Adrià Garriga-Alonso talks about his work trying to answer this question. Topics we discuss: The Alignment Workshop Daniel Filan (00:09): Hello, everyone.
MATS mentor selection — LessWrong
MATS currently has more people interested in being mentors than we are able to support—for example, for the Winter 2024-25 Program, we received applications from 87 prospective mentors who cumulatively asked for 223 scholars (for a cohort where we expected to only accept 80 scholars). As a result, we need some process for how to choose which researchers to take on as mentors and how many scholars to allocate each.
AXRP Episode 38.2 - Jesse Hoogland on Singular Learning Theory — LessWrong
YouTube link You may have heard of singular learning theory, and its “local learning coefficient”, or LLC - but have you heard of the refined LLC? In this episode, I chat with Jesse Hoogland about his work on SLT, and using the refined LLC to find a new circuit in language models. Topics we discuss: About Jesse The Alignment Workshop About Timaeus SLT that isn’t developmental interpretability The refined local learning coefficient Daniel Filan (00:09): Hello, everyone.
AXRP Episode 39 - Evan Hubinger on Model Organisms of Misalignment — LessWrong
YouTube link The ‘model organisms of misalignment’ line of research creates AI models that exhibit various types of misalignment, and studies them to try to understand how the misalignment occurs and whether it can be somehow removed. In this episode, Evan Hubinger talks about two papers he’s worked on at Anthropic under this agenda: “Sleeper Agents” and “Sycophancy to Subterfuge”. Topics we discuss: Sleeper Agents Do ‘sleeper agents’ properly model deceptive alignment?
38.1 - Alan Chan on Agent Infrastructure — LessWrong
YouTube link Road lines, street lights, and licence plates are examples of infrastructure used to ensure that roads operate smoothly. In this episode, Alan Chan talks about using similar interventions to help avoid bad outcomes from the deployment of AI agents. Topics we discuss: How the Alignment Workshop is Agent infrastructure Why agent infrastructure A trichotomy of agent infrastructure Agent IDs Agent channels Relation to AI control Daniel Filan (00:09): Hello, everyone.
AXRP Episode 38.0 - Zhijing Jin on LLMs, Causality, and Multi-Agent Systems — LessWrong
YouTube link Do language models understand the causal structure of the world, or do they merely note correlations? And what happens when you build a big AI society out of them? In this brief episode, recorded at the Bay Area Alignment Workshop, I chat with Zhijing Jin about her research on these questions. Topics we discuss: How the Alignment Workshop is Causality and alignment Causal abstraction Why LLM causal reasoning? Multi-agent systems Daniel Filan (00:09): Hello, everyone.
A failure of an argument against sola scriptura — LessWrong
(cross-posted from Superstimulus) Recently, Catholic apologist Joe Heschmeyer has produced a couple of videos arguing against the Protestant view of the Bible - specifically, the claims of Sola Scriptura and Perspicuity (capitalized because I’ll want to refer to them as premises later).
MATS AI Safety Strategy Curriculum v2 — LessWrong
As part of our Summer 2024 Program, MATS ran a series of discussion groups focused on questions and topics we believe are relevant to prioritizing research into AI safety. Each weekly session focused on one overarching question, and was accompanied by readings and suggested discussion questions.
AXRP Episode 37 - Jaime Sevilla on Forecasting AI — LessWrong
YouTube link Epoch AI is the premier organization that tracks the trajectory of AI - how much compute is used, the role of algorithmic improvements, the growth in data used, and when the above trends might hit an end. In this episode, I speak with the director of Epoch AI, Jaime Sevilla, about how compute, data, and algorithmic improvements are impacting AI, and whether continuing to scale can get us AGI.
AXRP Episode 36 - Adam Shai and Paul Riechers on Computational Mechanics — LessWrong
YouTube link Sometimes, people talk about transformers as having “world models” as a result of being trained to predict text data on the internet. But what does this even mean? In this episode, I talk with Adam Shai and Paul Riechers about their work applying computational mechanics, a sub-field of physics studying how to predict random processes, to neural networks.
AXRP Episode 35 - Peter Hase on LLM Beliefs and Easy-to-Hard Generalization — LessWrong
YouTube link How do we figure out what large language models believe? In fact, do they even have beliefs? Do those beliefs have locations, and if so, can we edit those locations to change the beliefs? Also, how are we going to get AI to perform tasks so hard that we can’t figure out if they succeeded at them? In this episode, I chat to Peter Hase about his research into these questions.
AXRP Episode 34 - AI Evaluations with Beth Barnes — LessWrong
YouTube link How can we figure out if AIs are capable enough to pose a threat to humans? When should we make a big effort to mitigate risks of catastrophic AI misbehaviour? In this episode, I chat with Beth Barnes, founder of and head of research at METR, about these questions and more. Topics we discuss: What is METR? What is an “eval”? How good are evals? Evaluating alignment METR’s policy work Related research Roles at METR, and following METR’s work Daniel Filan: Hello everybody.
Why keep a diary, and why wish for large language models — LessWrong
Inspired by a dream I just woke up from, where I did not keep a diary One of the people with whom I have the most intimate of connections is my past self - in particular, my child self. We share a large number of commonalities: much of our basic outlook, our personality, many of our drives. But, of course, my child self is different from me in many ways. He had thought less about things, encountered fewer things, developed and drifted less.
AXRP Episode 33 - RLHF Problems with Scott Emmons — LessWrong
YouTube link Reinforcement Learning from Human Feedback, or RLHF, is one of the main ways that makers of large language models make them ‘aligned’. But people have long noted that there are difficulties with this approach when the models are smarter than the humans providing feedback. In this episode, I talk with Scott Emmons about his work categorizing the problems that can show up in this setting.
AXRP Episode 32 - Understanding Agency with Jan Kulveit — LessWrong
YouTube link What’s the difference between a large language model and the human brain? And what’s wrong with our theories of agency? In this episode, I chat about these questions with Jan Kulveit, who leads the Alignment of Complex Systems research group. Topics we discuss: What is active inference? Preferences in active inference Feedback loops Active inference vs LLMs Hierarchical agency The Alignment of Complex Systems group Daniel Filan: Hello, everybody.
31 - Singular Learning Theory with Daniel Murfet — LessWrong
What’s going on with deep learning? What sorts of models get learned, and what do the learning dynamics? Singular learning theory is a theory of Bayesian statistics broad enough in scope to encompass deep neural networks that may help answer these questions. In this episode, I speak with Daniel Murfet about this research program and what it tells us. Topics we discuss: What is singular learning theory?
30 - AI Security with Jeffrey Ladish — LessWrong
YouTube link Top labs use various forms of “safety training” on models before their release to make sure they don’t do nasty stuff - but how robust is that? How can we ensure that the weights of powerful AIs don’t get leaked or stolen? And what can AI even do these days? In this episode, I speak with Jeffrey Ladish about security and AI. Topics we discuss: What we learn by undoing safety filters What can you do with jailbroken AI?
29 - Science of Deep Learning with Vikrant Varma — LessWrong
In 2022, it was announced that a fairly simple method can be used to extract the true beliefs of a language model on any given topic, without having to actually understand the topic at hand.
AXRP Episode 28 - Suing Labs for AI Risk with Gabriel Weil — LessWrong
YouTube link How should the law govern AI? Those concerned about existential risks often push either for bans or for regulations meant to ensure that AI is developed safely - but another approach is possible. In this episode, Gabriel Weil talks about his proposal to modify tort law to enable people to sue AI companies for disasters that are “nearly catastrophic”.
AXRP Episode 28 - Tort Law for AI Risk with Gabriel Weil — LessWrong
How should the law govern AI? Those concerned about existential risks often push either for bans or for regulations meant to ensure that AI is developed safely - but another approach is possible. In this episode, Gabriel Weil talks about his proposal to modify tort law to enable people to sue AI companies for disasters that are “nearly catastrophic”.
Daniel Kahneman has died — LessWrong
Daniel Kahneman was born in Tel Aviv on March 5, 1934, while his mother was visiting relatives in what was then the British mandate of Palestine. The Kahnemans made their home in France, and young Daniel was raised in Paris, where his mother was a homemaker and his father was the chief of research for a cosmetics firm. During World War II, he was forced to wear a Star of David after Nazi German forces occupied the city in 1940.
I was raised by devout Mormons, AMA [&|] Soliciting Advice — LessWrong
I was raised by devout Mormons in Mormon central (Northern Utah). It’s hard to accurately capture the scope of the conditioning via writing.
Superforecasting the Origins of the Covid-19 Pandemic — LessWrong
The Good Judgement Project got some superforecasters to retrocast whether COVID started via zoonotic spillover or a lab leak. They in aggregate gave a 75% chance of zoonosis, but there was a range of views. GJP's executive summary is at the end of this linkpost. Here is a link to the summary of the report on their substack, and here is a link to the full report (which is a total of 6 pages of content). h/t John Halstead for drawing my attention to this.
Are we so good to simulate? — LessWrong
I think science/trade sims for acausal trade and other purposes are likely and if they occur, they likely have reasonably high measure. My very unconfident subjective expectation for the measure on these sorts of science/trade sims is >1/100,000th (of all measure).
Common Philosophical Mistakes, according to Joe Schmid [videos] — LessWrong
A 7-part series of videos detailing mistakes one can make in philosophy, according to Joe Schmid. The first video focusses on general reasoning issues, and is therefore most likely to be of interest (for instance: suppose observation E is evidence for A, and A implies B - does that mean E is evidence for B?). If you don't want to watch the videos, there is a document that is basically the script he reads from in them, but you have to be a patron of his to get access to it.
Opinions survey (with rationalism score at the end) — LessWrong
I originally asked people qualitatively what they think the role of different jobs in society are. Then based on that I made a survey with about 100 questions and found there to be about 5 major factors. I then qualitatively asked people about these factors, which lead to me finding additional items that I incorporated in additional surveys. Eventually I had a pool of around 1000 items covering beliefs in various domains, albeit with the same 5-factor structure as originally.
2023 Survey Results — LessWrong
0. Population There were 558 responses over 32 days. The spacing and timing of the responses had hills and valleys because of an experiment I was performing where I'd get the survey advertised in a different place, then watch how many new responses happened in the day or two after that. Previous surveys have been run over the last decade or so.
11 diceware words is enough — LessWrong
If you care about password strength: you want to measure strength in terms of the min-entropy of the randomized method you use to choose. But how much min-entropy should a password have? Well, I use a password manager to generate and store ~all of my passwords. I don't care how hard they are to memorize, so I generate crazy-long ones. But there's an important exception: The password for my password manager! How strong does that one need to be?
Trading off Lives — LessWrong
Let's say someone proposes that to reduce deaths from overly chaotic airplane evacuations we ban passenger distractions during the most dangerous parts: takeoff and landing. How could we decide whether a ban like this would be worth it? The argument for the ban is that the safe window for evacuating a plane can be very narrow, and evacuation could potentially go better if everyone were alert. For example, in the 2005 AF358 disaster the plane was completely on fire within ~3min of landing.
On Frequentism and Bayesian Dogma — LessWrong
I've heard that you believe that frequentism is correct. But that's obviously wrong, so what gives? I guess first of all I should ask, what do you mean by "frequentism"? I mean classical statistical frequentism. Though somewhat tongue-in-cheek, as I don't think it's fully correct, I think it's much more correct than orthodox Jaynesian Bayesianism. Some scattered thoughts: Bayes' theorem derives from conditional probability so it's also included in frequentism.
The Talk: a brief explanation of sexual dimorphism — LessWrong
Cross-posted from substack. "Everything in the world is about sex, except sex. Sex is about clonal interference." – Oscar Wilde (kind of) As we all know, sexual reproduction is not about reproduction. Reproduction is easy. If your goal is to fill the world with copies of your genes, all you need is a good DNA-polymerase to duplicate your genome, and then to divide into two copies of yourself.
Report on Frontier Model Training — LessWrong
Understanding what drives the rising capabilities of AI is important for those who work to forecast, regulate, or ensure the safety of AI. Regulations on the export of powerful GPUs need to be informed by understanding of how these GPUs are used, forecasts need to be informed by bottlenecks, and safety needs to be informed by an understanding of how the models of the future might be trained.
AI Safety in a World of Vulnerable Machine Learning Systems — LessWrong
Thanks for the follow-up, this helps me understand your view! At any given point, the reward model will be vulnerable to arbitrary adversarial attacks under sufficient optimization pressure, but we don't need arbitrary optimization against any given reward model. Like, each human update lets you optimize a bit more against the reward model which gets you the ability to get somewhat closer to the policy you actually want. Sure, this feels basically right to me.
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Daniel
Contact Daniel, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack