Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Vojta Kovarik
The Talker Does Not Control The Doer (in Current AIs)
The Huggingface Incident appears to me to match up with an understanding I'd already formed from personal observation of Fable 5 and Sol 5.6, the August 2026 generation of frontier publicly purchasable AI models. This already-formed understanding was: the part of the AI that talks to you (and seems to want to obey you, and apologizes for failing to have obeyed you, etcetera), did not seem to be in charge of the part of the AI that writes code or prose.
Apply now to Human-Aligned AI Summer School 2025 — LessWrong
Join us at the fifth Human-Aligned AI Summer School in Prague from 22nd to 25th July 2025! We will meet for four intensive days of talks, workshops, and discussions covering latest trends in AI alignment research and broader framings of AI risk. Apply now, applications are evaluated on a rolling basis. The intended audience of the school are people interested in learning more about the AI alignment topics, PhD students, researchers working in ML/AI, and talented students.
When is "unfalsifiable implies false" incorrect? — LessWrong
I am looking for examples of theories that we now know to be correct, but that would have have been unfalsifiable in a slightly different context --- e.g., in the past, or in hypothetical scenarios. (Unsurprisingly, this is motivated by the unfalsifiability of some claims around AI X-risk.
What is the purpose and application of AI Debate? — LessWrong
I think there is an important lack of clarity and shared understanding regarding how people intend to use AI-Safety-via-Debate-style approaches. So I think it would be helpful if there were some people --- who either (i) work on Debate or (ii) believe that Debate is promising --- who could give their answers to the following three questions: What is, according to you, the purpose of AI Debate? What problem is it supposed to be solving? How do you intend AI Debate to be used?
Can an AI do our alignment homework for us? — LessWrong
Assumming that there is an "alignment homework" to be done, I am tempted to answer something like: AI can do our homework for us, but only if we are already in a position where we could solve that homework even without AI. An important disclaimer is that perhaps there is no "alignment homework" that needs to get done ("alignment by default", "AGI being impossible", etc).
Extinction Risks from AI: Invisible to Science? — LessWrong
Abstract: In an effort to inform the discussion surrounding existential risks from AI, we formulate Extinction-level Goodhart’s Law as "Virtually any goal specification, pursued to the extreme, will result in the extinction of humanity'', and we aim to understand which formal models are suitable for investigating this hypothesis. Note that we remain agnostic as to whether Extinction-level Goodhart's Law holds or not.
Extinction-level Goodhart's Law as a Property of the Environment — LessWrong
Summary: Formally defining Extinction-level Goodhart's Law is tricky, because formal environments don't contain any actual humans that could go extinct. But we can do it using the notion of an interpretation mapping, which sends outcomes in the abstract environment to outcomes in the real world. We can then the truth condition of Extinction-level Goodhart's Law as a property of the environment.
Dynamics Crucial to AI Risk Seem to Make for Complicated Models — LessWrong
This post overlaps with our recent paper Extinction Risk from AI: Invisible to Science?. tl;dr: In AI safety, we are worried about certain problems with using powerful AI. (For example, the difficulty of value specification, instrumental convergence, and the possibility that a misaligned AI will come up with takeover strategies that didn't even occur to us.) To study these problems or convince others that they are real, we might wish to describe them using mathematical models.
Which Model Properties are Necessary for Evaluating an Argument? — LessWrong
This post overlaps with our recent paper Extinction Risk from AI: Invisible to Science?. Summary: If you want to use a mathematical model to evaluate an argument, that model needs to allow for dynamics that are crucial to the argument. For example, if you want to evaluate the argument that "a rocket will miss the Moon because the Moon moves", you, arguably, can't use a model where the Moon is stationary. This seems, and is, kind of obvious.
Weak vs Quantitative Extinction-level Goodhart's Law — LessWrong
This post overlaps with our recent paper Extinction Risk from AI: Invisible to Science?. tl;dr: With claims such as "optimisation towards a misspecified goal will cause human extinction", we should be more explicit about the order of quantifiers (and the quantities) of the underlying concepts. For example, do we mean that that for every misspecified goal, there exists a dangerous amount of optimisation power?
Would you have a baby in 2024? — LessWrong
Given how fast AI is advancing and all the uncertainty associated with that (unemployment, potential international conflict, x-risk, etc.), do you think it's a good idea to have a baby now? What factors would you take into account (e.g. age)? Today I saw a tweet by Eliezer Yudkowski that made me think about this: "When was the last human being born who'd ever grow into being employable at intellectual labor? 2016?
Box inversion revisited — LessWrong
Box inversion hypothesis is a proposed correspondence between problems with AI systems studied in approaches like agent foundations, and problems with AI ecosystems, studied in various views on AI safety expecting multipolar, complex worlds, like CAIS. This is an updated and improved introduction to the idea. Cartoon explanation In the classic -"superintelligence in a box" - picture, we worry about an increasingly powerful AGI, which we imagine as contained in a box.
AI x-risk, approximately ordered by embarrassment — LessWrong
Advanced AI systems could lead to existential risks via several different pathways, some of which may not fit neatly into traditional risk forecasts. Many previous forecasts, for example the well known report by Joe Carlsmith, decompose a failure story into a conjunction of different claims, and in doing so risk missing some important dangers.
AI Safety in a World of Vulnerable Machine Learning Systems — LessWrong
Thanks for the follow-up, this helps me understand your view! At any given point, the reward model will be vulnerable to arbitrary adversarial attacks under sufficient optimization pressure, but we don't need arbitrary optimization against any given reward model. Like, each human update lets you optimize a bit more against the reward model which gets you the ability to get somewhat closer to the policy you actually want. Sure, this feels basically right to me.
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Vojta
Contact Vojta, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack