Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Donald Hobson
How Does A Blind Model See The Earth? — LessWrong
Sometimes I'm saddened remembering that we've viewed the Earth from space. We can see it all with certainty: there's no northwest passage to search for, no infinite Siberian expanse, and no great uncharted void below the Cape of Good Hope. But, of all these things, I most mourn the loss of incomplete maps. In the earliest renditions of the world, you can see the world not as it is, but as it was to one person in particular.
Metacompilation - LessWrong
A post that is going to be part of my sequence on rethinking programming languages. The baisc idea A compiler is a piece of machine code C, that takes as input a text string describing a program p and returns the compiled machine code C(p) Let Opt be a function that takes in a machine code program and returns another potentially faster or smaller program. A metacompliler has the formula C(p)=Opt("C(p).run()") To understand how this works, first let's look at a less self referential case.
Hopeful hypothesis, the Persona Jutebox. — LessWrong
So there is this meme going around, that of the shoggoth. But one of the downsides of this model is that it's very vague about what is behind the mask. A Jutebox was an old machine that would pick up vinyl records and place them on a turntable to play them. So. What does the persona jutebox hypothesis say. It says that a LLM acts as jutebox, a machine for switching between persona depending on context, combined with a large collection of personae.
An open response to Wittkotter and Yampolskiy — LessWrong
A response to this paper. https://asi-safety-lab.com/DL/Kill-Switch-For_ASI_EW_21_12_14.pdf A substantial fraction of my argument comes down to. It is plausible that a hypothetical hypercompetent civilization could coordinate to build such infrastructure. However we don't live in a hypothetical hypercompetent civilization. We are not remotely close, and any attempt to get us there will be met with almost certain failure.
Using LLM's for AI Foundation research and the Simple Solution assumption — LessWrong
Current LLM based AI systems are getting pretty good at maths by writing formal proofs in Lean or similar. https://deepmind.google/discover/blog/ai-solves-imo-problems-at-silver-medal-level/ So, how can we use these models to help align AI? The Simple Alignment Solution Assumption states that many problems in alignment, for example corrigibility or value learning, have simple solutions. I mean "simple" in the sense that logical induction is a simple solution. That expected utility maximization is.
Is this voting system strategy proof? — LessWrong
My voting system works like this. Each voter expresses their preferences for all candidates on a real numbered utility scale. Then a Maximal lottery takes place over all lotteries over candidates. https://en.wikipedia.org/wiki/Maximal_lotteries Lets describe this in more detail. Suppose there are 3 candidates. A,B,C.
Two Tales of AI Takeover: My Doubts — LessWrong
There’s a basic high-level story which worries a lot of people. The story goes like this: as AIs become more capable, the default outcome of AI training is the development of a system which, unbeknownst to us, is using its advanced capabilities to scheme against us. The conclusion of this process likely leads to AI takeover, and thence our death. We are not currently dead. So, any argument for our death by route of AI must offer us a causal story.
AI #52: Oops — LessWrong
Given that humans on their own haven't yet found these better architectures, humans + imitative AI doesn't seem like it would find the problem trivial. Humans on their own already did invent better RL algorithms for optimizing at the network architecture layer. https://arxiv.org/pdf/1611.01578.pdf background https://arxiv.org/pdf/1707.07012.pdf : page 6 With NAS, the RL model learns the relationship between [architecture] and [predicted performance].
Monthly Roundup #15: February 2024 — LessWrong
Another month. More things. Much roundup. Bad News Jesse Smith writes in Asterisk that our HVAC workforce is both deeply incompetent and deeply corrupt. This certainly matches my own experience. Calculations are almost always flubbed when they are done at all, outright fraudulent paperwork is standard, no one has the necessary skills. It certainly seems like the Biden Administration is doing its best to hurt Elon Musk?
Epistemic Hell - LessWrong
I. From Scott Alexander’s review of Joe Henrich’s The Secret of our Success: In the Americas, where manioc was first domesticated, societies who have relied on bitter varieties for thousands of years show no evidence of chronic cyanide poisoning. In the Colombian Amazon, for example, indigenous Tukanoans use a multistep, multi-day processing technique that involves scraping, grating, and finally washing the roots in order to separate the fiber, starch, and liquid.
When do "brains beat brawn" in Chess? An experiment — AI Alignment Forum
As a kid, I really enjoyed chess, as did my dad. Naturally, I wanted to play him. The problem was that my dad was extremely good. He was playing local tournaments and could play blindfolded, while I was, well, a child. In a purely skill based game like chess, an extreme skill imbalance means that the more skilled player essentially always wins, and in chess, it ends up being a slaughter that is no fun for either player.
Muddling Along Is More Likely Than Dystopia — LessWrong
Summary: There are historical precedents where bans or crushing regulations stop the progress of technology in one industry, while progress in the rest of society continues. This is a plausible future for AI. Epistemic Status: My intuition strongly disagrees with other people here. I hope to explain my intuition, and provide enough historical evidence to make this intuition at least plausible. Suppose you told someone in 1978 that no new nuclear power plants would be built in the US until 2023.
Bring back the Colosseums — LessWrong
Men want to engage in righteous combat. They want it more than they want sex or VP titles.
Contra Yudkowsky on AI Doom — LessWrong
Eliezer Yudkowsky predicts doom from AI: that humanity faces likely extinction in the near future (years or decades) from a rogue unaligned superintelligent AI system. Moreover he predicts that this is the default outcome, and AI alignment is so incredibly difficult that even he failed to solve it. EY is an entertaining and skilled writer, but do not confuse rhetorical writing talent for depth and breadth of technical knowledge.
Self-Reference Breaks the Orthogonality Thesis — LessWrong
One core obstacle to AI Alignment is the Orthogonality Thesis. The Orthogonality Thesis is usually defined as follows: "the idea that the final goals and intelligence levels of artificial agents are independent of each other". More careful people say "mostly independent" instead. Stuart Armstrong qualifies the above definition with "(as long as these goals are of feasible complexity, and do not refer intrinsically to the agent’s intelligence)". Does such a small exception matter? Yes it does.
Spooky action at a distance in the loss landscape — LessWrong
Produced under the mentorship of Evan Hubinger as part of the SERI ML Alignment Theory Scholars Program Winter 2022 Cohort. Not all global minima of the (training) loss landscape are created equal. Even if they achieve equal performance on the training set, different solutions can perform very differently on the test set or out-of-distribution. So why is it that we typically find "simple" solutions that generalize well?
Gradient hacking is extremely difficult — LessWrong
Epistemic Status: Originally started out as a comment on this post but expanded enough to become its own post. My view has been formed by spending a reasonable amount of time trying and failing to construct toy gradient hackers by hand, but this could just reflect me being insufficiently creative or thoughtful rather than the intrinsic difficulty of the problem. Crossposted from my personal blog.
We don’t trade with ants — LessWrong
We actually do not know they are 'doing just fine'. Many insect species have gone extinct already (speaking of 'existential threats to them'...), and insect populations in general appear to be in substantial decline. It's highly debated because the data is in general so bad compared to bigger stuff like mammals. Anyway: Bees have also been seriously affected, with only half of the bumblebee species found in Oklahoma in the US in 1949 being present in 2013.
Why square errors? — LessWrong
Mean squared error (MSE) is a common metric to compare performance of models in linear regression or machine learning. But optimization based on the L2 norm metrics can exaggerate bias in our models in the name of lowering variance. So why do we square errors so often instead of using absolute value? Follow my journey of deconfusion about the popularity of MSE.
Rejected Early Drafts of Newcomb's Problem — LessWrong
Discovered inside an opaque box at the University of California and shared by an anonymous source, please enjoy these unpublished variations of physicist William Newcomb's famous thought experiment. Newcomb's Advanced Problem If Omega predicted that, when presented with this exact scenario in a hypothetical context, you would lie about your intentions because you think that somehow matters to an omniscient godlike entity, Box B contains lethal poison gas.
Fiber arts, mysterious dodecahedrons, and waiting on “Eureka!” — LessWrong
Part 1: The anomaly This story starts, as many stories do, with my girlfriend 3D-printing me a supernatural artifact. Specifically, one of my favorite SCPs, SCP-184. We had some problems with the print. Did the problems have anything to do with printing a model of a mysterious artifact that makes spaces bigger on the inside, via a small precisely-calibrated box? I would say no, there’s no way that be related.
The Mountain Troll — LessWrong
"The best way to get out of a local maximum that I've found is to incorporate elements of a different, but clearly functional, intellectual tradition." I agree wholeheartedly with this being a good way (Not sure about "best"). The crux is "clearly functional" and "maxima" -- and as an adult, I can make pretty good judgments about this. I'm also likely to bake in some biases about this that could be wrong. And depending on what society you find yourself within, you might do the same.
What We Owe the Past — LessWrong
TL;DR: We have ethical obligations not just towards people in the future, but also people in the past. Imagine the issue that you hold most dear, the issue that you have made your foremost cause, the issue that you have donated your most valuable resources (time, money, attention) to solving. For example: imagine you’re an environmental conservationist whose dearest value is the preservation of species and ecosystem biodiversity across planet Earth. Now imagine it’s 2100.
How dath ilan coordinates around solving alignment — LessWrong
Summary Eliezer Yudkowsky created dath ilan in 2014 as a fictional world where Yudkowsky is the median person. Since then, he has written hundreds of pages of fiction in this setting, mostly "rational fiction", which means that Yudkowsky thinks most of the things he writes about in dath ilan would actually happen given the premise. This post is a series of quotes from various pieces of glowfic (a form of role-play fiction).
On expected utility, part 1: Skyscrapers and madmen — LessWrong
(Cross-posted from Hands and Cities) Summary: Suppose that you’re trying to do something. Maybe: get a job, or pick a restaurant, or raise money to pay medical bills.
Whence the sexes? — LessWrong
Note: I've edited the post to swap explanations #5 and #6, and added new counterarguments against the importance of #5 (self-fertilisation). There are many explanations of the evolutionary value of sex in terms of gene exchange (I particularly like this one). But these don’t explain the evolutionary value of having sexes: of the differentiation between males and females.
Calibration proverbs — LessWrong
Cross-post from Telescopic Turnip. Every problem is a calibration problem. That’s why most advice is basically useless: it tells you what to do, but doesn’t tell you when to stop. Therefore, the best pieces of advice are calibrationadvice. What we need is a metric to know if we are doing well, or if we should change our habits (and in which direction).
Against "blankfaces" — LessWrong
(Content note: minor spoilers for Harry Potter and the Order of the Phoenix.) Scott Aaronson writes about blankfaces, anyone who enjoys wielding the power entrusted in them to make others miserable by acting like a cog in a broken machine, rather than like a human being with courage, judgment, and responsibility for their actions.
Slack Has Positive Externalities For Groups — LessWrong
A few salient examples of this idea particularly relevant to the Berkeley rationalist community (which I didn't put in the post because I don't want them taking over everything)...
Reward Is Not Enough — LessWrong
1. Incentive landscapes that can’t feasibly be induced by a reward function You’re a deity, tasked with designing a bird brain. You want the bird to get good at singing, as judged by a black-box hardcoded song-assessing algorithm that you already built into the brain last week. The bird chooses actions based in part on within-lifetime reinforcement learning involving dopamine. What reward signal do you use? Well, we want to train the bird to sing the song correctly.
D&D.Sci April 2021: Voyages of the Gray Swan — LessWrong
I haven't looked much into dependence on time and direction yet, apart from noticing that the pirates decline in relative frequency but, I would like some clarification about whether the 100gp budget is for a single set of interventions we'll use on all trips, or is spent each time on a (potentially varying) arrangement.
Core Pathways of Aging — LessWrong
Most overviews of aging suffer from multiple problems: They dump a bunch of findings with no high-level picture. Many of the claims they make are outdated, purely theoretical, and sometimes even outright disproven by existing work. They are usually written by working academics, who are shy about telling us when their peers’ work is completely wrong. They are shy about making strong claims, since this would also implicitly mean denying some claims of the authors’ peers.
Chaos Induces Abstractions — LessWrong
I agree that this is an important concept, or set of related concepts that covers many of the more directly physical abstractions. If something isn't quantum field theory fundamental, and can be measured with physics equipment, there is a good chance it is one of these sorts of abstractions. Of course, a lot of the work in what makes a sensible abstraction is determined by the amount of blurring, and the often implicit context. For instance, take the abstraction "poisonous".
What's So Bad About Ad-Hoc Mathematical Definitions? — LessWrong
We talked about this issue in the comments on this post of yours 9 months ago :) I get the sense that you're considering problems where an open-ended search doesn't tell you if you're heading in the right direction. So for example, if we play Marco Polo, when I shout "Marco" and everybody else shouts "Polo," this gives me some information about where they are. They might move, but they can't necessarily move fast enough to avoid me.
How long does it take to become Gaussian? — LessWrong
To that point, skew and excess Kurtosis are just two of an infinite number of moments, so obviously they do not characterize the distribution.
It’s not economically inefficient for a UBI to reduce recipient’s employment — LessWrong
A UBI (e.g. paying every adult American $8k/year) would reduce recipient’s need for money and so may reduce their incentive to work. This is frequently offered as an argument against a UBI (or as an argument for alternative policies like the EITC that directly incentivize work). This argument is sometimes presented as economically hard-headed realism.
Non-Obstruction: A Simple Concept Motivating Corrigibility — LessWrong
Thanks to Mathias Bonde, Tiffany Cai, Ryan Carey, Michael Cohen, Joe Collman, Andrew Critch, Abram Demski, Michael Dennis, Thomas Gilbert, Matthew Graves, Koen Holtman, Evan Hubinger, Victoria Krakovna, Amanda Ngo, Rohin Shah, Adam Shimi, Logan Smith, and Mark Xu for their thoughts. Main claim: corrigibility’s benefits can be mathematically represented as a counterfactual form of alignment.
The date of AI Takeover is not the day the AI takes over — LessWrong
Instead, it’s the point of no return—the day we AI risk reducers lose the ability to significantly reduce AI risk. This might happen years before classic milestones like “World GWP doubles in four years” and “Superhuman AGI is deployed." The rest of this post explains, justifies, and expands on this obvious but underappreciated idea. (Toby Ord appreciates it; see quote below). I found myself explaining it repeatedly, so I wrote this post as a reference.
Message Length - LessWrong
I'll probably write up a proper review later. But right now I just wanted to say: the other day, someone sent me a big blob of code over IM. I looked at it, my eyes kinda glazed over, I was confused about why they sent it. I moved on with my day. Only later did I realize the code was an SVG file that a) my computer program was supposed to automatically read and interpret, b) I totally could have noticed and interpreted and realized I should open it in photoshop.
The Bayesian Tyrant — LessWrong
Long ago and far away, there was a kingdom called Estimor in a broad green valley surrounded by tall grey mountains. It was an average kingdom in most respects, until the King read the works of Robin Hanson and Eliezer Yudkowsy, and decided to institute a Royalist Futarchy. (This is a parable about the differences between Bayesian updates and logical induction. See also: Radical Probabilism.) The setup was very simple.
Market Misconceptions — LessWrong
Turn the question around: assume your goal is to lose money. If a market was 100% efficient, the price moves would be 100% unpredictable, i.e. random. You're just as likely to make money as to lose it trading such a market. How could you lose money if you tried? Think about it. I can only think of a few ways. The first, obvious, way is to trade a lot. You lose commissions and spread each time you trade. Even if some of your trades happen to make a lot of money, you can lose more by overtrading.
Dremeling - LessWrong
In high school, I was on a robotics team (FRC) who, in honesty, were basically incompetent. (Me included.) We were bad at building a robot, bad at programming it, bad at design, but most relevantly for this post bad at planning ahead. The particular instance which added a term to my internal monologue was this: We needed to cut slots in some aluminum in order to do...something, I don't remember what.
Book Review: Narconomics — LessWrong
I have often heard that the war on drugs is a failure, but the reason why varies. Sometimes, the war on drugs is a failure like the war in Iraq is a failure: because it is very hard to nation-build, to create a society that does not produce or consume drugs. Sometimes, the war on drugs is a failure like the war in Vietnam was a failure: because the opponents, drug cartels, don’t fight like we do.
Against strong bayesianism
Note that this post has been edited in response to feedback and comments. In particular, I've added the word "strong" into the title, and an explanation of it at the beginning of the post, to be clearer what position I'm critiquing. I've also edited the discussion of Blockhead. In this post I want to lay out some intuitions about why bayesianism is not very useful as a conceptual framework for thinking either about AGI or human reasoning.
Choosing the Zero Point — LessWrong
Summary: You can decide what state of affairs counts as neutral, and what counts as positive or negative. Bad things happen if humans do that in our natural way. It's more motivating and less stressful if, when we learn something new, we update the neutral point to [what we think the world really is like now]. A few years back, I read an essay by Rob Bensinger about vegetarianism/veganism, and it convinced me to at least eat much less meat. This post is not about that topic.
Why don't singularitarians bet on the creation of AGI by buying stocks? — LessWrong
With the recent stock market sale, I've been looking over stocks to see which seem to be worth buying. (As background information, I'm buying stocks to have fun and bet my beliefs. If you believe in the efficient market hypothesis, buying stocks at random should perform roughly as well as the market as a whole, and the opportunity cost doesn't seem super high.
Theory and Data as Constraints — LessWrong
There’s a widely-known legend about statistician Abraham Wald’s work on planes during WWII (the veracity of the legend is examined here). As the story goes, the military collected data on planes coming back from missions, marking the location of any bullet holes. They soon had statistics showing how many had been hit on the engine, the fuel system, etc. Based on this data, commanders wanted to add extra armor to reinforce the areas which were most often hit.
Material Goods as an Abundant Resource — LessWrong
If you want to understand the modern economy, as opposed to the economies of yore, one source I strongly recommend is a short story from the July 1958 issue of Astounding Science Fiction, titled “Business As Usual During Alterations”. It’s roughly a 15 minute read. I’m about to throw out major spoilers, so stop reading here if you want to enjoy the story first. One morning, two devices mysteriously appear in front of city hall, along with directions on how to use them.
Gears-Level Models are Capital Investments — LessWrong
Mazes The usual method to solve a maze is some variant of babble-and-prune: try a path, if it seems to get closer to the exit then keep going, if it hits a dead end then go back and try another path. It's a black-box method that works reasonably well on most mazes. However, there are other methods.
Mediums Overpower Messages — LessWrong
Firstly, some topics are just easier if they can be presented the right format. Geometry will be easier in a format that allows diagrams, compared to an audiobook. Secondly, formats, like websites are often Schelling points, not many serious scientists publish their work as a series of gifs on twitter. Most scientists know this, and so don't look for a series of gifs on twitter.
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Donald
Contact Donald, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack