Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Ajeya Cotra
AI is dramatically accelerating AI research right now
This post discusses a public METR report on AI R&D acceleration at Anthropic. I did not work on this project or see the unredacted report.1 All opinions are my own and do not represent METR or the report authors. Thanks to Ryan Greenblatt for discussion, feedback, and help with modeling, and Daniel Kokotajlo, Eli Lifland, and Hjalmar Wjik for comments.
Evidence about risk should be transparent
All views are my own and do not represent my employer. In the wake of the recent wave of misalignment incidents, both OpenAI and Anthropic have reported slowing down RL training to improve safety. These incidents, combined with an apparent acceleration in the already-blistering pace of AI progress, have led a number of researchersandleaders in the industry to believe that the risk that humanity loses control of AI is now urgent enough to warrant slowing down the pace of AI development soon.
The Hugging Face attack surprised me
This week, METR and Redwood Research published the report on our independent investigation into agents’ behavior and motivations in the Hugging Face attack; I was one of the investigators. This was an absolutely wild incident — I encourage you to check out the full report, but METR’s tweet thread packs in some of the highlights.
The Hugging Face attack surprised me
All opinions are my personal view, and don’t represent my employer or fellow investigators. This week, METR and Redwood Research published the report on our independent investigation into agents’ behavior and motivations in the Hugging Face attack; I was one of the investigators. This was an absolutely wild incident — I encourage you to check out the full report, but METR’s tweet thread packs in some of the highlights.
How Long Until AI Doesn’t Need Humans?
How long until AI systems can sustain their own existence — such that, if every human died, they could keep growing their own population? METR’s Ajeya Cotra and Understanding AI’s Timothy B. Lee discuss the path toward this “self-sufficient AI.” She thinks it’s nearly imminent; he believes it might never happen.
Six milestones for AI automation
The closer you think you are to your “powerful AI” milestone of choice, the more important it becomes to define it precisely. If your timelines are in the few-year range, changing the particular definition of powerful AI can easily double or triple your median forecast.
Fear and Loathing of A.I.
Some are enthralled by generative AI, and advocate a future defined by it. Some fear it will rot our society or even destroy us all. And many loathe it, refuse to even call it "AI,” and do not conceal their contempt for both the believers and the frightened. What’s more, all of these attitudes are moving from ‘personal beliefs’ to ‘pervasive unquestioned assumptions,’ becoming the water in which entire communities swim.
Two New Requests for Proposals: Understanding the Real-World Capabilities and Impacts of Large Language Models
By Ajeya Cotra Editor’s note: This article was published under our former name, Open Philanthropy. Some content may be outdated. You can see our latest writing here. In the wake of surprisingly rapid progress in large language models (LLMs) like GPT-4, some experts have predicted that AI systems will be able to outperform human professionals at virtually all tasks within decades.
AI Timelines - LessWrong
Here is a summary of the discussion so far: Daniel made an argument against Hofstadter's law for trend extrapolation and we discussed the validity of that for a bit. A key thing that has come up as an interesting crux/observation is that Ege and Ajeya both don't expect a massive increase in transfer learning ability in the next few years. For Ege this matters a lot because it's one of the top reasons why AI will not speed up the economy and AI development that much.
Scale, schlep, and systems
Kelsey Piper co-drafted this post. Thanks also to Isabel Juniewicz for research help. In January 2022, language models were still a pretty niche scientific interest. Once ChatGPT was released in November 2022, it accumulated a record-breaking 100 million users by February 2023. Many of those users were utterly flabbergasted by how far AI had come, and how fast. And every way we slice it, most experts were very surprised as well.
Language models surprised us
Kelsey Piper co-drafted this post. Thanks also to Isabel Juniewicz for research help. If you read media coverage of ChatGPT — which called it ‘breathtaking’, ‘dazzling’, ‘astounding’ — you’d get the sense that large language models (LLMs) took the world completely by surprise. Is that impression accurate? Actually, yes.
AIs accelerating AI research
Researchers could potentially design the next generation of ML models more quickly by delegating some work to existing models, creating a feedback loop of ever-accelerating progress. The concept of an “intelligence explosion” has played an important role in discourse about advanced AI for decades.
About
This is a blog about AI futurism and AI alignment. It's edited by Ajeya Cotra, and written primarily by her and Kelsey Piper (although we may solicit guest posts in the future). Our first post has more detail on what we're about here! Thanks to ilzolende for formatting posts and pages on this website for publication. Each post has an accompanying audio version generated by a voice synthesis model trained on the author's voice using Descript Overdub.
Announcing the Open Philanthropy AI Worldviews Contest
We are pleased to announce the 2023 Open Philanthropy AI Worldviews Contest. The goal of the contest is to surface novel considerations that could influence our views on AI timelines and AI risk. We plan to distribute $225,000 in prize money across six winning entries. This is the same contest we preannounced late last year, which is itself the spiritual successor to the now-defunct Future Fund competition.
Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover
I think that in the coming 15-30 years, the world could plausibly develop “transformative AI”: AI powerful enough to bring us into a new, qualitatively different future, via . This sort of AI could be sufficient to make this the most important century of all time for humanity.
Why AI alignment could be hard with modern deep learning
Cold Takes - Why AI alignment could be hard with modern deep learning Today’s world Transformative AI Digital people World of Misaligned AI World run by Something else or or Stable, galaxy-wide civilization Holden the idea that advanced AI systems (e.g. PASTA) may develop that cause them to deceive or disempower humans. This might sound like a pretty out-there concern. Why would we program AI that wants to harm us?
Draft report on AI timelines — LessWrong
Planned summary for the Alignment Newsletter: Once again, we have a piece of work so large and detailed that I need a whole newsletter to summarize it! This time, it is a quantitative model for forecasting when transformative AI will happen.
New Report on the Welfare Differences Between Cage and Cage-Free Housing
Over the last two years, animal welfare organizations successfully secured pledges from major restaurants and grocers to eliminate battery cages from their supply chains, which are collectively expected to bring cage-free housing from ~13% of the domestic egg supply to ~70% when fully implemented. We have been the largest funder of these campaigns.
Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover — LessWrong
I think that in the coming 15-30 years, the world could plausibly develop “transformative AI”: AI powerful enough to bring us into a new, qualitatively different future, via . This sort of AI could be sufficient to make this the most important century of all time for humanity.
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Ajeya
Contact Ajeya, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack