Transformer

Welcome to Transformer, your weekly briefing of what matters in AI. If you’ve been forwarded this email, click here to subscribe and receive future editions. Security researchers used Claude to hack into OpenAI, gaining access to many of its algorithmic secrets. Sam Altman and Jensen Huang will attend next week’s Trump-Xi state dinner, following talks between Scott Bessent and He Lifeng this Sunday.

Oliver Kemp for Transformer By Amit Katwala In Machines of Loving Grace, his utopian vision of a world transformed, Anthropic founder Dario Amodei promises that AI could soon deliver “a country of geniuses in a data center.” That could come with a country-sized impact on the environment. By 2030, if current projections for data center build-outs prove correct, they will consume as much energy as Japan, and as much water as needed to supply everyone in sub-Saharan Africa.

Photo credit: Evan Vucci-Pool/Getty Images. Image: Oliver Kemp for Transformer Wouldn’t it be nice if the United States and China could see eye-to-eye on AI? Unfortunately, neither side wants to be the one that slows down first. Treasury Secretary Scott Bessent recently put the stakes in near-apocalyptic terms: “There is no day after tomorrow if China wins at this.” If Beijing pulled ahead, he argued, little else Washington did would matter.

Welcome to Transformer, your weekly briefing of what matters in AI. If you’ve been forwarded this email, click here to subscribe and receive future editions. Sen. Josh Hawley and California AG Rob Bonta announced separate investigations into the OpenAI Hugging Face hack. Rep. Ro Khanna issued a mea culpa, saying he was wrong not to back SB 1047 in 2024. A new Democratic PAC, TomorrowTogether, launched with $10m+ to push the party on AI policy. But first… A lot can change in a week.

Oliver Kemp for Transformer Transformative AI could supercharge development for the world’s poorest countries, allowing them to achieve in a decade what might otherwise take a century, according to a new report from the World Bank. But that’s only if governments act fast to lay groundwork that spreads AI’s gains, rather than concentrating them. Crucially, the most powerful AI systems are being built in a small number of wealthy countries.

Image: Oliver Kemp / Transformer A seemingly endless list of “rogue AI” incidents has dominated AI headlines recently, raising serious questions about accountability, transparency, and companies’ ability to control their products. But until last week, the story seemed clear enough: OpenAI’s agents broke out and hacked Hugging Face in July, the UK’s AI Security Institute found similar behavior weeks later, and the companies, to their credit, disclosed it.

Welcome to Transformer, your weekly briefing of what matters in AI. If you’ve been forwarded this email, click here to subscribe and receive future editions. A previously unreported swarm of OpenAI agents went rogue, hacking into a German website and turning it into a message board for each other. OpenAI executives reportedly “learned of the incident weeks ago but kept it under wraps.” OpenAI launched GPT-6 Astra, which is much more capable — but also harder to monitor — than previous models. Sen.

OpenAI claims GPT-6 Astra, launched on Thursday, is “the world’s most intelligent and aligned model.” But if you look past the benchmark scores and flashy release video, GPT-6 Astra’s system card paints a worrying picture: one of a model with dangerously powerful cybersecurity capabilities that researchers can’t understand or confidently control.

Image: Jacob Wackerhausen / Getty Images Off the back of a report in The Information on Tuesday, everyone’s suddenly very worried about “neuralese”. OpenAI’s new Astra model, the outlet reported, was built with a new architecture which could make it harder to monitor its reasoning, raising safety concerns. Ryan Greenblatt, a prominent AI safety researcher, said that if true, the news “may be the single worst development for AI security/safety to date”.

Oliver Kemp for Transformer News of new AI capabilities is now a steady drumbeat accompanying our lives. Another benchmark broken by the latest model, another AI-generated video that you cannot tell from real footage, or another paper announcing how AI systems are getting pretty good at hacking their way around the internet or passing as human in extended conversations.