Notes from the other existential risk
Five lessons the climate movement learned the hard way, and how they might apply to AI safety
I’ve spent the bulk of my working life getting people, businesses and entrepreneurs to care about climate change. This summer I got round to some long-delayed reading about the trajectory of AI and the ethics and safety conversations around it, and something started nagging me. To my utter surprise, given I thought I was learning about a completely new field, much of what I read seemed… quite familiar. Not so much the content as the form and framing of many of the debates.
Given that some of the potential risks associated with advanced AI may move a lot faster than the long-term impacts of climate change, people working on AI safety might have less time to learn things the hard way, and less excuse for repeating mistakes someone else has already made. So here’s my humble attempt at sharing what the climate community has learned over the past few decades while working on one of the other global risks that keep a lot of people awake at night.
#1: We built a cliff that wasn’t there
The climate movement spent over a decade analysing and debating the point beyond which we'd be entering dangerous territory. First the headline goal was to limit warming to well below 2°C, then we added the more ambitious 1.5°C, and these days the question is whether 1.5°C is achievable at all. To be fair, the numbers did real work as they focused minds and gave advocates something to hold governments accountable to. But there was a flip side to all this. The 1.5°C target turned into a cliff edge in a lot of people’s heads. Trouble is, the science never said that, pointing instead to lots of separate thresholds at different temperatures, some unfolding over a decade and some over thousands of years. So when 2024 turned out to be the first full year to exceed 1.5°C, sooner than anyone expected, the movement found itself on the back foot, explaining that we hadn't actually lost and hastily rebuilding the narrative around something less black and white, like "every fraction of a degree matters".
A similar dynamic seems to be taking shape in conversations about the potential arrival of artificial general intelligence. Questions about when it's predicted to land, which definition is the right one, and which benchmarks capture it accurately seem to take up a lot of airtime. As with climate change, my hunch is that there may be no clear way to pinpoint exactly at which point the technology flips overnight from safe to dangerous. Which is why I was pleasantly surprised to find that the authors of two of the most prominent AI scenarios, who disagree about their long-term predictions, sat down together and found there's quite a lot they agree on.
That kind of focus, on what's known and what isn't, and where the agreement lies, might matter more than predicting the distance to the figurative cliff and arguing over whether one exists. Perhaps the rule of thumb should be to work out which preparations hold up whichever risk scenario turns out to be right? Climate definitely learned that lesson late.
#2: We treated adaptation as surrender
For years adaptation was close to a dirty word in climate circles. Something you simply didn’t bring up, since it would be equated with defeatism. Al Gore, considered by many the champion of the movement, dismissed it outright in 1992:
“Believing that we can adapt to just about anything is ultimately a kind of laziness, an arrogant faith in our ability to react in time to save our skins.” Source
To be fair, Gore came round by 2008. But by then climate adaptation had spent decades as the poor cousin of mitigation, arriving late in many of the places that needed it most, still underfunded, and treated as separate from the real work.
I’m wondering if there’s a similar flinch in discussions about AI safety, where the much-needed work on things like technical alignment and governance might be overshadowing other responses and creating an implicit hierarchy.
Researchers at GovAI put it more precisely than I ever could:
“Existing strategies for managing risks from advanced AI systems often focus on affecting what AI systems are developed and how they diffuse. However, this approach becomes less feasible as the number of developers of advanced AI grows, and impedes beneficial use-cases as well as harmful ones. In response, we urge a complementary approach: increasing societal adaptation to advanced AI, that is, reducing the expected negative impact.” Source
The crux of their argument is that prevention only really works while the number of builders is small, and since the opposite seems to be the case, adaptation has to be part of the answer.
Climate's false either/or, the claim that reaching for things which absorb shocks competes with acting to prevent them, was clearly a mistake. The AI safety community would be wise not to fall for the same urgency-induced instinct to pitch strategies against one another.
#3: We were (mostly) wrong about China
For a long time, many in the climate space saw China as a delayer or an outright wrecker. And there were good reasons to do so. For years Beijing refused binding targets and contributed to the collapse of a pivotal climate summit in Copenhagen. A decade and a half on, China is filling the gap left by the US withdrawal from global cooperation on climate, makes over 80% of the world's solar panels, and installs more renewable capacity than the rest of the world combined. That clean energy build-out is a large part of the reason some of the worst-case warming scenarios have been dropped altogether.
A similar argument about China's role seems very much alive in discussions about the governance and development of advanced AI. To a climate ear, a lot of the talk about not letting China take the lead strikes a familiar note. To put it more cynically, the AI arms race framing might actually serve as a convenient line for US frontier labs that would rather not face stringent guardrails at home. Meanwhile there's some early evidence that China, for all its strategic self-interest, is engaging constructively by taking part in multilateral fora, showing concern for safety in its revised governance framework, and backing open-weight models that challenge the proprietary nature of the US ones.
None of the above is to say China is a partner one can trust without verification. In climate action, despite significant progress, it remains the largest emitter by far, and its plans to reach net zero are rated as highly insufficient. But if the past decades are anything to go by, a country which isn’t a natural ally can still act constructively, and might even turn out to be where mechanisms of change are built. Writing key actors off early, or casting them as permanent adversaries, could prove the costlier choice.
#4: We mistook facts and fear for persuasion and advocacy
Climate advocates spent years making better graphs, promoting technical evidence bases like the IPCC Assessment Reports, and even writing whole scripts for debunking denialist talking points, all on the hope that if people just 'got it' and had the facts straight, they'd see the light and put pressure on governments to do something.
So for a long time the conversation stayed centred on messages like 97% of scientists agree, progress charts showing which countries are missing their targets, and technical terms like net zero, anthropogenic sinks and decarbonisation. None of which made it easy to connect climate change to anything in people's daily lives, like why this summer cut so many days off my daughter's school year in the Netherlands, and what to do about it. Plus, we leaned too heavily on fear, without stopping to think much about what a constant flow of doomsday messaging actually does to people. The answer wasn't pretty. Instead of engagement, paradoxically, it produced a lot of denial and helplessness.
I'm wondering whether the conversation about AI's adverse impacts is repeating some of the same pattern. The safety advocates seems to lead its calls to action with existential risks like AI-enabled bioweapons, coups, or cyberattacks on critical infrastructure. Not unreasonably, for sure, since these are genuinely worrying and could plausibly lead to catastrophe. But in practice it means the bulk of the conversation is shaped around things that worry technical experts, rather than what’s on the public’s mind. When you actually ask people, as Anthropic did, rogue misaligned AI came last, at 27% of those surveyed, and job loss topped the list at 64%. And, to add a personal data point, at the recent UN Global Dialogue on AI Governance the call to action that got the loudest applause wasn't about bioweapons but about AI's effects on children.
Climate spent too many years treating the everyday as the lesser conversation, on the grounds that the global stakes were too big for small talk. Turns out those more personal, value-led arguments were the way in all along, and the thing that got people to care.
#5: We overestimated what a crisis can do
“If only there were a crisis big enough, we’d finally act” is a common view in climate circles (though rarely said out loud). Thing is, it doesn’t work in practice. Just think about Hurricane Sandy (2012), Pakistan’s floods (2022), or this year’s European heatwaves, none of which produced, or look likely to produce, change in any way proportional to the underlying problem.
Roman Krznaric’s explanation is one of the clearest and convincing I’ve found on the topic. He argues that rapid, transformative change needs not just one thing but several at once, namely:
a crisis that destabilises the system
disruptive movements that challenge the people in charge
ideas ready to replace what’s being torn down.
One of climate's gaps was that we had crises in the form of severe weather events and ideas in abundance, but what we didn't have for a long time was much of a movement powerful enough to galvanise political will. Things started to shift, to a degree, when it grew beyond climate science and the experts, and expanded into school strikes, divestment campaigns, and litigation brought by elderly Swiss women, among many others. All of whom made climate their issue.
Seen in that light, the recent response to the OpenAI x Hugging Face incident, which many read as a warning shot about the potency of autonomous AI systems, is interesting. Most of the coverage I followed was about disbelief, shock, or whether this crisis was big enough to produce robust regulation. Except that, going by Krznaric, a crisis-like moment was never going to be the deciding factor on its own. What tends to happen is what happened after every climate event: the initial shock, then the counter-framing (for instance, that the Hugging Face hack was a test with the guardrails deliberately switched off), and then the news cycle moves on. Without anyone sustaining the pressure, the window closes by itself.
Of the other two ingredients, ideas aren’t the problem in AI governance. There’s an early and growing body of policy proposals, and a fast-expanding institutional apparatus to go with them. What the field doesn't have just yet is a broader movement. That, to me, looks like the gap worth filling, not only because it might be the missing element in whichever change framework you pick, but more simply because, unlike any crisis, it's the part advocates can actually work on.
Where the parallels break
I’m writing all of this knowing that not every lesson transfers cleanly between two communities, and two sets of challenges, that differ in important ways. For starters, climate change has billions of emitters, while AI has a handful of frontier labs and a dozen or two states that matter, at least for now, which might make it the more tractable problem. Nobody ever claimed fossil fuels would cure cancer either, whereas AI’s benefits are large and actively pursued by the people building it, which makes the argument for restraint far harder than any we in the climate space ever had to make. And climate spent years scrutinising the evidence base and building a scientific consensus that reassured policymakers, something AI safety advocates are still working on and might not have the time for, given the pace of technological change.
Still, I offer these in the spirit of bridging divides and building bigger changemaking tents, since both communities are ultimately trying to get people and institutions to take global risks seriously before they get out of hand.
Curious what people in the field make of them, and what I've missed. Comments more than welcome!
I really liked this article! Of course I am not objective, reminded me a lot of what I am trying to say: if you want actual impact, you need to understand how politics and public opinion actually works. Your points 4 and 5 especially. Wonder if you have 5 suggestions as well? :D