When I talk to people about what might be done about AI threatening approximately everything that everyone cares about, I notice a common oddity in their resistance to a variety of ideas. They seem to take for granted that certain entities—especially Trump and China—would be acting against their own interests, were they to cooperate or take proactive action to avert the building of dangerous AI.
The speaker often thinks there is a fairly substantial risk of the AI thus produced killing or disempowering everyone, including Trump and China. And I imagine in a situation where a certain course of action were going to produce a 20% chance of Trump being shot in the head or China being heavily nuked, that these parties would actually be considered to be ‘following incentives’ to avoid it. Yet they talk as though the idea of Trump or China responding to such risks is akin to the idea of these parties suddenly becoming zealous proponents of universal selfless love randomly.
It’s like while believing in the risk, they also kind of believe that it’s a totally uncompelling story that nobody in real geopolitics would ever be touched by.
Or that these parties actual utility functions are ‘be the bad guy’, which, you know, sometimes involves extreme selflessness as long as it puts everyone else at risk too.
Any other hypotheses for what’s going on?
I think people aren't taking into account how much more legit AI safety will look in these scenarios [conditional on it in fact being an important thing to worry about].
I doubt this is everyone's perspective, but kinda rhymes with 2010s-era Yudkowsky talking about societal incompetence meaning that we shouldn't expect a response to AI risk. But it turns out we have a more gradual curve than he expected, and society is able to notice risks once it starts seeing empirical evidence of them. Presumably we'll continue to see the evidence pile up.
A related hypothesis: AI risk field selects for people inclined to:
- focus on the possibility of important things going wrong
- take "heroic responsibility" over those problems, and implicitly assume low responsibility on others' part
so this leads to worrying "oh man, what if these guys are super dumb and bad, that would suck, what can we do about it".
I think you would be right about many politicians, but there's an especially unfortunate feature of the current administration, which is that Trump's anti-intellectual-elite brand has resulted in a positive preference to do what elites *oppose*, regardless of the actual merits. (E.g. canceling PEPFAR.) It's not only a larger hill to climb, but it's almost an anti-inductive environment.