I’ve argued that the AI situation is not clearly an ‘arms race’. By which I mean, going fast is not clearly good, even selfishly.
I think this is a hard point to get across. Like, these people are RACING. They say they are RACING. They are GOING FAST. If they stop RACING the other side will get there first. How is it not a RACE??
Which is a fair response.
It’s like if I said “this isn’t a chess tournament” gesturing at a group of chess champions aggressively playing chess. How could it not be?
Well, maybe all the prizes and recognition available in the circumstances are based on winning at checkers. That would make it, in a very important sense, not a chess tournament. They can play chess all they like, but it doesn’t make the incentive structure into that of a chess tournament. If they want to win at a tournament, their strategy is just badly mistaken.
It’s true that many people are trying to build AI very fast. But many people building AI very fast is different from being in a game where going very fast is the best selfish strategic move.
And this becomes important when “it’s really important to win at the race” becomes justification for a) moving fast at very high costs to other people, and b) giving up instead of trying to coordinate other players not to move fast, since other players are presumed to be immovably committed to winning the race due to that being so incentivized.
These justifications both require the structure of incentives to actually be a race, not just for people to be racing.
‘Is AI really an arms race or are people just racing?’ might sound like an abstract question. But if someone is saying they need to risk your family’s lives to fuel their quest to win an extremely high stakes chess championship, it’s very concretely important whether they are really in a chess championship!
While this is a basic point, my guess is that the distinction between what people are doing and what it is in their interests to do is too subtle and non-memorable to be tracked in the conversation.
So I propose an image I think might keep the incentives and the behavior separate more intuitively: AI as a Trojan horse race.
Various groups are working really hard to get various wooden horses through their own gates, resolute on doing so before their enemies pull in such a prize and outclass them with the contents. It’s an open question whether each horse contains fantastic treasure or a bunch of enemy agents. (This time in history we are even pretty confident that it includes a bunch of agents of some sort, and not at all confident of their loyalty..)
Is it enough to know that other cities are pulling horses through their gates? Are you satisfied then to have the biggest one pulled into your own town square?
Good post! I think too that people conflate between race as ‘arms race’ as opposed to ‘a competition you win by being first to cross a finish line.’
I think the first is more true than the latter - unless RSI/takeoff are much more powerful than expected by many of the people making arguments for racing.
And then maybe it’s correct to call it a race, but more important to note how terrifying the whole situation is
You can lead an AI trojan horse to the data center, but you can't make it stop to think.