The other side of the tidal wave
I guess there's maybe a 10-20% chance of AI causing human extinction in the coming decades, but I feel more distressed about it than even that suggests—I think because in the case where it doesn't cause human extinction, I find it hard to imagine life not going kind of off the rails. So many things I like about the world seem likely to be over or badly disrupted with superhuman AI (writing, explaining things to people, friendships where you can be of any use to one another, taking pride in skills, thinking, learning, figuring out how to achieve things, making things, easy tracking of what is and isn't conscious), and I don't trust that the replacements will be actually good, or good for us, or that anything will be reversible.
Even if we don't die, it still feels like everything is coming to an end.
Sometimes I think about breathing. Imagine a world in which humans evolved in a way that breathing wasn't necessary, like we absorbed oxygen through some sort of continuous, ambient process. Then, one day, that process is replaced by breathing. What? You're telling me I have to suck in air every few seconds? What if I'm talking? What if I'm eating? How will we hang out underwater? What? We can't? This sucks!
It just seems obviously bad, even horrifically bad, like one of the most ruinous things that could ever befall humanity.
But breathing doesn't actually seem bad. It seems good, it seems great. It seems like one of the absolute best parts of being alive.
I'm not sure if this thought experiment is totally relevant to what you're noticing, but it seems related. The things it makes me think about are:
- we really can get used to anything, not just used to it, but lovingly attached to it
- a lot of things, maybe most things, aren't intrinsically good or bad, but acquire value in juxtaposition with other things, in constellations which have an overall structure, and it is these bigger structures that really matter, and they are less arbitrary and more stable
- maybe some of the things on your list (writing, pride?) are like breathing, and some of them (learning, explaining?) are more like these larger structures
I agree with your prediction that things are going to get weird. But then, breathing is weird.
I share these same concerns... but I also hold out some hope.
Chess has already been "disrupted" by superhuman AI, but people still enjoy playing chess (and honing their skills, despite no hope of reaching AI performance).
Music has long since been "disrupted" by professional recordings, but amateurs still enjoy playing music.
While this one might be a stretch, one might argue that friendships have already been "disrupted" by the onslaught of modern entertainment options (shows, podcasts, video games, livestreamers, etc.), not to mention the parasocial relationships that people can form around them... and while this has certainly impacted the time we spend with our friends, it has by no means eliminated it.
So, while I'm far from confident, I at least hold out hope that we will find ways to maintain sane lives in the era of transformative AI.