In the sequence of variously wild AI developments in the last decade, a thing that was especially surprising to me was the advent of big esteemed companies like Microsoft releasing products like Sydney.
It’s like how you can believe a fictional world has dragons, but it strains credibility if characters start being totally indifferent to social status apropos of nothing.
I can warily accept that boxes of wires can now talk like humans. But that huge official companies now proudly present products that are like crazy confused ladies that do a lot of tasks for you, many accurately, but also try to steal you from your spouse (or these days encourage you to kill yourself or believe in new spiritualities), feels like a scene written by someone with poor familiarity with the character of companies.
But it’s actually written in reality, so what don’t I understand? I guess just that if something is hypey and perceived to be in-future-profitable enough, normal standards of professionalism must fall to that?
I phoned CVS, hoping to move an antibiotic prescription to a different pharmacy address so I could pick it up before leaving on a flight. I was answered by an AI, but the smoothest, most charming and person-realistic AI answering machine I had experienced. It asked me in a friendly woman’s voice if I wanted this and that medication sent over, and since I didn’t really know exactly what my medication was, I agreed to everything it said. This seemed great. I hung up surprised that phoning CVS had actually led to the thing I wanted in short order, and reluctantly impressed with their phone AI.
I tried to pick up the medication. It seemingly hadn’t been sent to the new pharmacy at all, and they didn’t seem aware of anything like that being planned. Huh. I still managed to get them to move it from the pharmacy, and catch the flight. I got a message from my doctor’s office saying that they had received a request from CVS for two other antibiotics, but they didn’t think that would be the best choice at this point. From what I can gather then, this AI assistant just made up a couple of different plausible antibiotics that I hadn’t been prescribed and tried to order them for me from my doctor.
Ways this could have gone wrong: a) I don’t get antibiotics in time and get a more serious infection, especially for instance if I’m old or don’t have heaps of time to follow up at pharmacies, b) my doctor’s office just trusts the ‘pharmacist’ request for meds and I get sent meds that were literally hallucinated by an AI, and take them.
So beyond dangerous, this all seems wildly unprofessional. If a person behaved in this way, there’s no way they could work as a phone operator for a respectable brand like CVS. Even if they were part of a huge union of people willing to work for free.
Why is this suddenly acceptable because it’s AI?
It's because Wall Street investors are demanding that every company integrate AI into its operations ASAP. If they do, stocks go up, regardless of the quality of said AI. If they don't, stocks go down, even if they have good reasons to be wary of AI.
Also, the executives pushing AI are often out-of-touch with day-to-day operations and don't understand the limitations of AI in those contexts. AIs are pretty good at summarizing your emails and creating briefings on publicly available information, which are two things executives need a lot of, so they overestimate how good AI is at other tasks.
It seems like we might be on the verge of this shifting as investors get nervous about the costs of AI, but who knows?