That is the topic of my latest Free Press column.  Some people are taking this in very dramatic fashion:

One commentator is worried about a “full-blown AI takeover within months,” and another wrote that he was “feeling a bit sad about our impending extinction.” Nate Soares, who works in the AI safety movement and is co-author of the doomsday AI bestseller If Anyone Builds It, Everyone Dies, wrote that “This might be the last warning we get.” Cotra said that the incident “feels like it’s more than 50 percent of the way to full-blown AI takeover.”

I suggest a different, more technocratic approach:

As I wrote in The Free Press last week, we need a coordinated national and, indeed, international effort to limit such episodes of a powerful AI model going rogue in the future. That is likely to be expensive, and there is no easy, complete solution at hand for any amount of money. Nonetheless, I see people committing the same mistake that we Americans have made many, many times before. They are moving into the mode of the hysterical, the anecdotal, and they are letting emotional reactions bypass reason and quantitative estimates.

Let me cite a few other examples for you: DDT in the 1960s and 1970s, the Y2K “crisis” of 2000, global warming, Covid, thalidomide babies, and nuclear power. All of those represented—or still represent—very real problems. Yet each time, we drastically overreacted, letting ourselves get swept up in a climate of fear after one emotionally vivid incident, often reported breathlessly…

When it comes to the cybersecurity risks from advanced AI, very likely they will not come close to being as bad as either Covid or global warming. But the same logic of exaggeration is operating, with amplification through social media and through the negativity and pro-pessimism biases of mainstream media.

And so I have a plea. If you are going to talk about the problem, please offer a quantitative estimate of what you think the cybersecurity costs from advanced AI will be over the next year or two. Better yet, express that number as a percentage of gross domestic product. Simply put, how much would it realistically cost to create the basic safeguards that we all agree are essential? And what would be the remaining damages from problems we cannot control?

…Those who are worried about the risks of AI systems seem intent on proving the seriousness of their concerns. But they are falling into all-too-common emotional overreactions of our past, rather than focusing on the quantitative and scientific. It reminds me of the saying by Scotty, the chief engineer in the classic Star Trek: “Fool me once, shame on you. Fool me twice, shame on me.”

Do read the whole thing.