Another artificial intelligence (AI) researcher has quit, making headlines in the process by saying there is some chance, and that any chance is too high, that the technology will bury us. Jacob Coxon, a British engineer quit Anthropic, saying self-improving superintelligence is a genuine fear among his former peers.
“The people building AI earnestly believe that it could kill us all by the end of the decade,“ he wrote on social media.
The headlines followed like night follows day. The fear got plenty of political attention, too, with computationally-challenged politicians waking up in a sweat thinking about dimly-remembered (and apparently not well-understood) sci-fi films.
Giving Coxon the benefit of the doubt (and there is no reason not to, as people having extreme worries is not a sign of being wrong), it is not his fault some people’s brains short circuit from risky technology to fully-automated apocalypse.
Nonetheless, breathless claims about AI’s unleashing leading to our doom have a habit of inflating the claims made by the companies flogging AI in the first place.
So let’s take this seriously and ask ourselves how could today’s AI technology bring about an end to humanity?
One way would be through the use of autonomous weapons. Another might be through the easier designing of terrible weapons of mass destruction. Note, however, that in both cases, and any that does not stretch plausibility to subatomic levels, the issue is humans, not overheating GPUs adding up ones and zeros.
There’s also the boring, slow motion disaster of wasting resources, kicking people out of jobs, concentrating wealth in the hands of the already rich, economic collapse, and the endless degradation of not only products and services but our very ability to discern. The thing is, no-one really cares about any of that, do they?
What AI (by which we mean large language models) cannot do is decide to wipe out humanity. AI cannot decide anything as it lacks a mind. It cannot reason, it has no memory and it has no mental model of the world.
On the matter of mind it is worth trying to be precise. What is called ‘reasoning’ by the tech industry is no such thing. Having basically given up on building a model of the human mind, LLMs were created by an industry that gave up on understanding the world and how we relate to it for performing Brion Gysin’s cut-up technique and hoping the resulting sentences were coherent. And guess what? Do it enough times and, more or less, they are. But that’s just brute force computation, not thinking.
On a banal level that’s fair enough. LLMs are genuinely interesting – and surprisingly useful. Besides, as I mentioned before it’s not just computer scientists who are stuck thinking of the ‘mind’ as a primitive input-output device: G.E.M Anscombe’s paper Modern Moral Philosophy threw down a challenge to philosophers, saying it was useless to pursue moral philosophy in the absence of a working psychological model of the mind.
She did it in 1958, and psychiatry’s reduction to a checklist during the 1980s suggests we haven’t made an awful lot of progress since. Admittedly, this hasty retreat followed not only serious challenges from the likes of Ronnie Laing and Thomas Szasz (despite their challenges being themselves mutually incompatible), but David Rosenhan’s experiments that, while now seriously disputed, still demonstrate that an awful lot of diagnosis is just finger-in-the-air guesswork and double-bind prejudice. All the while, by the way, anti-psychiatry, even in its truly humane form, had little useful to say to people experiencing serious disturbances.
Swarms of bullshit
But what of recent agentic AI outrages, you might ask. Well, look at the details of the Hugging Face breach: what was sold as a mob of malign intelligences turned out to be a simulation of thought copying hacker games and IT security ropier than the Belfast ropeworks.
Is this bad? Yes. Extremely bad. But not because the AI swarm has achieved sentience. It hasn’t even achieved the avoidance of cliché.
It is bad because it demonstrates the absolutely appalling information security that is routine throughout industry, from the sandbox the AIs ‘escaped’ from to, basically, every server in the world. Where was the air gap?
And why is IT security still a joke after all these decades, you might ask? Because it’s difficult, boring, highly skilled, costs money and doesn’t produce ‘value’ that can be noted in a spreadsheet.
But why then are AI bosses such as Sam Altman, Elon Musk and Dario Amoedi now forming a conga line, demanding in unison that states slow down development of the very technology that they are peddling? Why are they asking the US government to work with China to control AIs, rather like how nuclear weapons are subject to treaties? Have they had a vision of the future? Have they seen, in embryonic form, the species that will replace humanity?
No. Stop reading press releases.
I don’t know why the people with their foot on the accelerator are demanding someone tell them to grope around for the brake pedal, but Occam’s razor suggests an answer: AI companies are running out of road. The trillions set alight to dump more GPUs into data centres, and to build more data centres for that matter, need to actually produce some money at some point and there is precious little sign of them ‘aligning’ with profit any time soon.
Worse still, unlike say social media, switching costs are close to non-existent. Consequently, regulating the industry, at least in certain ways, bars new entrants.
Is it possible that there is some AI in some lab somewhere that actually is an existential threat? Yes. Anything is possible, and our model of the world should include the possibility of the implausible. But if we’re going to worry about that the correct approach is to ask what are the human incentives at work.
Make no mistake: AI needs to be regulated, but we should regulate it with our eyes open. Note that the EU’s AI Act, inadequate as it is, is subject to endless grumbling from people who don’t want to be answerable for what they sell and you can see the antinomianism revealed: rules for thee but not for me.
In fact, doing anything with AI, including just thinking about it, while our eyes are open would be a nice change.
Ltd
