It’s time to talk utility, not possibility, in AI
Somewhere between the breathless claims that artificial intelligence (AI) will cure cancer and the fears that it is a bubble held up by accounting tricks sits a boring truth: the technology works. It works just badly enough to be dangerous.
For criminals, which is to say businesses acting outside the law, AI doesn’t have to be good. If you are using AI to write better phishing e-mails, for instance, a 50% success rate, even if you define ‘success’ as merely a coherent-looking e-mail that isn’t a typo-strewn wasteland, is a great leap forward. For a business in the regular economy, though, seeking to adopt AI demands a task success rate rather closer to 100%.
Of course, some businesses get around this by making the business function an absolute swamp and then sinking an AI blob into the middle of it. Customer contact, for instance, was being let rot away to nothing long before large language models (LLMs) appeared on the scene. This is true of various forms of automation, from self-service checkouts to anything companies can’t be bothered doing but are forced to, either by regulators or just lingering market expectations, damn them!
Palantir boss Alex Karp (pictured) recently appeared on CNBC, resulting in some online mirth at his excitable takedown of how businesses risk destroying themselves with their AI practices, such as so-called ‘tokenmaxxing’, the trick of asking the AI to do as many things as possible simply to juice usage metrics, as well as giving away corporate information to the entities that run LLMs. Describing this, Karp said: “Something has gone completely wrong.”
Despite his point being a tendentious sales pitch and his exuberance borderline eccentric, the core of what he was saying was more than defensible: if businesses use AI, they should use it in a way that does not damage their actual core activities.
In Karp’s view AI in its current form is made up of compute, an application layer and a model – only the first two make money.
I don’t want to think
Palantir is, I suppose, an AI company. Certainly it is a data business that makes use of AI, among other technologies. Operationally, though, it seems to be more like a professional services firm, going into businesses to run specific projects and units. This, of course, tells us everything we need to know about Karp’s complaints about the AI sector in general: his business model is different, and one that relies on money being exchanged at least for services, rather than hope, prayers and British Leyland-style government bailouts.
“The basic view among enterprises in this country is ‘I’m going to chillax and waste my time with tokens, I’m going to get no value, and they’re [AI companies are] going to get my IP’,” he said.
Karp also talked about national security, battlefields (literal and metaphorical) and other matters, all of which are genuinely of enormous importance but need to be bracketed here as getting into the ethics – and morality – of AI would expand this simple column from a mere musing to a full-blown nervous breakdown in print, or at least my going to bits in bits. So, in light of that, let’s stick to a narrower claim: when it comes to the, at once vague and somehow also totalising, hype around AI, Karp is not wrong: something has gone wrong.
Indeed, some of the criticism of his TV appearance this week is worth pausing on. The people laughing at Karp, whether for being Palantir’s boss or for his odd manner, are themselves judging output for fluency and affect rather than interrogating its content. This, as you have doubtless now realised, is precisely what this column is about.
It’s only human, I suppose: events and individuals are not free-floating phantoms, they arrive into existing lives, histories and situations, and come with judgements attached. That’s worth thinking about precisely because as AI use grows, so too does the human tendency to use it in the laziest ways imaginable: not merely wasting tokens, but blindly trusting whatever comes back.
Trial by algorithm
According to a report published this week by The Register, a video conferencing company is suing a threat-intelligence firm that published a blog linking it to a Chinese corporate espionage operation. The video conferencing company’s chief executive, Michael Robertson, said that he suspects the connection to malware was dreamed up, or ‘hallucinated’, by an AI.
“They admit to using AI for their analysis,” he told The Register. “Maybe a human made it all up? Maybe it was AI?”
He also stated, very plainly, why people should care: “We’re on the doorstep of an era where AI will be used to make critical life-altering decisions on people’s lives: Did you pay your taxes, what your credit rating should be, will you get admitted to the university, do you qualify for the home loan, should you be on the no-fly list, etc.”
The precedent is not hypothetical. In the Netherlands, tax authorities used an algorithm to flag suspected childcare benefit fraud. The system disproportionately flagged families with dual nationality; officials, trusting the machine and disinclined to check it, clawed back benefits from tens of thousands of families who had done nothing wrong. People were pushed into financial ruin and more than a thousand children were taken into care. In 2021 the government resigned over it.
It’s worth noting that no large language models were involved. The toeslagenaffaire was accomplished with an existing generation of automated decision-making software, plus the one ingredient that the British Post Office scandal has reminded us never goes out of stock: institutional eagerness to believe the machine, because believing it is cheaper than checking it.
The danger is that by blindly accepting instrumental reason, now producible ‘at scale’, we falsely seek to fully rationalise individuals, processing citizens as data-points, and thus produce systematic irrationality and cruelty precisely through efficiency.
With LLMs and other forms of AI – and let’s not forget, the lazy, bored, unhappy, overworked people in charge of them – the threat is worse than ever. Indeed, it seems to me that it is this eagerness that is the constant. What AIs change is the fluency of the error.
Subscribers 0
Fans 0
Followers 0
Followers