Fridges cooking food. Britain telling citizens to stock up on canned goods and water. A computing pioneer mulling a Chernobyl-grade “massive brownout” and financial destruction. How are you? How’s the new phone?
This week, Britain’s National Cyber Security Centre urged businesses to stop doing tech “on the cheap”. Gratifying stuff for those of us who think information technology seems more than a little wobbly. And a sign of fun times to come in the world of information security, but when is that not true?
To the extent that any criticism, particularly one operating largely at the level of culture, is received at all it seems to me the most likely response to saying tech isn’t all it’s cracked up to be is an eye-roll: Yeah, whatever. But what if we take the critique seriously?
Think about the history of personal computing. Any era. Here’s a single vignette: when IBM and Microsoft fell out of love during the development of their proposed next-generation operating system OS/2, intended to replace DOS in the way Windows eventually did, Microsoft’s chief complaint was that its spouse was insufficiently responsive. IBM was methodical. Which could also be characterised as slow, plodding and dull.
Maybe it was. Over decades, approaching development slowly and according to a plan will create a conservative culture. Despite the PC, IBM’s real world was one where computers were handled only by operators in white gloves and bearded priests mumbling arcane incantations.
And Microsoft was probably engaged in special pleading: it had realised Windows was a more profitable horse to ride and was already planning to wave IBM goodbye. Nor was Microsoft simply wrong. IBM measured output by lines of code, which is the dictionary definition of how to create a perverse incentive.
OS/2 was delivered late, aimed at older processors that should have been ignored and, by the time any of this mattered, IBM lost the ground to its competitor and erstwhile partner.
It was also known to be more stable, more reliable, and easier to use than Windows. There are countless other examples, and it’s worth understanding that personal computers themselves were invented by creative amateur hackers, bodged together from calculator parts, not truly descended from IBM or Sperry mainframes.
The culture of speed prevailed, and never truly developed the habit of proving its tools trustworthy. Today, we call programming software engineering, but it lacks much of the control associated with engineering disciplines.
No going back
This is not to say that the old world was a shining city on the hill. Personal computing was a genuine democratic revolution, and that should not be forgotten. There never were any good old days, and the excitement at this novel technology was real and justified.
Besides, at least one of the actors in the world of bigger, more critical computers (which has by now fully merged with our jumped-up calculators) was surprisingly plain spoken about the dangers to come. When Ken Thompson of Bell Labs gave his famed speech, Reflections on Trusting Trust, he pointed out it was possible to do something genuinely staggering: inject invisible, self-replicating backdoors into software without leaving a trace in the source code. This is not a technical problem, or not only one. It’s an epistemological and teleological landmine.
In other words: who even knows what these machines are doing, and how they are doing it? Fewer people than you’d think. You cannot truly trust code you did not write entirely yourself, and that includes the language compiler. And Thompson co-wrote B, ancestor of C, the language most of the software you use was written in.
The specific application of this reasoning to artificial intelligence (AI) is worth mentioning – even the source code access possible under the EU AI Act doesn’t tell us how models have been trained – but the point is wider. The now-constant news of breaches follows the fact that our systems are fundamentally insecure as sure as night follows day.
News reports have been long on the impact of AI on cyber security, but most of what AI is doing is finding decades-old errors. This is impressive, because the errors didn’t stop the machines from working, but had an adverse impact on stability, speed, utility and, as is increasingly obvious, security.
Last month’s PHP snafu, as reported here in TechCentral, was almost comical. Almost. An AI-assisted source code audit on the PHP programming language found flaws that could allow attackers to take control of Web servers without needing a password or user interaction.
At least it being free software meant that once the problem was understood, the code could be easily patched, so two-and-a-half-cheers are in order for free-wheeling development.
But patching? What does the ubiquity of this term tell us? Anything can need a fix, but software seems to be unique in that it is released with the expectation that it will be fixed later. Its post-material nature is, obviously, the core of this, but it is also a choice.
Fast and loose
Personal computing has been dominated by a seat-of-the-pants approach from day one. This certainly resulted in dynamic development and there is much to be said, frankly, for taking a loose approach to work, even a ludic one.
But free-associating ideas, at least relevant ones, and generally making it up as you go along may be charming in a minor newspaper hack who suffers from impostor syndrome, but it’s probably not the ideal character for someone creating systems we all rely on for crucial, daily tasks. Putting my own analogy in the bin, let’s just say a freeform, build-it-and-they-will-come approach has proven a useful, but partial, view of reality.
A flaw in a modern application, network stack or operating system matters not so much because of a potential breach in the single machine itself but the fundamental interconnectedness of information technology. Someone hacking an Amstrad PCW in 1985 might have caused chaos in the solicitor’s office or parish hall, but not food going off in supermarkets across Europe nor winding up the financial system until it suffers one of its periodic emotional meltdowns.
They say bad workmen always blame their tools. They say a lot of things. I say slow down. Develop less, write fewer lines of code. Think. Test. If this sounds boring, that’s fine. Maybe in their ubiquity computers have become boring. Maybe they should be. If anything needs to be done urgently, it’s the checking of existing code, not adding more.
Ltd
