Have we lost our faith in algorithms?
I suppose the term “algorithm” conjures up a lot of things for most people, but my focus in this month’s column is software – an algorithm is a piece of software or computer program, which is designed to solve a problem. Algorithms are the central components that bring about the covert mechanics of all our products, gadgets and services that we use on a daily basis.
Process-focused vs. agile development
I’m sure most of you have read the sensational headlines when a so-called algorithm has gone wrong. One such example would be the algorithms that were used in the British education system at the start of the pandemic. Software or algorithms were used to grade students based on their overall performance. There was an angry reaction from students, parents, teachers and the public in general, where the British government had to rethink their next steps.
As a former software engineer, I am aware of the challenges faced by developers today, especially as hardware has become increasingly complex and our dependency has, likewise, grown. As an engineer, I adhered to several doctrines to ensure my software was as it should be when I delivered it. In particular, I am a big supporter of a process-driven software development lifecycle. I have worked with ISO 9001 / TickIT and the capability maturity model (CMM) at the highest level (5). I have also worked in an agile development environment, but really this has a feeling of “Let’s make it up as we go along,” with the added bonus of the software plan being malleable; in other words, the plan is easily adaptable to accommodate changing needs from the client and end-user. It also (apparently) aids in an accelerated delivery. As such, with the promise of an adaptable plan and an accelerated delivery schedule, for me, this breeds contempt. The development lifecycle echoes a theme of “organic growth” rather than a known input (the requirements) and a defined output (the deliverable).
Don’t worry, your software has crashed!
I certainly do not like or approve of this philosophy or way of working, but I know that there are many out there who do – so call me old school! A process-driven plan is solid and presents identifiable stages of the software development lifecycle – each stage can be tested. Meaning, if you like, there is a beginning, middle and clear end. If a change needs to be made, then there is a process for that and, yes, it will impact upon the delivery schedule. In my experience, any significant changes can be placed into a second phase of development so as not to largely impact the first delivery – it’s all about getting it to work first and to include additional functionality later. After all, a process-driven plan comprises identifiable stages of the development lifecycle. What’s more, there’s an enormous amount of traceability in the development process, along with an exhaustive and traceable testing regime.
With this in mind, there is a reason for discussing this topic, since I have seen over a couple of decades that we have come to accept “dodgy” software. For example, I was working at one of the top three television manufacturers in the world about a decade or so ago and, despite operating with a development ideology of a very strict CMM level 5, it was somehow acceptable for the TV to “crash” because most consumers would not know. The assumption made was that ordinarily most consumers would dismiss what had happened and simply restart their television.
I fear, if we continue to forgive these technology giants, or anyone else developing key algorithms intrinsic to our lives, then it might just become increasingly worse.
If I had developed software like that, I would be fired!
This is just one of many, many examples and it seems to be irrespective of your chosen development ideology that excuses are made. Whatever smartphone, for example, you are using, you come to realize it is not perfect, but you tolerate its so-called foibles and live with it. Likewise, with your smart home hub, where you might use an intelligent virtual assistant (IVA) you come to realize that the faux artificial intelligence is actually rather simple, because you have to explain everything in a command-like format. It could be a Google Home Hub, Amazon Echo, Cortana, or Siri but, regardless of make, you have to be very specific, since it won’t tolerate your imperfections in dialogue. You see, as humans, we are sometimes confused about what we want to say, we fumble with our words and invariably hesitate whilst we mentally look up what we want to say or even change our minds midway through, to which our IVA will typically dismiss us with a beep or a “Sorry, I didn’t get that” response.
We have become dependent on software or algorithms to make decisions on our behalf, where many businesses, in fact, are reliant on software-driven decision-making processes. It has helped industries and businesses reduce their costs, whilst driving efficacy across their business. I see today’s numerous products, application, and their associated periphery services as often being incredibly painful and frustrating to use on occasions – we have seemed to have lost our focus. And I’m not sure if it’s going to get any better, because today’s new generation has accepted this ineptitude as part of the overall user experience. It’s cringingly awful and, I dare say that, if I developed software like that, I would have been fired! I was always told, “Don’t deliver the software if it isn’t right,” which is akin to a Michelin star chef in his kitchen saying: “Don’t deliver that dish if it isn’t right” – the same pride and principles apply. Our forgiving attitude today is not acceptable, hence why I am so high up on my soap box right now. I am eager to define a new standard of expectation of good software development practice and ethics, where we must all take part in defining what’s right and what’s wrong.
Until next time …
I fear, if we continue to forgive these technology giants, or anyone else for that matter tasked with developing key algorithms, which nowadays have become intrinsic to our lives, then it might just become increasingly worse. The purity and sheer joy in developing software, for me, has been a privilege but today I think that, sadly, it has become a chore for some. I have reveled in the marvel of solving a problem or delivering a solution through software, where my experience has been ethereal and, on occasions, “God-like”.
So, this is where your “hardcore assembler and C-loving former software engineer” Dr. G signs off.
Originally published in Technically Speaking.