How techies missed what’s wrong with Horizon, how that lead to multiple deaths and what can we learn from it all?

Andras Gerlits

ITNEXT

Andras Gerlits

·

Follow

Published in

ITNEXT

·

7 min read

·

Jan 9, 2024

6

The Horizon scandal is a multi-decade saga that involved a lot of politics, technology, real-life drama and incompetence. It’s an accounting system developed by Fujitsu for UK Post Offices and introduced in 1999. Problems immediately started after its introduction, but those were mostly brushed under the carpet by management and chalked up to human negligence or misconduct. The human aspects have been front page news recently, but for us, technologists, it also brings up questions of prudence about our everyday practices and if we are doing enough to make sure we understand the nature of the systems we expose other people to.

Preface

I think it would be bad manners to not acknowledge the human suffering. To quote The Mirror:

Out of a total of 736 employees wrongly accused over 230 were jailed, including a pregnant woman, four committed suicide, 33 died, others were declared bankrupt while many saw their personal lives fall apart as they endured public humiliation.

It has become increasingly clear, that they are all innocent for the crimes they were made to take the blame for. In other fields of engineering, big tragedies are occasions for sombre self-reflection and adjustment. Safety procedures changed radically after the Titanic, findings after plane-crash investigations are implemented across the entire industry on short order, yet the changes our industry takes on board are mostly about how we could feel better about our own jobs.

I think it’s important to dwell on this for a bit, as we in enterprise engineering are usually far removed from both the end-users of our software and the business that makes use of them. Perhaps if we spent more time observing the real effects of our work, we could find a reason to identify better with our craft and have better consideration for what’s important.

J’Accuse… ourselves?

Computer systems have bugs all the time but they don’t usually take 13 years until they are recognised. How could even a (clearly very competent) consultancy called Second Sight give an initial report in July 2013 that concluded after a year of investigation that:

We have so far found no evidence of system wide (systemic) problems with the Horizon software

Even if we consider this report to be the first indication of a serious software fault in Horizon¹, these problems have plagued the system for 14 years until that point. This was one of the largest IT systems in the UK with daily (monetary) transactions running in the millions, and this report was the first to hint at the fact that the problems might have to do something with “transactionality”. This is not just a regular case of management “covering its ass” and blaming people caught in the middle (although that’s clearly an important aspect of it). No, in this case even well-respected and diligent professionals genuinely didn’t understand the nature of the faults in the system for well over a decade.

Our sorry state of affairs

It’s not like we haven’t tried to improve things around software-engineering. The last 10–15 years has seen the mass-adoption of both DevOps practices and Agile methodologies. We changed a lot of things about the human aspects of what we do, so that we can feel more comfortable about being in it. This is not a bad thing, but as everything else, changes in an industry should generally serve one of two things: efficiency and/or quality. One can argue that our industry produces more than it did 20+ years ago per head, but I’m pretty sure I would be hard-pressed to find anyone who would tell me that enterprise software nowadays is more reliable or just generally “better” than it used to be, especially in the light of these events.

So what went wrong?

For those who know a lot about distributed systems and consistency, it might be enough to say: “their distributed consistency and resilience model was a mess”. Now, I already see many of you shrugging and saying “well, my model is fine”, but bear with me here for a bit, as I’m about to say something provocative, but one with near perfect scientific precision. For others, who don’t understand what I’m talking about here, I suggest watching the following video with the excellent Steven Murdoch:

There were whistle-blowers from Fujitsu who shared some details about the problems around the development process. Their statements include things like “incompetent staff”, “management uninterested in technical issues” or “budgetary concerns over product-correctness”. I have no problem accepting these as facts. You could be excused for concluding that fixing these problems would have meant a stable system and that ACID guarantees or distribution had little to do with this. You would be wrong. You can read a detailed, point-by-point demonstration about the points Mr Murdoch makes in this video on his blog, referencing the official court-documents:

https://www.benthamsgaze.org/2021/07/15/what-went-wrong-with-horizon-learning-from-the-post-office-trial/

Your system is (probably) a distributed system

A distributed system is one where a request is served by collecting and aggregating data from multiple data-silos, no matter if it’s done on-demand or through some form of streaming. Most enterprise systems involve things like:

- an authentication service,

- some locally developed services,

- some SaaS and

- other, self-hosted ones, developed by 3rd parties (like SAP).

The catch is that you can’t hide from these issues just be being one of these independent data-silos.

Take this for an example: A client wants their credit-card deactivated after making one last, important payment. They open the app, authenticate, invoke both commands in order and then carry on with their day. Say that you’re responsible for a system that manages the balances for each account and tracks how much money can go out of each. There’s also another system that turns the cards on and off (Card Management System or CMS). These must be separate, as the card-provider is an SaaS. In this case, the “truth” that decides whether you should send out the money from the frozen account or if you should report an illegal use of the card is not your decision. It’s up to some system-wide agreement about the relative order of the “debit-” and the “disable-” operations. Unless this order is agreed on by every actor in the entire system, the results can be very different. This is the definition of strong consistency, which is usually considered (in SQL-databases at least) the C in ACID².

Your model isn’t “fine” unless it’s ACID

To add insult to injury, no system is ever fail-proof. Crashes on both application-, VM- and OS-levels happen all the time. This is especially true in the VMs provided by cloud-providers, which (by design) do not run on enterprise-grade hardware. In other words, you must be prepared to handle partial failures within your system. In essence, any failure must veto all operations for the same request. This is the very definition of Atomicity, which is: either all change-operations must happen for an operation or all of them must be reverted.

I think it’s clear where I’m going with this. When you’re advised by microservice books and articles to “cut along domain-model boundaries”, it’s them waving their hands on how this can be achieved. You’ve only actually achieved a stable platform once you can promise each letter in ACID.

The moral of the story is that unless these services:

- work in isolation

- fulfil all the requirements all its users have

- from their local storage only,

you are already a distributed system with all the complexities it entails.

They did tell you it would be “fine”. They were wrong

If you do a search for “microservice consistency” and open the first link on Google, you’ll come across something resembling this article:

Data Consistency in Microservices Architecture

In this article, I’d like to share my knowledge and experience in Garanti BBVA, about moving from monolithic to…

dilfuruz.medium.com

This is a particularly good example in that it sums up the prevailing thinking around microservices. It also misses the aspects we were discussing above entirely, leaving anyone who follows them in a Horizon-like situation. How badly this exposure will affect your end-users depends on the specific environment your software was built for and other conditions like “does your management show exceptional critical thinking skills?” and “how widely distributed is your system?”, but these questions are overwhelmingly out of developer’s hands. The only thing we can do about the situation is to start focusing on what’s important: reliability and correctness.

What we should all learn from Horizon’s terrible example is that what we do matters to the people using them. We either take ownership of the correctness of our software or management will (quite rightly) start trusting us (our software) less and less, which will lead to much worse work-environments for all of us.

Correctness should start to matter again. If for nothing else: our own benefit.

¹ To be fair to Second Sight, they did go to war with the management of Post Office over the fact that they weren’t given access to legal documents pertinent to the case, so they weren’t allowed to finish their work. This was covered in this in 2015:

Post Office obstructing Horizon probe, investigator claims

The Post Office is obstructing a probe into problems with its Horizon IT system by refusing to hand over files relating…

web.archive.org

²Computer Science usually uses a different definition of the “C” in ACID, namely “Linearizability”, which is a much stronger requirement than the one we’re using here. We allow ourselves this liberty as we consider SQL databases to have ACID properties, as is the usual parlance within industry.

Software Development

Distributed Systems

MOST RELEVANT

Olaf Schlüter

Olaf Schlüter

about 1 month ago (edited)

Or people? My most important take-away from learning agile software development was "treat people like adults". As the first thing I noticed was that in any implementation of agile, when those people supposed to do something within a certain budget and time constraint tell you that it is not possible to do that given the constraints, they are taking seriously.

It was one major reasons of the so called "software crisis" in the 2000s that management did overrule production people on a regular basis, making promises to customers no one in their company was able to keep. Trillions of dollars (billions in Germany alone) have been wasted that way, invested in software projects that did fail and needed to be abandoned. If one does not pay what it costs and does not give the resources to a project, that it needs, it doesn't go anywhere but to a failure.

And most certainly this change of project management produces better and more reliable output. Software may be buggy still, but failed development projects don't happen that often any longer. If you take all those failed projects into account being buggy in any use case, you will see that the quality of software development was abysmal before the resolution of the software crisis by agile methods.

2

Abdelfattah Sekak

Abdelfattah Sekak

about 1 month ago

Complex topic, but well explained.

1

Thomas Lunsford

Thomas Lunsford

about 1 month ago

Please describe the topic at the beginning of the article with the assumption that the reader just came from a yearling coma.

1

Fabien Ninoles

Fabien Ninoles

about 1 month ago

Great article and I like the reminder that there is no silver bullet (Microservices, NoSQL, Containers, Serverless) to the problem of correctness, scalability and reliability. I also like that you pointed out that one challenge is to decouple around the domain boundaries. Very often, this isn't done because such boundaries are often shared across teams. "Make the teams also cross-domain" is often the answer to that, and you'll find that is not fixing anything. Worse, it can create a big bad of muds of different techs integrated with various standards, making everything more complex. Aspects like security (AAA), scalability, observability, etc. are cross-domains, and also are infrastructure, QA, SCM, maintenance, migration, and many others. However the number of standards use in a company, most of them ends up with too many.

However you try to split a system, we most admit two things:

1. Any complex system will always grow to a point where he couldn't be maintained by a single group.

2. Problems inside the group boundaries are usually the ones easy to fix. The difficult ones always happens across those boundaries, wherever those boundaries are set.

So, yes, as engineer, I must understand one important thing: even if the subsystem I wrote is good or even fully ACID compliant, it is still my responsibility if the system that is part of isn't. That level of ownership, which transcends boundaries, is rare but key to solve the complex problematic of our field.

1

PixelTree

PixelTree

about 1 month ago

Your exploration of the Horizon scandal and its implications for technologists is both insightful and thought-provoking. The profound impact on individuals involved and the systemic flaws in the development process highlight the need for a reevaluation of our practices in the tech industry. Your emphasis on the importance of understanding the nature of systems we create and the consequences they may have on end-users resonates deeply. The call for greater focus on reliability and correctness serves as a crucial reminder for all of us in the software development community. Thank you for sharing this valuable perspective. 👏🔍

1

Andrei Dascalu

Andrei Dascalu

about 1 month ago

"We either take ownership of the correctness of our software or management will (quite rightly) start trusting us (our software) less and less"

the principle is correct but the lack of practical way forward is disappointing.

What this article brushes over is the fact that Horizon was launched in 1999/2000 as a product of essentially outsourcing is a very top-down environment in a time lacking all the concepts that we're taking for granted nowadays.

Back then it wasn't about management trusting "us" from the start and losing the trust along the way but in a top-down system as found in any Japanese (but not only, not by a longshot) enterprise, it's the fact that management controls everything from the get-go and engineers simply execute whatever they are told. It's not even that trust wasn't there to begin with but more that there was no such concept.

And in the decades following, when engineers helped build a culture of ownership, but faced with a legacy system with known and unknown holes but management unwilling to even acknowledge the shortcomings ... what exactly should be done?

We do live in a different world now but whether the lessons count as learned depends on how the enterprise is managed. Take a cue from even today's software giants ... Google used to have an engineering first culture during which its top products were developed and moved to a marketing-first culture where innovation and quality stopped.

Hide replies

Andras Gerlits

Andras Gerlits

AUTHOR

about 1 month ago

I agree that this is mostly management's fault, but if engineers knew about the unique challenges distributed systems pose, they wouldn't have released it this way. We have the head architect saying in court that the system is robust. He wouldn't have said so had he known about these issues. It's true that distributed consistency received a lot more attention recently and also that we have better tooling. Sadly, that doesn't translate to better software. I work as a distributed systems consultant and the practices I see would make grown men cry. People don't know more about these questions than those engineers did in 99-00 and so the systems they build are similar to Horizon. My point is that Horizon should be a cautionary tale that stops us from building more badly thought through microservices and integration solutions.

If you'd like to see what the future will look like, look at my recent articles. We're bringing a generic solution to market, which fixes all these problems via SQL and ACID guarantees

Hide replies

Andrei Dascalu

Andrei Dascalu

about 1 month ago

Why exactly wouldn't the head architect have said the system is robust in court had he known about the issues?

Court is a lot about ass covering and a robust system isn't free from these issues, it's more Bing able to recover from them. However, the denial can easily become "the truth" in such a long running project, years in development ahead of the 1999 launch. Be with a project for a long time, even if you were told about the issues in 1998, orders from management to go ahead will be obeyed.

It's hard to grasp now how in economies dead set to prove their IT worth in a time of near total US dominance their engineers didn't have the power to stop a launch.

It's difficult to come in the spotlight in 2020 about stuff that happened pre-1999 and say something else than what you've convinced yourself to be the truth while blocking out any details that might contradict that.

Why Horizon is just the tip of the iceberg for Britain’s crumbling IT

Outsourcing the state’s technology has led to an embarrassing litany of failures

James Titcomb

21 January 2024 • 10:00am

Related Topics

Post Office scandal, Post Office

Imagine if four in every five bridges built in Britain fell down. There would be national outrage. The companies building those bridges, the civil servants who ordered them and the ministers in charge would all likely pay with their jobs or worse.

Fortunately, the majority of bridges do not tend to fall over. But there is one area that suffers such rates of failure.

Twenty years ago, researchers at Oxford calculated that 84pc of large IT projects in the UK failed in some form – either by being cancelled, failing to deliver, arriving late, or going significantly over budget.

Software and bridges are not the same. But growing anger about the Post Office Horizon scandal has shone a new light on the dismal record in building IT systems.

Faulty software built by the Japanese tech giant Fujitsu led to hundreds of sub-postmasters’ lives being ruined after misguided prosecutions.

We are becoming increasingly reliant on computer systems across our lives. Could another Horizon scandal be lurking in the code?

In 2011, a committee of MPs called the Government’s approach to IT a “recipe for rip-offs” while researchers have called Britain “a world leader in ineffective IT schemes for government”.

Insiders describe a sclerotic approach to computer systems and a chronic shortage of skilled staff, with Westminster captured by an oligopoly of IT giants growing rich.

Last year, Parliament’s Public Accounts Committee warned that Britain’s IT infrastructure continued to be utterly reliant on ageing computer systems, with civil servants more likely to expensively prolong the lifespan of old programmes rather than risk starting afresh.

This, critics say, has led to an embarrassing litany of failures in the computer systems that the country relies on for everything from pensions to stopping airliners from crashing into one another.

The failures of Fujitsu’s Horizon system, in which transactions were often double-counted, leading to artificial deficits at Post Offices, have attracted enormous attention because of the miscarriages of justice it led to: sub-postmasters being prosecuted, jailed and committing suicide.

Those cases emerged because the Post Office vigorously pursued private prosecutions aided by its own investigative branch.

But it is only one of a series of notorious public computing projects that have failed to deliver. They include New Labour’s botched National Programme for IT in the NHS, which was abandoned more than a decade after it was announced and as it was billions over budget; and Fujitsu’s Libra, designed to provide a national system for magistrates, which was described as “one of the worst IT projects ever seen” and partly taken away from the Japanese giant after going hugely over budget.

There are also the systems that did come into use and failed those who rely on them. In 2020, a botched Ofqual algorithm caused mayhem in schools when it was used to deliver A-level results while schools were shut.

Last year, 2,000 flights were cancelled after a bespoke air traffic control software system crashed.

People wait near check-in desks at Gatwick Airport

An IT meltdown in air traffic control systems last August left passengers stranded CREDIT: Carl Court/Getty Images

“Horizon is an extreme example of this but from a technical point of view there are other systems that are similarly parlous and poorly designed,” says Mike Bracken, who ran the Government Digital Service and is a founding partner at the Public Digital consultancy.

He adds that the Post Office scandal was made worse by inadequate technical knowledge and governance.

“The other key issue with Horizon is that the Post Office was judge and jury and could choose to prosecute.”

Britain was once a world leader in major IT systems. Colossus, the pioneering programmable computer used by the Bletchley Park codebreakers during the Second World War, was actually designed by a Post Office engineer.

The Central Computer and Telecommunications Agency was credited with developing technical skills within government until it was wound down at the end of the century.

Instead, in the Nineties, successive governments embarked on a series of giant IT projects, outsourced to a handful of large companies.

In 1994, Inland Revenue awarded a 10-year £3bn contract to the US software giant EDS, founded by the two-time presidential candidate Ross Perot. As part of the deal, 1,900 IT staff at Inland Revenue moved to the company.

The deal would eventually go wrong when EDS’s system failed to handle new tax credits, but at first it was seen as enough of a success to be imitated across the public sector. Enormous contracts were awarded, almost exclusively to the major IT companies large enough to handle them: the likes of Capgemini, Accenture, IBM – and Fujitsu.

Helen Margetts, a professor of internet and society at the Oxford Institute and director of the Alan Turing Institute’s Public Policy Programme, says this separated Britain from many other countries modernising their own systems.

“What was unique about the UK was the size and length of the contracts, these were really big contracts that fewer and fewer companies could bid for, we were outsourcing entire IT divisions,” she says.

“All the expertise was stripped out. Contracts got bigger and bigger and people in government became less and less keen on being involved. The companies promised to take over all the misery and risk of these great big contracts.”

Very public failures such as the fiasco around the National Programme for IT at the NHS, combined with pressure on the public finances, led the Coalition government to declare in 2010 that the era of “mega IT contracts” had come to an end.

Techies returned to Whitehall as part of a Government Digital Service division, which led to relative successes such as the unified gov.uk website.

However, reversing the mistakes of the last three decades has been slow. Fujitsu has won £4.9bn of public contracts since a court found in 2019 that the Horizon system was at fault.

Much of that has been spent on extending the life of existing systems, leaving services reliant on ageing technology.

Last year, the Public Accounts Committee found that departments were continuing to use outdated “legacy” systems instead of building new ones fit for the 21st century, in part because of a lack of skills.

“These have resulted in services which, although they might look good on the surface, are costly and problematic to run,” it said.

In one case in 2014, Whitehall struck a £5.8m deal with Microsoft to support tens of thousands of computers still running Microsoft’s 13-year-old XP operating system, rather than upgrade to a modern version.

Some systems such as those for paying pensions and managing borders date back to the 1970s.

In a blog post last week, Bracken decried a “yawning gap in knowledge and curiosity around technology [that] remains true across senior levels of government, the civil service, parliament, and the judiciary… buttressed by a marriage of convenience between those leaders and large enterprise IT companies like Fujitsu”.

He added: “When the alternative is for them to engage in wicked problems that are unglamorous, difficult to understand and do their careers little good, many are all [too] happy to hand over cash and control to the contractors.”

Margetts says a continued reliance on old computer systems, rather than replacing them with modern ones, has set us up poorly for advances in areas such as artificial intelligence.

Both the Conservatives and Labour believe they can modernise the state using AI. Chancellor Jeremy Hunt last week said at Davos that he hoped to harness the technology to make public sector services more efficient.

However, these systems rely on vast quantities of data that are inaccessible on our ageing computer systems.

Margetts says: “It makes it difficult for agencies to innovate when you’re operating from a very old system that doesn’t generate data.”

Outrage about the Post Office scandal has led to promises that the victims’ injustices will be resolved swiftly. Fixing Britain’s shaky IT systems will take far longer.

Why Horizon is just the tip of the iceberg for Britain’s crumbling IT

Outsourcing the state’s technology has led to an embarrassing litany of failures

James Titcomb

Related Topics

- Post Office scandal,

- Post Office

Imagine if four in every five bridges built in Britain fell down. There would be national outrage. The companies building those bridges, the civil servants who ordered them and the ministers in charge would all likely pay with their jobs or worse.

Fortunately, the majority of bridges do not tend to fall over. But there is one area that suffers such rates of failure.

Twenty years ago, researchers at Oxford calculated that 84pc of large IT projects in the UK failed in some form – either by being cancelled, failing to deliver, arriving late, or going significantly over budget.

Software and bridges are not the same. But growing anger about the Post Office Horizon scandal has shone a new light on the dismal record in building IT systems.

Faulty software built by the Japanese tech giant Fujitsu led to hundreds of sub-postmasters’ lives being ruined after misguided prosecutions.

We are becoming increasingly reliant on computer systems across our lives. Could another Horizon scandal be lurking in the code?

In 2011, a committee of MPs called the Government’s approach to IT a “recipe for rip-offs” while researchers have called Britain “a world leader in ineffective IT schemes for government”.

Insiders describe a sclerotic approach to computer systems and a chronic shortage of skilled staff, with Westminster captured by an oligopoly of IT giants growing rich.

Last year, Parliament’s Public Accounts Committee warned that Britain’s IT infrastructure continued to be utterly reliant on ageing computer systems, with civil servants more likely to expensively prolong the lifespan of old programmes rather than risk starting afresh.

This, critics say, has led to an embarrassing litany of failures in the computer systems that the country relies on for everything from pensions to stopping airliners from crashing into one another.

The failures of Fujitsu’s Horizon system, in which transactions were often double-counted, leading to artificial deficits at Post Offices, have attracted enormous attention because of the miscarriages of justice it led to: sub-postmasters being prosecuted, jailed and committing suicide.

Those cases emerged because the Post Office vigorously pursued private prosecutions aided by its own investigative branch.

But it is only one of a series of notorious public computing projects that have failed to deliver. They include New Labour’s botched National Programme for IT in the NHS, which was abandoned more than a decade after it was announced and as it was billions over budget; and Fujitsu’s Libra, designed to provide a national system for magistrates, which was described as “one of the worst IT projects ever seen” and partly taken away from the Japanese giant after going hugely over budget.

There are also the systems that did come into use and failed those who rely on them. In 2020, a botched Ofqual algorithm caused mayhem in schools when it was used to deliver A-level results while schools were shut.

Last year, 2,000 flights were cancelled after a bespoke air traffic control software system crashed.

People wait near check-in desks at Gatwick Airport

An IT meltdown in air traffic control systems last August left passengers stranded CREDIT: Carl Court/Getty Images

“Horizon is an extreme example of this but from a technical point of view there are other systems that are similarly parlous and poorly designed,” says Mike Bracken, who ran the Government Digital Service and is a founding partner at the Public Digital consultancy.

He adds that the Post Office scandal was made worse by inadequate technical knowledge and governance.

“The other key issue with Horizon is that the Post Office was judge and jury and could choose to prosecute.”

Britain was once a world leader in major IT systems. Colossus, the pioneering programmable computer used by the Bletchley Park codebreakers during the Second World War, was actually designed by a Post Office engineer.

The Central Computer and Telecommunications Agency was credited with developing technical skills within government until it was wound down at the end of the century.

Instead, in the Nineties, successive governments embarked on a series of giant IT projects, outsourced to a handful of large companies.

In 1994, Inland Revenue awarded a 10-year £3bn contract to the US software giant EDS, founded by the two-time presidential candidate Ross Perot. As part of the deal, 1,900 IT staff at Inland Revenue moved to the company.

The deal would eventually go wrong when EDS’s system failed to handle new tax credits, but at first it was seen as enough of a success to be imitated across the public sector. Enormous contracts were awarded, almost exclusively to the major IT companies large enough to handle them: the likes of Capgemini, Accenture, IBM – and Fujitsu.

Helen Margetts, a professor of internet and society at the Oxford Institute and director of the Alan Turing Institute’s Public Policy Programme, says this separated Britain from many other countries modernising their own systems.

“What was unique about the UK was the size and length of the contracts, these were really big contracts that fewer and fewer companies could bid for, we were outsourcing entire IT divisions,” she says.

“All the expertise was stripped out. Contracts got bigger and bigger and people in government became less and less keen on being involved. The companies promised to take over all the misery and risk of these great big contracts.”

Very public failures such as the fiasco around the National Programme for IT at the NHS, combined with pressure on the public finances, led the Coalition government to declare in 2010 that the era of “mega IT contracts” had come to an end.

Techies returned to Whitehall as part of a Government Digital Service division, which led to relative successes such as the unified gov.uk website.

However, reversing the mistakes of the last three decades has been slow. Fujitsu has won £4.9bn of public contracts since a court found in 2019 that the Horizon system was at fault.

Much of that has been spent on extending the life of existing systems, leaving services reliant on ageing technology.

Last year, the Public Accounts Committee found that departments were continuing to use outdated “legacy” systems instead of building new ones fit for the 21st century, in part because of a lack of skills.

“These have resulted in services which, although they might look good on the surface, are costly and problematic to run,” it said.

In one case in 2014, Whitehall struck a £5.8m deal with Microsoft to support tens of thousands of computers still running Microsoft’s 13-year-old XP operating system, rather than upgrade to a modern version.

Some systems such as those for paying pensions and managing borders date back to the 1970s.

In a blog post last week, Bracken decried a “yawning gap in knowledge and curiosity around technology [that] remains true across senior levels of government, the civil service, parliament, and the judiciary… buttressed by a marriage of convenience between those leaders and large enterprise IT companies like Fujitsu”.

He added: “When the alternative is for them to engage in wicked problems that are unglamorous, difficult to understand and do their careers little good, many are all [too] happy to hand over cash and control to the contractors.”

Margetts says a continued reliance on old computer systems, rather than replacing them with modern ones, has set us up poorly for advances in areas such as artificial intelligence.

Both the Conservatives and Labour believe they can modernise the state using AI. Chancellor Jeremy Hunt last week said at Davos that he hoped to harness the technology to make public sector services more efficient.

However, these systems rely on vast quantities of data that are inaccessible on our ageing computer systems.

Margetts says: “It makes it difficult for agencies to innovate when you’re operating from a very old system that doesn’t generate data.”

Outrage about the Post Office scandal has led to promises that the victims’ injustices will be resolved swiftly. Fixing Britain’s shaky IT systems will take far longer.