The past decade provided us with immense technology development - and horrible tech fails.
Reflecting on the failures in IT is essential to draw conclusions and learn from someone else’s mistakes.
One would think that the process of developing software has been streamlined to perfection over the past years, especially given the rapid development of technology. Sadly, one would be wrong - even the largest companies with decades of experience in managing software development provide us with stories of epic IT outsourcing fails. From software built by $9/hour by remote developers to multi-million dysfunctional patient records systems, here are some of the largest IT failures of the past decade.
Boeing 737 MAX disaster and offshore outsourcing
The two fatal crashes of Boeing 737 MAX in 2019 have uncovered a myriad of software problems plaguing the plane. Boeing 737 MAX has been grounded ever since, and new glitches were found throughout 2020. Each fix seems to snowball into multiple new problems.
Mature companies, which are under a constant threat of new products and competitors, may be motivated to reach new heights of tech advances, but may also be prompted to take every shortcut in the name of cost-cutting.
Having laid off many experienced engineers, Boeing negotiated with suppliers to cut costs, resulting in a large percentage of their workers to work on temporary bases for as little as $9 an hour, often working for large Indian companies. In hindsight, choosing an outsourcing partner based solely on their pricing might not have been a good idea which resulted in two plane crashes.
Software developers working for such low rates are either inexperienced or undervalued, neither of which results in a high quality code. Using their services is like playing russian roulette: the risk is just not worth the possible desirable outcome. Unreliable contractors can make even the most trivial tasks, like building a website, a nightmare. An unoptimised website can cause sudden crashes during a marketing campaign, tarnishing the reputation of your business and rendering the advertisement investment useless. The loss of money can be the least of your problems, as legal issues are very real in case of badly implemented data protection.
Picking developers with no experience in aerospace engineering and trusting them with the passengers’ lives, Boeing made a huge mistake. It was reported the code written by the contractors often required corrections and did not meet the company’s standards. The language barrier made communication difficult.
The most attractive aspect of software outsourcing in India - the low hourly rates - are offset by the cost of someone who has to manage and oversee these offshore teams, making their work far less attractive money-wise. The monetary loss caused by the tarnished reputation remains to be evaluated.
State of Indiana’s Failed Welfare System
This time the company hired to develop new software was IBM - a well-known US-based corporation with a generally good reputation. However, this time the project was a failure for both sides which resulted in mutual lawsuits. A county court ruled in favor of IBM awarding $12 million. The development of a $1.4 billion system failed as the task simply overwhelmed IBM and both sides failed to properly assess risks.
The state of Indiana needed to update their welfare processing system through outsourcing. For the contractor, the size of the project seemed to be a deciding factor to accept it - despide other states’ similar projects failing.
The state of Indiana had a problem with letting things change. The state took a controlling approach to the change mechanism, refusing their partners to add more resources even when the project’s conditions changed. Time brought unexpected change which the State of Indiana resisted, which could only lead to a disaster.
A fixed-price approach and refusal to adapt to changes has led to a giant loss of taxpayer’s money. There’s no straightforward way to make sure a giant 10-year-long project will progress without issue, but adapting Agile development practices and Time & Materials payment scheme could have made it easier to scale the project up or down when needed.
IBM is as much at fault here as its extensive experience should have allowed them to assess the risks of the project.
Medical Software and a Story of Horrible UX
The Cambridge University Hospitals NHS Foundation Trust had a goal of introducing a new patient record system. A US company was hired to perform the task and develop a system which would allow medical staff to access patient records via mobile devices in order to speed up their work. The results of their work was disastrous - a 20% drop in performance which is critical when you deal with people’s lives and health. On top of that, a failed £200m IT project caused a plunge in Cambridge University Hospitals’ finances into the red resulting in a £64m budget deficit in 2015, in part due to the mishandling of the project.
The Trust did not prepare for the amount of work it would take to implement a system like this, which resulted in an undertrained and confused staff. Combined with poor UX/UI and a lack of product design, the project ended up costing lots of money while not delivering the system’s benefits to those who needed them.
The system was also riddled with bugs, like inaccurate discharges for patients which would lead to some of them not receiving follow-up care. If proper QA standards were implemented, issues like these would have been easily caught early on.
The case of the Cambridge University Hospitals NHS Foundation Trust’s failure is an extreme example of how costly it can be to omit crucial parts of developing high-quality software. Even when working with a public institution which is not very familiar with software development as well as a tight budget, the right approach can lead to incredible savings. Well-designed software often does not require training and can be simply introduced to staff.
What We Can Learn
Outsourcing isn’t a bad idea. Often it is the best thing a company can do to ensure their project is successful. But the examples above show that outsourcing should be a strategic initiative with proper planning and execution.
Cost Should Not Be The Only Consideration
While it is tempting to hire offshore developers with a very low hourly rate to save money, this approach often ends up costing more. Inexperienced or undervalued developers produce low quality code, often lack relevant experience and are sometimes difficult to establish effective communication due to language and cultural barriers.
Rather than choosing the lowest bidder, one should take into consideration the IT partner’s experience, track record and reputation. After all, if your main criteria for selecting a provider is cost then you will get what you pay for.
Failing to Accept The Need For Flexibility
Large projects can change over time, technology developments happen rapidly in the world of IT, which require a degree of flexibility to change the scope of the project. Common-sense changes are a natural part of any software development project and allow it to remain relevant.
Not Seeing The Relationship As A Strategic Partnership
Long-term software development projects require constant communication and support of both sides to make sure the project is on the right track. The service provider can become an invaluable partner in achieving business growth. Viewing software development projects as a “necessary evil” often leads to a disaster.