The following is a true story about a company that almost went bankrupt due to a disaster that obliterated their website. And worse still, lost customer data gathered by the site over a number of years. I’ve changed the company name but in essence the story is true.
Cyber Solutions (my name for them) prided themselves as a company using cutting-edge technology and providing exceptional customer service. The heart of their business was a meticulously maintained website that was the envy of their competitors. Little did they know what was around the corner.
One day, as luck would have it, Cyber Solutions website fell victim to a hacker that left the site unusable. Initially, they were not too concerned since they had diligently backed up the website daily, confident that it would serve as their saviour in times of crisis.
Unperturbed, Cyber Solutions’ tech team embarked on the restoration process. Armed with their backup, they believed that they would swiftly reclaim get the company back on track. However, fate had a different plan in store for them.
As they initiated the restoration from their carefully preserved backup, a sense of anticipation filled the air. But to their dismay, the process failed. Moreover, all the backup copies they had stored also failed.
The website, which once stood as a beacon of their success, remained corrupted and inaccessible together with the customer data they had complied over the years.
The Cyber Solutions tech team had followed best practices and diligently stored their backups, but their hopes of a swift recovery were shattered. It was a moment of utter despair, as they grappled with the realization that their backup strategy had failed them when they needed it the most.
Undeterred, the team rallied together to troubleshoot the issue. They sought assistance from expert technicians, determined to uncover the root cause of their failed restoration. Immediately, they discovered a flaw in their backup process—a silent glitch that had rendered their backup unusable.
What’s more, they soon realized that the glitch had been introduced when the backup scripts had been set up some years previously. They had been diligently backing up every day oblivious to the fact that what they were producing and preserving was useless.
It wasn’t all bad news though. With a renewed sense of determination, Cyber Solutions set out to rectify the mistakes. They revised their backup strategy, implemented additional layers of redundancy, and partnered with a reputable backup solution provider.
Eventually they clawed their way back to full operation but not after considerable expense to rebuild the website and rebuild their customer database. This also came with considerable disruption to many of their customers and much good will from some of them.
So what went wrong? The technical details are not all that relevant here. The key failure is in the strategy. The one this Cyber Solutions failed to do was to test the backups. They continued to blindly produce daily backups in the assumption that they were error free.
Had their backup strategy included regular tests to check that the backup could be restored, they would have found the problem years earlier. You would be amazed at you many people fall into this trap.
To sum this up in a single phrase, “You get what you inspect, not what you expect”.
