Disaster Recovery doesn't have to be a disaster!



Disaster Recovery doesn't have to be a disaster!

Thursday the 30th of May 2019

Being on the front line when it comes into play, a disaster recovery scenario, a large company loses connection to the outside world, your companies main internet connection goes down through no fault of any of the team, but a failure when it comes to the provider.

This is your main office, which hosts services to the rest of your World Wide company, your file servers can no longer be reached, all use of remote desktops being accessed from the outside world as well as your internal systems being hosted on Thin Clients, everybody gets, chucked, thrown back to the log in page; a situation like this, in a high paced sales environment – Not even your PIN-based printing services work anymore!

Luckily today, in the scenario we faced, our SIP phones were still functional, which gave our busy staff the ability to still make outgoing calls and accept incoming and explain to the customers, our clients that we will not be able to fulfill their requirement at this present moment in time due to ‘The systems being down’ – Walking around the office today, you wouldn’t believe how many times I heard this.

So what can we do at a time like this?
How do we fix a problem we don’t even know how happened?

Today I saw something beautiful, the moment the alert was raised, our networking team sprang into action, starting the diagnosing stage of their troubleshooting – Watching them run into our communications room was like watching a slow-mo emergency Baywatch scene.
The Service Desk team I work with hands-on phones ready to take the influx of calls. Myself – It was my duty to walk the floors and see our clients.

Quick paced I marched across the hall onto the main sales floor to be greeted by raised heads and focused eyes like I had stumbled into Meerkat territory, bank of desk to bank of desk, team by team I stopped and addressed our customers, informing them that, yes, we are aware and yes it is being looked at and every effort is being put into resolving this issue, and sorry, no I haven’t been given an estimated time of fix; - As a point of personal opinion, you’re probably all thinking, why not use an internal comms system to inform everybody of the issue, personally, I believe in a situation like this, it should be handled personally, so the people you are serving, the people who are currently unable to work thus unable to make money feel like they are being looked after better and being kept in the loop.
I speak loudly and clearly and make sure everybody can hear me and I am understood, the move on to the next bank of desks.

The customer response to my visits was overall a pleasant one, they are not a group of strangers to me, I often visit desks even if there is no DR situation occurring to help with general issues and to check generally on our equipment, make sure it’s all functioning correctly.

It took roughly, one and a half to two hours to identify the issue and re-route all of the internet through the backup connection, overall that is a fairly quick turnaround.
When I heard the news that we would be back up and running again in the next 5 to 10 minutes, once again I made my way back to the sales floor, bank by bank, team by team, giving them the good news, one team even took up and applauded (including a few woos) me which has never happened to me in my life.

Overall – The moral of this story!

Look after your customers, if things get rough, they’ll be on your side if you’re on theirs,
Secondly, ALWAYS have a disaster recovery plan, always have people who have allocated roles if things should arise!


Peace


  



Comments

Post a Comment

Popular posts from this blog

When I grow up I want to be a …

When I grow up I want to be a … What if the career you wanted as a child was the career you actually got as an adult? I’m sure a lot of you remember what you wanted to be as an adult when you were a child being asked the question ‘What do you want to be when you’re grown up?’ If that answer became reality for every child who knew what they wanted to do, I’m sure the world would have a lot more Doctors, Vets, Scientists and Astronauts. What if this same child now, saw what they were doing, twenty to thirty years from that moment after answering that question? Sure, I mean a few of us have gone ahead and followed our dreams and became what we wanted as a child, but what of the rest of us? I want to try something with you: Be yourself, travel back in time, stand in that room where you were asked that question, remembering replying to that question then reflect on what you became? Did your dream change? Did your options increase as time went on? Do you still lust...
Read more

Never give up on your dreams, even if it turns into a giant fireball

Never give up on your dreams, even if it turns into a giant fireball Following my previous post about speaking to your younger self about what you have achieved, yesterday evening presented one of those opportunities to create something which could fly, originally the idea was to create a Chinese lantern with crate paper and a wax soaked rag, but this was deemed too weak and if we were going to do something like that, we would have to… Make it more interesting shall we say. Previously Dan and I had played with PC Fans and volt steppers and made some “super cool” USB Powered desk fans which now sit beautifully on Dans desk ready to keep him cool when playing Battle Royal Games; however, this time, we wanted to use the same desk fan idea but create a fire tornado as a method of propulsion [If you haven’t seen a fire tornado I would highly recommend you check it out] – The concept behind it, you have a fuel source which is ignited and the open top container is rotate...
Read more