An IT Question: What Has Disaster Taught You?

Experience, it is said, is the best teacher.  Oh so true.  I guess that is why we do disaster recovery tests rather than just waiting until disaster really strikes.  This got me thinking - what is the most surprising thing you learned as a result of an actual disaster recovery or even a test?

Mine was that people expect your disaster recovery process to cover everything.  In one of my previous jobs we had a major fire at the corporate office and it was necessary to relocate people to the plant about one hundred miles away.  From an IT standpoint we were able to keep things running and or data loss.  What surprised me was that when people came to the plant they expected IT to provide them with a PC, which we did, but a PC complete with office, furniture, office supplies and administrative assistants.  Apparently when people fail to plan for disasters they look to those that have a plan as their rescuers for everything.

So, what is the most surprising thing you learned as a result of an actual disaster recovery or even a test?

[Update 10/12/2007 afternoon - corrected typos including "did not suffer any systems downtime" and "they look to those that have a plan"]

"Help" photo by Cobber99

Got a question you'd like me to post for future discussion?  Email it to me using the "Email Mike" link in the left hand column.

If this topic was of interest, you might also like the other posts in the IT Question category.

Comments

[Tom Fluker]

Mike,

Here comes a blatant commercial - delete if you like.

One thing I am finding as we are targeting that vary topic is that business units do NOT test for business continuity. IT tests DR and then hand things to operations. Too many times, that is where it gets dropped.

When Rita threatened Houston, our Payroll did not ship since traffic was so messed up. Everyone looked to me to ask what to do. Kind of an interesting challenge since all of the printers that supported that function were in Houston.

One result of all of that is the company I am working for now actually works with the business units to test business continuity. We have moved out of working with IT on DR to really test the end to end solution. Interesting results to say the least.

Tom

[Mike]

Tom,

You're absolutely right. The planning has to go beyone IT's DR plan. What's the point of having of functioning computer system if your factory can't produce, your stores are without power, etc. etc. As you say, you really have to test for an end to end solution.

Mike

[Charles]

Its not that surprising. People expect it, why I'm not sure. They should be thankful that their data was preserved. I can't count the number of businesses I've run into that don't even back up their data at all.

[Mike]

Charles,

I guess it is the old "if I don't think about it it won't happen" mind set which is always one of the great challenges of being in IT. Any thoughts on how to get more people engaged in thinking (and actually preparing) for business continuity beyond just IT's DR plans?

Mike