Planning for the unthinkable: IT contingencies

The information age has made possible many new computing- and telecommunications-based activities and services that were in the realm of science fiction only a decade or two ago. Today computers are at the heart of every aspect of business endeavor. Computers, and the networks that connect them, have made it possible to do countless processes faster and easier than ever before, as well as making it possible to do new things that were simply beyond our reach previously. But along with all the good, at least one old problem has been seriously exacerbated. Dependence upon information created by computers and networks is like dependence on foreign oil. What happens when access is cut off? Modem business can no longer function at all without its information technology (IT) infrastructure.

WHEN DISASTER STRIKES

This technological dependency is serious business. Virtually every organization using computing and networking technology needs to consider what might happen if its IT capabilities were suddenly and without warning wiped out. One only needs to reflect back to the eruption of Mt. St. Helens, or the bombings of the World Trade Center in New York and the Federal Building in Oklahoma City, or the recent earthquakes around San Francisco, or the hurricanes on the Gulf Coast (not to mention tornadoes, floods, fires, computer viruses, and disgruntled employees) to appreciate the risks involved. It is clear that key portions of a corporation's information systems could disappear literally over night.

We are talking about events with an extremely low likelihood of occurrence that could potentially result in huge, perhaps fatal, losses for the firm. These scenarios have always driven executives right up the wall. How much money should management spend to protect a company against something like this, an unthinkable situation that is very unlikely to happen, but that could ruin the company if it did?

It is a tough question in general, and an extremely tough one for the executives responsible for a company's IT function. This is because the IT function is different from other business functions. If a company has a manufacturing plant that is wiped out by a hurricane or destroyed by a flood, chances are that insurance will pay for the damage, and the company can get on with its business, perhaps somewhat degraded but still viable. However, if a firm's data center is destroyed by the same event, insurance will probably pay for the facilities lost (and maybe for some business interruption losses). But without its specific, interdependent, and often unique operating systems, hardware configurations, applications systems, networks, and databases, the firm will not be able to conduct its business. The information needed for operational, managerial, and strategic decision-making throughout the firm would be irrevocably inaccessible. Checkmate! And the match would be over.

DISASTER RECOVERY PLANNING

Contingency planning to help avoid or recover from catastrophic situations has been a nagging issue since the beginning of the computer era. But executives often do not appreciate the wisdom of budgeting large sums of money for disaster-recovery planning and testing every year to protect against events that are almost certainly not going to happen. It can seem a little far-fetched to worry about (say) a meteor or jumbo-jet falling out of the sky and taking out the data center, or a terrorist with an Uzi in the lobby, or whatever. And situations such as potential Y2K problems have not helped to change opinions in this area.

The fact is that it is difficult to make these choices. But this is still a very serious matter. The undeniably negative consequences of catastrophic IT loss will undoubtedly sober those executives who stop long enough to think through the issues. Probably the easiest thing to do is to gamble that nothing bad will happen. Clearly, however, denial is not prudent management, and if the worst case did come to pass, such denial would be considered irresponsible at best and criminal at worst.

BEST PRACTICES IN CONTINGENCY PLANNING

There are techniques (called "best practices") that provide guidelines and standards for managing IT Some of the practices deal with planning for (and avoiding to the extent possible) disasters. Understanding these practices is now critical because IT disaster-recovery planning has become a crucial contingency-management issue. Because technology today is both ubiquitous and thoroughly essential to the management and operation of modem businesses, planning for the unthinkable is more important than ever before in IT management.

IT contingency-planning issues that management should address include risk analysis, establishing alternative physical facilities for use in an emergency, and planning for data (and software) recovery. These basic thrusts are focused further with strategies for dealing with centralized and decentralized computing, as well as contingency planning for end-user computing requirements and computer networking. As part of the planning process, a corporation must determine which of these key facilities is truly essential to providing critical managerial and operational capabilities to keep the organization functioning at an acceptable level during an IT crisis. Planning should then concentrate on providing these critical IT capabilities and ignoring the less critical. The contingency planning process includes formulating, documenting, and communicating complex action plans to be followed in the event of an emergency. For example, it will be necessary to move staff, backup computer tapes, and maybe even some equipment to a new location out of harm's way where the critical IT functions can be recreated and reestablished quickly and in safety.

TESTING THE PLAN

Of course, it is not enough just to plan what to do in disastrous situations. Contingency planners must focus upon the key decisions that will have to be made when a disaster strikes, including which employees will do what and when. Key employees (organized into highly focused teams) will be responsible for executing several series of key tasks to effect a recovery.

Contingency planners must also plan for the mobilization of disaster recovery teams during an emergency. By definition, the recovery cannot be undertaken at an already-destroyed facility and, in an emergency, the recovery teams must be able to set up shop efficiently and effectively at some predetermined remote location. Having to move personnel, documentation, data, and software offsite greatly complicates the recovery process. Because of this complexity, it is necessary to ensure that the plan is workable and everyone involved knows what to do in the event of an IT emergency. This preparation is accomplished by staging unannounced, mock disasters periodically to assess the completeness and effectiveness of the plan and the ability of the participants to execute it as intended. The disaster recovery plan should be tested annually and the plan updated and improved based upon the problems discovered in each round of testing.

CONCLUSION

Developing and testing an IT contingency plan is an expensive proposition. It is little wonder that executives tend to balk at the prospect of paying. But there is also little doubt that such planning is needed and that it is of little use if it is not kept sharp and up-to-date.

To conclude with an anecdote, the IT management for a major U.S. corporation tried to convince executive management to fund a disaster recovery function for the company. After IT management had spent years presenting ideas and rationales in various proposals to the executive management without success, the directive to proceed came streaking down the chain of command with a sense of real urgency. One of the members of the Board of Directors for this company had a nightmare that the company's computing facilities had been destroyed in a massive fire! He awoke in a cold sweat, and IT management finally got the funding for its IT disaster-recovery function.

Dr. Charles K. Davis is an associate professor in the Cameron School of Business at the University of St. Thomas in Houston, Texas, and is a past Chapter President of Phi Kappa Phi. He has taught previously at the University of Houston and held analyst and management positions with IBM, Chase Manhattan Bank, Occidental Petroleum, Pullman Incorporated, and Deloitte & Touche.

Copyright National Forum: Phi Kappa Phi Journal Winter 2001
Provided by ProQuest Information and Learning Company. All rights Reserved