Black Swans in IT

IT has always been a risk-filled enterprise. That's because software development is a first-of-a-kind undertaking. Programmers write programs to make software and hardware systems function in new ways. Often, the novelty of the development effort is small, requiring minor adjustments to existing programs. In this case, development risk is generally low. To the extent that the programming entails exploring truly new territory, then development risk goes up.

Programmers and systems analysts have been aware of this aspect of development risk for decades. However, in an intriguing 2007 book, titled The Black Swans, Nassim Nicholas Taleb suggests that with our single-minded focus on the traditional perspective on development risk, we are ignoring a category of risk whose consequences can be devastating -- what he calls the black swans.

The idea of black swans arose among English philosophers in their discussion of David Hume's problem of induction. The induction problem holds that even if you observe 1,000 white swans, you cannot say with certainty that the next swan you encounter will be white. (As a matter of fact, black swans were unknown until their discovery in Australia in 1790.)

Taleb uses black swans as a metaphor for totally unanticipated high-impact events. DoD managers know them as unk-unks, i.e., unknown-unknowns. He points out that when you look at problems individuals, organizations, and societies face from a historical perspective, those with greatest consequences are the total surprises. The 9/11 attacks and the global financial crisis of 2008 are recent examples.

When applied to IT undertakings, we find that in certain respects, IT efforts are well-prepared to deal with black swan events, while in others they are not. They are well-prepared in the sense that all capable IT organizations have established disaster-recovery plans to deal with catastrophic challenges to their operations. By having cool, warm, and/or hot back-up sites, IT enterprises are able to handle disasters that bring down their data processing and communication capabilities. Thus they are able to handle the consequences of power outages, earthquakes, fires, hurricanes, and other natural disasters. As the 9/11 attacks showed, they can even withstand horrific terrorist onslaughts.

However, in the day-to-day pursuit of IT projects, IT organizations do not generally equip themselves to deal with black swan events. Their risk management efforts are largely directed at catching problems through testing and resolving them by fixing bugs. Seldom do they explicitly take into account broader forces that can jeopardize IT projects, e.g., funding cutbacks, changes of management, technology changes that render their solutions obsolete, and other things of this ilk. These are the forces that are likely to produce black swans in IT. The fact that most IT projects gain funding support only after project champions promise to do ten months of work in six months, and promise to deliver more functionality than is possible to deliver, increases the negative impacts that black swan events can have on IT enterprises and their projects.

These are uncertain times. For the first time since the turmoil of the Great Depression, Americans recognize that when bad things happen, things do not always turn out for the best. It behooves all organizations -- including IT shops -- to recognize that even though black swans are rare-occurrence events that are not predictable, they need to be taken seriously. What may have been viewed as recoverable setbacks one or two years ago, may yield catastrophic consequences today.