While few managers revisit their decisions to see how they turned out, when they do, the picture is often not very pretty. For example, Wharton Professor Ian MacMillan, co-academic director of Wharton's Strategic Thinking and Management for Competitive Advantage executive program, studied planned versus actual venture capital investments. In reviewing 50 plans from 12 successful firms, MacMillan tracked the disparity (at the 5-year mark) between projected and actual results. Actual results revealed half the revenues, double the costs, double the investment, and high failure rates. Absent a crystal ball, companies can and must do better at making decisions, said MacMillan at an Impact Conference on decision making in May sponsored by Wharton's Mack Center for Technological Innovation.

Deciding Under Uncertainty

An environment of uncertainty is a particular challenge for decision making, but there are tools designed to limit failures in such environments. In particular, MacMillan and a colleague developed Discovery Driven Planning (DDP), a method that allows companies to take baby steps, assess at each step, and either to stop the project, proceed, or proceed in a different direction. "The core problem," said MacMillan, "is that the assumption-to-knowledge ratio is huge." As a result, companies need to plan differently. In essence, they must see the process as "planning to learn instead of planning to accomplish."

"DDP is not rocket science," MacMillan joked, but it was adapted from rocket scientists. In fact, the theory was born when President Kennedy gave NASA the ultimate challenge: put a man on the moon, and bring him back safely. Each step in this challenge required thinking and rethinking. This eventually paved the way for the realization of Kennedy's dream and broke new ground in learning.

The six key disciplines in DDP are:

1. Framing a worthwhile challenge
2. Recognizing competitive market metrics
3. Understanding the required deliverables
4. Identifying and documenting assumptions
5. Identifying major checkpoints
6. Testing assumptions at check points

According to MacMillan, the majority of firms do many of these steps, but most fail to do all of them, particularly the last three. "No assumption should go untested," said MacMillan. Ultimately, he said, DDP forces people to spend their imaginations before spending real money.

Knowing When To Kill a Project

Ron Pierantozzi, director of New Business Development in the Corporate Development Office at Air Products, said he finds DDP particularly useful when facing uncertainty in new markets or when there is high uncertainty internally (i.e., it is not clear where leadership stands). The projects in his domain generally have a 7- to 10-year horizon and are considered feasible only if they can produce at least a $500 million return for the company.

Pierantozzi introduced DDP in a few pilot studies. "When we used DDP, we discovered that conversations would go differently," he said. "We were thinking differently about new business, and DDP vocabulary, such as the ‘reverse income statement,' is now part of our lexicon." Air Products has used DDP with its larger businesses, in research programs where there is high uncertainty, and when entering into new geographies with existing businesses.

In one example, his team used it with a potential licensing agreement in a new coating market. The numbers looked exciting at first, he said, "but as we went through the DDP process, the market size kept diminishing." Through the DDP process "we learned that the only way this could work was if Air Products captured 100 percent of the market." In the end, they killed the project, saving Air Products a significant amount of money on licensing fees. "The process empowers team members to kill a project," he said.

Indeed. MacMillan noted that roughly 6 out of 10 potential plans that go through the DDP process end up getting scrapped. The DDP process does not, however, just offer a "Go, No-Go" outcome, said MacMillan. By stopping and analyzing at checkpoints along the way, potential actions could include redirection, scaling up or down, speeding up or slowing down, spinning off a product, or creating a joint venture. Even failures are not failures, said MacMillan. "You don't just turn off the lights and go home." Smart companies will document their learning during these processes, he said, amassing a dynamic knowledge base.

Concluded Pierantozzi, "typically, when you go to management and say you don't know a lot, that's career suicide." But uncertainty is a reality of business. "DDP allows you to identify what you don't know (assumptions) and test your assumptions in a rigorous fashion," he said. When the potential pitfalls are clearly spelled out, said Pierantozzi, "that's when management goes into problem-solving mode."

The Right Tools for the Job

Paul Schoemaker, research director of the Mack Center and author of Winning Decisions, said uncertain environments require different approaches to decision making. In low-risk/low-reward environments, traditional tools for analysis might work well. These include extrapolative forecasting, Net Present Value analysis, decision trees, Bayesian updating, expected utility theory, Monte Carlo simulations, portfolio optimization, and stochastic modeling. But in environments with higher risk and uncertainty, it's best to chose from newer and more flexible tools, such as influence diagrams, scenario planning, real options analysis, hedging/derivatives, total risk management, dynamic monitoring, and learning models or simulations.

Schoemaker, co-academic director of Wharton's Critical Thinking: Real-World, Real-Time Decisions executive education program, also stressed the importance of framing. For example, in discussing the Challenger Disaster as a classic case of poor decision making, he noted that the way questions were framed affected the decisions that were made. One of the key pieces of data NASA engineers consulted when deciding whether or not to launch was the number of O-ring incidents and the temperatures at which they occurred. The chart reflecting this information showed several scattered points, but no clear pattern. This narrow view helped make the "Go" decision, said Schoemaker. If engineers had framed the problem differently — for example, had they looked at incident-free flights, he said, they would have seen a clear trend that showed when it was safe to fly.

Schoemaker said that surveys have found that decision makers say they intend to spend roughly 20 percent of their time framing the problem, but in practice they spend only about 5 percent of their time on this critical exercise. Key questions to ask during this phase include, "Are you solving the right problem?" "What are the hidden assumptions?" "What are our success criteria?" and "Who else should be involved?"

Improving Decision Making

Two executive education programs are designed specifically to help managers cut through the sea of data and conflicting expert opinions to make better decisions. The Critical Thinking: Real-World, Real-Time Decisions program recognizes that more and more managers are asked to make decisions in an atmosphere of increasing time pressure and uncertainty. Topics such as framing the problem, gauging uncertainty, scenario planning, leadership in decision making, decision making under pressure, group decision making, and forecasting all serve to help participants reframe issues and look clearly at alternative decisions.

The second program, Strategic Thinking and Management for Competitive Advantage, helps managers with the challenges of building strategy. It posits that too many companies try to be everything to everybody and often fail to identify and sustain their competitive advantage. One of the topics addressed in this program is strategy under uncertainty. This has become a hot topic in recent years, in part because so many of the options for growth — alliances, acquisitions, and internal corporate ventures — are based on uncertainty.

Related Courses

• Strategic Thinking and Management for Competitive Advantage
• Critical Thinking: Real-World, Real-Time Decisions

• Wharton Executive Education
• The Wharton School

This month's articles:

• Thought Leaders
  Leaders of Wharton's Baker Retail Initiative discuss transformations in retail.
  
  
• Fellows
  How does Alessi turn household utensils into objects of art?
  
  
• Conferences
  How can you make better decisions under uncertainty? A Wharton conference offers insights.
  
  
• Advanced Management Program
  What does poetry have to do with business? A lot more than you may think.
  
  
• Education à la Carte
  Improve your own shelf life.