Solving Complex Business Problems Wicked Problems

At the risk of gross simplification we can state that business Problems as well as other
problems fall into two classes Complex (Wicked) Problems and Technical
Problems. Of the two Wicked problems are by their name and their nature most
difficult. The following pages contain the following some extracts on Social
Complexity as a contributory factor to Wicked Problems, A discussion of Wicked
Problems in the Context of Software, and finally a 1 page graphical representation of
Technical versus Wicked Problesm
Extract 1 is from the paper Wicked Problems and Social Complexity" by Jeff
Conklin http://cognexus.org/id42.htm
Contributory causes to wicked problems are Technical and Social Complexity.
Social complexity, refers to the number and diversity of players who are involved in a
project. The more parties involved in a collaboration, the more socially complex. The
more different those parties are, the more diverse, the more socially complex. The
fragmenting force of social complexity can make effective communication very
difficult. Social complexity requires new understandings, processes, and tools that are
attuned to the fundamentally social and conversational nature of work.
For example, in a joint project involving several companies and government agencies,
there was a prolonged struggle over the mission statement simply because of a
terminology difference: each sponsoring agency had their own term for the core
concept2, and to pick one term meant disenfranchising one of the agencies. This is a
very simple example of fragmentation of meaning. Social complexity means that a
project team works in a social network, a network of controllers and influencers
including individual stakeholders, other project teams, and other organizations. These
relationships, whether they are with direct stakeholders or those more peripherally
involved, must be included in the project. For it is not whether the project team comes
up with the right answer, but whose buy-in they have that really matters. To put it
more starkly, without being included in the thinking and decisionmaking process,
members of the social network may seek to undermine or even sabotage the project if
their needs are not considered. Social complexity can be can be understood and used
effectively, but it can be ignored only at great peril.
Wicked problems fragment the process of project work, especially when the problem
is misdiagnosed as tame. Wicked problems also fragment direction and mission if
you cant agree on what the problem is, how can you be aligned on a solution? Social
complexity fragments team identity the ideal of team unity is compromised by the
dynamics of competing interests and hidden agendas. The duality of design tends to
divide allegiances between requirements and implementation. Social complexity also
fragments meaning key terms and concepts are used in different ways by the
different stakeholders. Project teams are often geographically distributed, further
fragmenting relationships and communications. Participants in a modern project team
are pulled in a thousand different directions by the centrifugal forces of wicked
problems, social complexity, and technical complexity (see Figure 6).

Virtually all creative work is a process of design9. All problems call for designing a
solution. All projects are essentially designing something. Design, in both the
technical and artistic sense, is the process of creating something new e.g., a new car,
a strategic plan, a software program, a stickier web site, next years budget, a new
environmental policy.
The antidote for fragmentation is coherence. How, then, do we create coherence? In
organizations and project teams in situations where collaboration is the life blood of
success coherence amounts to shared understanding and shared commitment.
Shared understanding of meaning and context, and of the dimensions and issues in the
problem.Shared commitment to the processes of project work and to the emergent
solution matrix. Coherence means that stakeholders have shared meaning for key
terms and concepts, that they are clear about their role in the effort, that together they
have a shared understanding

1.

Wicked Problems
Our infrastructure is essentially developed. The easy problems have been solved. Designing
systems today is difficult because there is no consensus on what the problems are, let alone how to
resolve them. The year is 1973 and the topic is public policy. In the landmark article
Dilemmas in a General Theory of Planning[1], Horst Rittel and Melvin Webber observed that
there are a set of problems that cannot be resolved with traditional analytical approaches. They
labeled such problems Wicked Problems.
It seems that wicked problems are not unique to public policy. In a wonderful book published in
1990 called Wicked Problems, Righteous Solutions,[2] Peter DeGrace and Leslie Hulet Stahl
pointed out that many of the systems problems facing software developers have all the
characteristics of wicked problems. Judge for yourself.

Wicked Problems
Wicked problems, according to Horst and Webber, have ten characteristics:[1]
1. There is no definitive formulation of a wicked problem.
Formulating the problem and the solution are essentially the same thing. Each attempt at
creating a solution changes the understanding of the problem.
2. Wicked problems have no stopping rule.
Since you cannot define the problem, it is difficult to tell when it is resolved. The problem
solving process ends when resources are depleted, stakeholders loose interest or political
realities change.
3. Solutions to wicked problems are not true-or-false but good-or-bad.
Since there are no unambiguous criteria for deciding if the problem is resolved, getting all
stakeholders to agree that a resolution is good enough can be a challenge.
4. There is no immediate and no ultimate test of a solution to a wicked problem.
Solutions to wicked problems generate waves of consequences, and it is impossible to
know how all of the consequences will eventually play out.
5. Every implemented solution to a wicked problem has consequences.
Once the web site is published or the new customer service package goes live, you cant
take back what was on-line or revert to the former customer database.
6. Wicked problems do not have a well-described set of potential solutions.
Various stakeholders will have differing views of acceptable solutions. It is a matter of
judgment as to when enough potential solutions have emerged and which should be
pursued.
7.

Every wicked problem is essentially unique.

There are no classes of solutions that can be applied to a specific case. Part of the art of
dealing with wicked problems is the art of not knowing too early what type of solution to
apply.1

8. Every wicked problem can be considered a symptom of another problem.

A wicked problem is a set of interlocking issues and constraints which change over time,
embedded in a dynamic social context.
9. The causes of a wicked problem can be explained in numerous ways.
There are many stakeholders who will have various and changing ideas about what might
be a problem, what might be causing it, and how to resolve it.
10. The planner (designer) has no right to be wrong.
A scientist is expected to formulate hypothesis, which may or may not be supportable by
evidence. A designer doesnt have such a luxury, they are expected to get things right.

Recognizing Wicked Problems

What kind of problems are wicked problems? Here are some examples:
1. Locating a new freeway or homeless shelter.
2. Optimizing all the features on a new model car.
3. Deciding on the best way to re-engineer a business process.
Wicked problems arise when an organization must deal with something new, with change, and
when multiple stakeholders have different ideas about how the change should take place.
How might you identify a wicked problem? The thing to look for is divergence. If requirements
are volatile, constraints keep changing, stakeholders cant agree and the target is constantly
moving, in all likelihood, you are dealing with a wicked problem. If considerable time and effort
has been spent, but there isnt much to show for it, there is probably a wicked problem lurking
somewhere. If business process reengineering is involved, there is a good chance of encountering
a wicked problem.

Tame Problems
According to Rittel and Webber, the opposite of a wicked problem is a tame problem. Tame
problems may be quite complex, but the lend themselves to analysis and solution by known
techniques. A traditional linear processes is sufficient to produce a workable solution to a tame
problem in an acceptable period time, and it is clear when a solution has been reached.
It is possible, but not advisable, to pretend that a wicked problem is a tame problem. This makes
it easy to address the well-formulated problem with standard techniques. In time, however, the
wicked problem will surface as changed constraints, volatile requirements, or stakeholder
resistance. If a problem was truly a wicked problem in the first place, treating it like a tame
problem before it is actually tamed is a recipe for disaster.

How to Handle Wicked Problems

The most fundamental rule for handling wicked problems is that they must not be treated like
tame problems. To quote Rittel and Webber: The classical systems approach is based on the
assumption that a project can be organized into distinct phases: understand the problems,
gather information, synthesize information, work out solutions and the like. For wicked
problems, however, this type of scheme does not work. One cannot understand the problem
without knowing about its context; one cannot meaningfully search for information without the

orientation of a solution concept, one cannot first understand, then solve.[1]

The appropriate way to tackle wicked problems is to discuss them. Consensus emerges through
the process of laying out alternative understandings of the problem, competing interests, priorities
and constraints. The application of more formal analysis tools is impossible before the problem
can be articulated in a concise, agreed upon, well-bounded manner. In other words, the problem
must first be tamed.

Wicked Projects
Wicked projects arise when a project is organized to tackle a wicked problems as if it were a tame
problem. Another candidate for a wicked project is a Death March Project (as defined by
Yourdon in [3], this means a mission-critical project with less than half the time and resources
necessary to do the job). Think about it. Death March Projects are often caused when volatile
requirements, changing constraints and stakeholder disagreements meet up against immovable
deadlines. They are frequently a symptoms of management's unwillingness to tackle wicked
problems.
Failure to recognize wicked projects has given Software Development a bad name. A 1994
Standish Group Report[4] found, for example, that about a third of software development projects
get canceled and half do not meet their original cost projections. Some have taken this to indicate
that the state of software development is in disarray. However, it can also be read as strong
evidence that there are a large number of wicked software development projects out there, trying
to address wicked problems with the wrong approach.
A typical management reaction to a failed software development project is to conclude that the
organization is immature and to aim for more maturity. This usually means imposing more
requirements documentation, more analysis, more planning and tracking against the plan.
Managers feel that more use of the classic project management processes will avert future
disasters. If the failed project was addressing a tame problem, this approach will probably be
beneficial.
However, classic project management practices simply do not work for wicked projects. In fact,
referring again to Rittel and Webber, attempting to baseline requirements and then use an
analytical approach to reach a solution is a recipe for disaster with wicked problems. These
problems are resolved through discussion, consensus, iterations, and accepting change as a normal
part of the process.

How to Deal with a Wicked Project

Step 1: Recognize that this is a Wicked Project.
The first step to combating most systemic problems is to recognize them admit that they are
there. If the project has to satisfy stakeholders who do not agree on fundamental issues
surrounding the project, then it is definitely a wicked project. Its fine to say that there should be
an executive sponsor or that the customers should speak with one voice. What if they dont?
What if they pretend to agree, but deep down, they really dont? This is a wicked project and
everyone would be better off if it was recognized as such.
Step 2: See if the Project can be Tamed.
In recent Harvard Business Review[5], Robert Herbold gave an excellent example of taming a

wicked project. As new COO of Microsoft in 1994, he realized that he had to impose centralized
financial, purchasing and human resource systems on staunchly independent Microsoft business
units. Heres how he tamed the problem:
1.

Executive Support. Bill Gates made it clear to all divisions that anyone not
reporting results through the new system would be deemed to have no results at all.
Steve Ballmer told balking general managers: We define the measures. You put all
your creative juices into growing them.

2.

Clear problem Definition. All central systems were implemented with off-the-shelf
software which dumped data into a data warehouse. There were few system decisions
to be made beyond which package to use and how to configure it. This was
accomplished by a small group of technical and business experts because, according to
Herbold, you can get the job done in the time it takes a cross-functional committee to
decide on its goals.

3.

Separation of Concerns / Reduction of Stakeholders. Business units had webbased access to the data warehouse for each system, and they could get at the data
however they wanted to, but they did not have access to the packaged software. Thus
business unit managers had no stake in the packaged software, and where they did have
an interest, which was data access, they had a high degree of control and flexibility.
This had the effect of reducing the number of stakeholders for the central system to a
very few.

A project is not easily tamed at a low level, although a component-based architecture helps to
reduce stakeholders and thus tame the problem.
The main approach for taming a wicked project is to obtain appropriate executive support. This
means that the sponsor must recognize and resolve wicked problems. If the sponsor cannot
resolve conflicts and the customer does not speak with one voice, the problem has not been tamed.
Step 3. Use Adaptive Processes
Wicked problems are resolved through discussion, consensus, iterations, and accepting change as
a normal part of the process. There is nothing that difficult about an adaptive process. It has been
used successfully in World Class new product development organizations for decades. Most
public policy is developed this way. Most business policy is developed this way. Most marketing
is done with adaptive processes. All sophisticated process control systems use adaptive control
techniques.
There is nothing wrong with using adaptive processes to solve wicked software development
problems. In fact, it if the problem cannot be tamed, this is the only good choice.
There is intense pressure not to use adaptive processes for software development, because they are
not considered mature or predictable or controllable. In fact, empirical control techniques are
just a successful at controlling processes as defined control techniques, when used for the right
kind of problems. Since classic processes have such a poor track record in controlling wicked
projects, it is time to give adaptive processes a chance. If the project is a wicked one, the chances
of success will be significantly improved.

Scrum
Perhaps the best adaptive project management approach for wicked software development projects

is Scrum. This Rugby term was first used by Takeuchi and Nonaka in 1986[6] to describe the
way in which world class companies develop new products. Considering that a new product
development project is by its nature a wicked problem, the best new product development
processes present a good model for solving other wicked problems.
Ken Schwaber and Mike Beedle describe the use of Scrum for software development projects in
their book Agile Software Development with Scurm[7]. The important thing about Scrum is that it
forces incremental action which creates a basis for stakeholder dialog and project feedback. A
scrum project collects stakeholder input in a feature list called the Backlog. Each month, the
development team starts at the top of the Backlog and selects as many of the top priority features
as they think can develop in a month. The team is then left alone for a month, when the result is
demonstrated to the stakeholders. This provides a basis for rethinking the backlog features and
priorities. The stakeholders are allowed to modify and reprioritize the backlog, after which the
team selects its next months work from the top of the list .
Scrum provides a way for the development team to make regular progress even if the problem is
not well understood. At the same time, it provides a method for stake holders to discuss the
problem and reach consensus. It provides a way for solutions to emerge, as is necessary for the
resolution of wicked problems.
At the same time, the Scrum process provides a high degree of predictability. Each line on the
Backlog is estimated, and the estimates are added to create an overall project completion estimate.
After three months, a graph of the Backlog estimate against time is a highly reliable indicator of
the projects progress and estimated completion date. Features may be added or subtracted from
the Backlog monthly to adjust for constraints as well as changing stakeholder interests. The trend
in the Backlog estimate is a reliable indication of whether the project is converging or diverging.

Conclusion
If software development projects have not responded well to traditional project management
practices, the fault may lie in the attempt to apply inappropriate methodologies to software
development projects. When projects must deal with conflicts in stakeholder requirements and
changes in management constraints, an adaptive process is far more likely to succeed than
traditional methodologies. At minimum, adaptive project management practices will quickly
determine if a project will converge on a solution, and thus provides actionable data for project
portfolio management.
_____________________________

1.
2.
3.
4.
5.
6.

Rittel, H., and M. Webber; Dilemmas in a General Theory of Planning pp 155-169,

Policy Sciences, Vol. 4, Elsevier Scientific Publishing Company, Inc., Amsterdam, 1973.
DeGrace, P., and L. H. Stahl. Wicked Problems, Righteous Solutions. Prentice Hall,
Yourdon Press, 1990.
Yourdon, Edward; Death March Projects, the Complete Software Developer's Guide to
Surviving Mission Impossible Projects. Prentice Hall, 1999, 1997
The Standish Group. Charting the Sea of Information Technology Chaos. The
Standish Group International, 1994.
Herbold, Robert J.; Inside Microsoft: Balancing Creativity and Discipline, Harvard
Business Review, January 2002.
Takeuchi, H., and I. Nonaka. The New New Product Development Game Harvard
Business Review, January-February 1986

7. Schwaber, Ken, and Mike Beedle. Agile Software Development with SCRUM. Prentice Hall,
2001