December 7, 2018

Why we suck at "solving wicked problems"

Problems: Wicked, complex, intractable, or adaptive  … whatever we choose to call them, we seem to suck at solving them and we often get profoundly overwhelmed and stuck.  I believe the reason for this is hidden in the language I used in the title and previous sentence  (… and no it's not the adjectives ... )

Two words: “problems” and “solving” …

I’ve sat in many workshops where well-meaning people try to tackle what they invariably end up calling wicked problems.  From transforming organisations, to health and safety to larger issues like youth unemployment to wildlife crime to systemic poverty, we keep circling around and around these issues and never seem to make much progress.  Here's how I see it:

What we think we need, but often don't ...

  • problem-solving skills (and a solution mindset)
  • new and different answers
  • expert advice and proven practices (best practices from other areas where they've had some success)
  • clarity: e.g. clearly defined problem statements (preferably at the level of root causes and drivers) & clear goals and focus areas
  • a clear vision of the ideal end state
  • alignment (of stakeholders etc)
  • measurable outcomes
  • a lot of money and the very best technology

What we actually need ... (but often avoid)

  1. Sense-making skills and pattern intelligence

We are always dealing with different contexts.  Looking for example at wildlife crime, there are aspects that are quite ordered and predictable for example obtaining a permit to export lion bones legally; or drafting a new quota policy.  However, there are other aspects that are profoundly complex e.g social dynamics and perceptions around social justice, subsistence and conservation.  This means that something that should be straightforward, like implementing a quota can have profound unintended consequences.  Before we jump into linear problem-solving, I believe we need to do sense-making. The framework I prefer to use is Dave Snowden’s Cynefin framework as it enables me to distinguish between ordered aspects (where I can find root causes; do problem-solving, involve experts;  apply best practice; manage change etc) and complex aspects where I am dealing with emergent patterns with no clear linear causality.  Here,  I need to engage with the system to gain an understanding of how things are connected, run safe-to-fail experiments, and learn and adapt as I go.  If we apply linear thinking and ordered approaches such as root cause analysis; traditional scenario planning and best practices to complex problems we invariable end up making things worse.

I find Glenda Eoyang’s framing very useful in this context: we need to learn to see patterns not problems – or acquire pattern intelligence.  If I look at wildlife crime as a problem to solve, I can very quickly become overwhelmed.  To solve wildlife crime I need to (among others) change consumption habits of entire cultures in order to reduce demand;  I need to disrupt organised criminal networks and bring kingpins to justice; on the supply side I need to solve social justice issues; systemic poverty, unemployment and corruption;  and … the list goes on.

A way to become unstuck and feel less overwhelmed is to look at wildlife crime as a pattern, with multiple interrelated influences or modulators and therefore multiple potential entry points.  Instead of trying to find solutions, I focus on shifting the pattern through multiple small interventions, nudges, or experiments. Every time I interact with the pattern it changes, new opportunities open up and I learn more about the system.  A metaphor I find useful in this context is James Carse’s finite and infinite games. If we think about finding solutions, we see it as a finite game with an outcome, a winner, and a loser.  Here we tend to think that if we can only find that one breakthrough idea that will unlock everything we can permanently solve the problem e.g we can put an end to poverty forever.   If however we see think about an infinite game, where the rules, boundaries, and players keep changing and where there is no clear outcome or winner we see things differently.  The purpose becomes to keep the game going, and becoming aware and adaptive enough that you become the one creating the shifts that other players must respond to, not the other way around.  If we see poverty or wildlife crime as an infinite game or ever-shifting pattern that we can influence, the way we engage with it will be very different, and I believe much more effective.

2. New and different questions (we live in the world our questions create) -  curiosity and different questions

 “We live in worlds our questions create.” - David Cooperider

I think most of us would say we live in worlds our answers create.    Invariably in think tanks and projects, we are looking for new answers.  If we can just get the right experts in the room, or enough big data or employ AI solutions, we can find the answers we need to solve the problem.  I believe if we start from a position of curiosity and finding new and different questions to ask, it would be much more useful.  Answers are based on our current questions which are based on current assumptions or knowledge.  In volatile and fast-changing contexts, what we think we know today may not be true tomorrow.  Or what is valid in one context, may not be valid in another.  Discovering new questions, broadening our perspective, and being curious (vs judgmental) will get us much farther than looking for answers.

We also need to be open to new and different sources of knowledge: it is often the people we disregard, those with boots on the ground who have the contextual knowledge we need.  If we combine this local knowledge with oblique or naïve perspectives i.e. people from other disciplines or with expertise in adjacent fields the chance that radically different insight will emerge is much greater than when we get a room full of "experts" together.

3. Ambiguity and nuance vs certainty and clarity

Our need for certainty and clarity often leads us to oversimplify things.  Issues become black and white, people are either good or evil e.g. poachers become evil criminals that we can kill if we encounter them vs the more nuanced view that a poacher may be a desperate human being, trapped in poverty, needing to care for an extended family with no education and no prospects of getting a job.  These simplistic binary views cause us to implement simplistic solutions that end up making things worse.  For example, deploying the army to assist rangers with anti-poaching activities alienates the communities that we expect to be our allies in the fight against wildlife crime. Or making assumptions about the "demand-side" of the wildlife crime problem - that we need to change cultural traditions and behaviour that we view as silly or barbaric.  The solution is to have a communication strategy where we target people on the other side of the world with messages aimed at changing their cultural beliefs and behaviours and making them more like ours – is this not another form of colonialism?  Who decides what the “right” cultural values are for a nation or people group?

We can also choose to see the root cause of all of this to be economic: it's "all about the money" - so if we can find the right incentives we can manage the market.  Counter greed with greed.  No unintended consequences there I'm sure ...

We need to get to a place where we can see all of these interacting influences; all of the tensions and contradicting values at play; all of the nuance and be able to contain and work within that ambiguity.   It is only when we are able to hold the tension, see the “AND” vs the “OR” that we will be able to engage with the system in a constructive way.

4. Focus on the present and what is possible from where we are (vs working backwards from an ideal future state)

An ideal future state is usually unattainable and described from a limited perspective.  For example, for conservationists the ideal state may look like a world where wildlife crime has been completely eradicated; where animal populations recover and thrive and no one ever feels the need to use a tonic made of rhino horn or lion bones again.  I’d wager that that is not the ideal future state that a traditional healer or even an economist would describe.   Defining an ideal state and designing solutions to close the gap is a trap.  While we need a clear sense of direction and purpose, and in the shorter term goals and objectives, we need to be grounded fully in the present; meeting the system where it is and evolving or nudging it towards adjacent possibles (i.e. potentially beneficial patterns that already exist in the system and that isn’t too far away from the current state).  This is a much more pragmatic and sustainable (and often more cost-effective) strategy.    To use an analogy: it’s like crossing a river on stepping stones i.e. I know my direction is towards the other side, but I have no specific spot in mind – I follow the path as it emerges: vs designing and building a bridge.

5. Seek coherence not alignment

We often fall into the trap of wanting alignment between different stakeholders, projects, contexts etc.  Alignment to the same vision or goal; aligned initiatives; aligned values.  But in a world where we are dealing with competing values and tensions, we need diversity to respond to diversity.  Alignment usually requires consensus, and it is valuable in ordered contexts where we are working towards known outcomes with fixed scopes and budgets; where we need coordination and collaboration.  Here if we have no alignment it is like a relay race where athletes aren’t ready to accept the baton from each other or a football game where the teams try to score on the wrong side of the field.

When dealing with complex problems, consensus is often impossible as we are dealing with competing values, paradoxes, and multiple inter-related causal influences.  Experts would often disagree, and be able to offer valid evidence that support these competing views.  Here consensus and alignment are dangerously limiting: so we need to think about enabling coherence.  This is key to being able to implement portfolios of safe-to-fail experiments where some may contradict others.   Having a coherent sense of direction with clear boundaries to create "safety guardrails" allows us to maintain local diversity, implement potentially conflicting experiments and engage with the system with the common objective of learning and evolving together.  If we acknowledge that we are playing an infinite game i.e. there are no winners or losers;  we can experiment, learn and adapt together and hopefully shift the pattern to one that is more beneficial for all.

One final note:

Money and technology are hugely valuable resources:  they are certainly necessary but they are not sufficient.  Simply throwing more money and/or more advanced technology at a problem will not make it go away.  We need to fundamentally change our thinking paradigm and approach things in context-appropriate ways, otherwise, we will never move the needle on these so-called wicked problems.

5 comments on “Why we suck at "solving wicked problems"”

    1. Thanks Brett. Yes I love Harold’s work and there are a lot of synergies. One of the things I value most is a diverse knowledge network that can support but also challenge. We can so easily get stuck in a comfortable echo chamber!

    1. Thank you Ron! I will definitely have a look at those links you’ve posted. I’ve seen quite a few references to Donatella Meadows’ work recently so it seems the time is right for a deeper dive :).

  1. Lindblom identified "muddling through" or "successive limited comparisons" as a way to manage wicked problems in 1959: some excerpts from the paper and a citation:

    Rational-Comprehensive (Root)
    1. Clarification of values or objectives distinct from and usually
    prerequisite to empirical analysis of alternative policies
    2. Policy-formulation is therefore approached through means-ends
    analysis: first the ends are isolated and then the means to
    achieve them are sought.
    3. The test of a "good" policy is that it can be shown to be the
    most appropriate means to desired ends.
    4. Analysisis comprehensive; every important relevant factor is
    taken into account.
    5. Theory is often heavily relied upon.

    Successive Limited Comparison (Branch)

    1. Selection of value goals and empirical analysis of the needed
    action are not distinct from one another but are closely
    entertwined.
    2. Since means and ends are not distinct, means-end analysis is
    often inappropriate or limited.
    3. The test of a "good policy" is typically that various analysts
    find themselves directly agreeing on a policy (without their
    agreeing that it is the most appropriate means to an agreed
    objective).
    4. Analysis is drastically limited: Important possible outcomes,
    alternative potential policies, and possible outcomes are all
    neglected.
    5. A succession of comparisons greatly reduces or eliminates
    reliance on theory.

    Policy is not made once and for all; it is made and re-made
    endlessly. Policy-making is a process of successive approximations to
    some desired objectives in which what is desired itself continues to
    change under reconsideration.

    A wise policy-maker consequently expects that his policies will
    achieve only part of what he hopes and at the same time will product
    unanticipated consequences that he would have preferred to avoid. If
    he proceeds through a *succession* of incremental changes, he avoids
    serious lasting mistakes in several ways.
    1. Past sequences of policy steps have given him knowledge about
    the probably consequences of further similar steps.
    2. He need not attempt big jumps toward his goals that would
    require prediction beyond his or anyone else's knowledge,
    because he never expects his policy to be a final resolution of
    a problem. His decision is only one step, one that if successful
    can quickly be followed by another.
    3. He is in effect able to test his previous predictions as he moves on to
    each further step.
    4. He often can remedy a past error fairly quickly--more quickly
    that if policy proceeded through more distinct steps widely
    spaced in time.

    From The Science of "Muddling Through" by Charles E. Lindblom
    in Public Administration Review, Vol. 19, No. 2 (Spring, 1959), pp. 79-88
    http://www.jstor.org/stable/973677

Leave a Reply

Your email address will not be published. Required fields are marked *