Stop calling software development complex
Andrew Blain, Managing Director
There seems to be a misconception in the agile community that the discipline of software development can be classified as complex under David Snowden’s Cynefin Framework. This is leading to what I feel are some uncomfortable tropes regarding approaches to planning, estimation and the reasonable need for a level of predictability.
From my understanding of Cynefin, the discipline of software development cannot be classified as ‘complex’. Used correctly, Cynefin is a framework that helps you to make sense of a domain and to evolve fit for purpose systems with contextually appropriate constraints. It is not designed for use as a tool to support blanket statements like ‘software development is a complex task’; ‘software development is inherently unpredictable’; or ‘estimation is an anti-pattern in software development’.
Read about how Elabor8 improved the speed and predictability of delivery teams by 300% at an online risk trading company
For starters, according to my understanding, the transition between the domains in Cynefin is as important as the domains themselves. The process of human innovation simplifies complexity over time until such point that something previously considered impossible (e.g. writing this blog on a regional train using a cloud based word processor with offline storage) is a base expectation (and your author gets frustrated when we hit a telecommunications blackspot):
- In 1996, before Australia’s CSIRO invented WiFi (1), technology companies and standards groups around the world were all trying to tackle the problem of high-speed wireless connectivity. The engineering standards body, the IEEE, drafted the 802.11 wireless standard in 1990 and companies spent years on developing the building blocks of the technology. An approach driven by a CSIRO radio astrophysicist named John O’Sullivan, which combined several existing 802.11 related developments, became the technology of choice. Nowadays if you want to connect wirelessly to a network or a device you can choose from a range of commodity solutions built around WiFi technology which are widely enough known and implemented to mean they only require a small degree of technical knowledge. This is an example of the transition of a problem from the complex towards the obvious domain, where it will most likely remain until such point as a newer, better technology emerges or a major security flaw presents in the WiFi specification (which would drop the entire system off the cliff into chaos).
Secondly, a problem shouldn’t be considered from a single perspective under Cynefin. The same problem can present in different Cynefin domains according to the perspective of the actors in the system and the timing of the event.
- From the perspective of a novice in networking, setting up a home WiFi network is a daunting prospect that will often require the input of an expert in the form of a friend, family member or call centre employee.
- To the experienced friend or family member willing to offer a favour, configuring a home WiFi networks on one of the many available devices that are packaged up with an internet connection is relatively straight-forward.
- Any reader who has setup a network as a favour will know that the nature of the problem changes when the recipient of said favour experiences a drop out at a crucial time (e.g. right before an assignment is due to be submitted). Providing technical support over the phone to a novice who is under the stress of a deadline is an underrated skill.
- Furthermore, to a security professional with detailed knowledge of the threats in wireless connectivity, setting up a home network is a different challenge — and one that requires careful selection of a technology solution, continuous monitoring of advisories, and regular, proactive interventions.
Due to this temporal sensitivity and the importance of perspectives, Cynefin is a sensemaking framework rather than a classification framework. The Cynefin framework can be used to help understand the nature of problems and how they change over time, but it is not designed to be used as a quadrant that you can classify things into.
The transition of software development through the Cynefin domains. Base image courtesy Cognitive Edge (www.cognitive-edge.com)
It follows that the approach to planning and estimation should be contextually appropriate. Traditional approaches to estimation are an anti-pattern for the duration that a problem resides in the complex domain. Here we are better suited to enabling constraints like time-boxed experimentation.
However, once there is experience implementing a pattern or solution within a domain, we can estimate to a reasonable degree of accuracy via a range of techniques. It is also possible to fit known mathematical distributions to our delivery data to forecast the future and determine the approximate likelihood of hitting a date. The variance between estimates and actuals tends to relate to the handoffs and dependencies in delivery rather than the actual activities, and the complexity in software usually resides in the social systems.
Note: asking an experienced developer to estimate how long an unexperienced developer will take to complete a task is a bit silly unless they’re going to be pairing on it and have built some learning slack into the estimate.
Instead of saying ‘software development is a complex task’ we should say ‘aspects of software development are complex and don’t lend themselves to traditional approaches to planning. Instead of saying ‘software development is inherently unpredictable’ we should say ‘the systems in which software developers operate have a tendency to reduce predictability’. Instead of saying ‘estimation is an anti-pattern in software development’ we should say that ‘estimates are an anti-pattern for some aspects of software development’.
We should always offer solutions as well — some level of predictability is a very reasonable customer expectation. Fit-for-purpose system design should incorporate fit-for-purpose models for planning and estimation.
The Cynefin Framework has been invaluable as a tool for the agile community to communicate the changing nature of problems, the accelerated rate of innovation, and hence the need for more iterative and evolutionary approaches. We need to be careful as a community that we don’t misuse it and paint everything that we do in software development as some sort of dark art.
Unpredictability is a system condition, not a domain condition.
---
Originally published in Elabor8 Insights, October 2018
---
(1) Of additional interest is the multiple perspectives on who ‘invented’ the WiFi solution — CSIRO’s role in bringing WiFi to the world was formalised in the patent courts, to the victor the spoils:
http://www.abc.net.au/catalyst/stories/2708730.htm
