Tags
assurance processes, external auditor, financial risk, Internal Audit, internal audit function, internal auditor, internal auditors
One of the greatest pleasures I get as an auditor is working in a cross–disciplinary way across my client organisations. This means I can be a marketer, IT person, HR person, finance etc. The way I do this is not to be an expert in each area, but to bring my professional expertise of being an auditor to each of these disciplines and areas of my client organisation. I do this primarily through being a qualified internal auditor (not chartered accountant – it’s not the same), but also through being multidisciplinary myself, being a chartered accountant, holding a generalist MBA, also being qualified in risk management and IT audit.
I mention cross disciplinarity because as an auditor you can see this playing out in different professional areas of the business. So as a recent example, IT professionals have now discovered ‘agile’ systems development and also my international development programme management colleagues have discovered adaptive programming. The two are quite similar.
It’s difficult to find a good working definition of agile so I will attempt to define how I see it. For a good paper see US Govt: http://www.gao.gov/assets/600/593091.pdf Both agile and adaptive development to my mind have similar traits. They adopt short windows of work incrementally, are close to customers and beneficiaries, locate success in meeting customer and beneficiary needs and have less burdensome documentation, and frequently change direction. In other words it is an incremental and iterative, rather than a linear and process oriented way of delivering projects.
Agile is a process through which it is recognised that software systems need space to react, move, develop, iterate and incrementally develop. It works on the idea that most systems are still valuable with 80% of the functionality specified and do not need to be perfect. So instead of a linear process of specifying needs, then building them, testing them and releasing them, it provides for iterative loops until a good enough system is developed (i.e the ‘technical debt’ is paid off – the gap between the system and users’ requirements).
Adaptive programming is quite similar. In international development the variables creating the ‘wicked problems’ preventing development are too many and numerous to calculate in advance with any reliability. So why not try something, then adapt it as you go along, and, once working, scale it up? Most projects are not linear, so why not be upfront about it recognise it.
So the common challenge in these two approaches is control. Control because the way organisations control things is through management approval, normally on a hierarchical and linear basis, of a set plan. This plan is then prioritised and resourced and the party that is approved to deliver it has a set of inputs (resources) from which processes to deliver outputs and then ultimately outcomes, related to the original objectives, are delivered. Variances to budgets, processes, outputs and outcomes are then measured and value for money and success are then delivered.
This command and control process does not work well in the context of adaptive or agile programming. Programmes are not well understood at commencement; the starting point varies considerably from the potential range of end points. Variances provide poor, if any, indicators of performance; value for money is extremely hard to judge until the final completion of the programme.
So is adaptive or agile work simply poorly controlled or does it recognise our human nature and allow for complex problems to be solved? As an auditor, but also a socially scientific auditor, I am torn. My professional training tells me that control should be established, that order and documentation make sense. Anarchy cannot be allowed to reign. Yet the social scientist in me, a realist one at that, tells me that this makes better sense of the real world. People, organisations and problems are messy. Why not be realistic and remove the linear planning processes we put in place to manage it? The same arguments are deployed in international and IT development as are deployed for research. Namely – you cannot plan research, you cannot know where you will end up at the beginning of a project.
Yet more scientific disciplines seem to manage. House builders, architects, physicists, manufacturers and many other disciplines seem to be able to design, build and deliver things from the outset and use budgets, input process and output measures to control the activities. These are also complex things. Boeing builds complex aeroplanes. Mercedes complex cars. So why should IT, international development and academic research be any different?
I guess as a socially scientific auditor I see a position in between. I see adaptive, agile, serendipitous activities as valuable. Valuable as part of a portfolio. A minority part of a portfolio. All universities, companies, international development NGOs and IT functions need some space to be creative. Space to allow freedom to adapt and change. This is where the truly imaginative and creative breakthroughs will occur. But most organisations will need to balance this. They will need to justify the use of the resources applied. They will need to be able to have overall value for money. High risk (in the uncertainty sense) high return (in the innovation sense) processes are fine, but you need some lower risk but still substantial return projects to balance this out. Any organisational portfolio that only comprises these elements will fail at some point; it is just a matter of time.
So is serendipitous, adaptive or agile work auditable? Sure. First question – is it suited to the task? i.e. does the project need something that mostly works or 100% works. I would not like to see agile work on airplane construction for example. Second question – are these types of project too significant at a portfolio level? If the they are, the organisation is put at significant risk of failure. Third question – If it fails, can the organisation cope with all of the impacts? For this think not just financial, but also legal, political and most importantly, reputational. Reputation risk is difficult to predict and even more difficult to control. Fourth question – is the project controlled? For being adaptive, agile, or serendipitous is not being out of control. I would expect to see excellent risk management. Constant updates to paperwork in an efficient manner. A really strong audit trail of decisions taken and escalation of decision making where they required it.
So I would argue that these flexible methods, applied well, in context, in proportion, by the very best people the organisation has to offer, can be perfectly well controlled. It can be equally well audited with an auditor with the right mindset.
My experience tells me that too often though, these structured methodologies are taken to be a lack of structure, a relaxing of control, a lack of suitable accountability, and too often they are done with others’ resources without recourse to the funder. For the methodology is never a justification for poor control, only different control. As auditors we will need to lighten up, be less scientific and more flexible, for these are spaces in which independent, intelligently applied, internal audit has a legitimate and helpful remit.
So when is your next agile, adaptive or serendipitous audit?
James Christie said:
I pretty much agree with the article, though I’d like to offer a slight different perspective. Firstly, technical debt isn’t something that is paid off by a successful development. It’s the accumulated overhead that is incurred by patching, extending, and adapting applications over time. Short term decisions are taken that lead to long term inefficencies and expense supporting applications that end up as a mass of fragile spaghetti code.
Technical debt also arises from the initial development it it is carried out in a traditional linear fashion. At the start it’s impossible to envisage what is required or exactly how it will be delivered. The end result is lashed into shape by developers working under extreme pressure fixing faults in testing.
That’s one of the reasons why Agile offers the possibility of better outcomes. We don’t know enough at the start to estimate accurately or define the whole project. We can’t know, because that is the nature of the problem. Any pretence that we can is a delusion. However, it is a reassuring delusion because we like to pretend, and have often been effectively ordered to pretend, we know exactly what we will have to do; to pretend we are carrying out a form of engineering like building a bridge.
Construction engineering is a very complicated activity, unlike software development which is mainly complex. I’m using the distinction that the Cynefin Framework makes. Complicated activities are predictable and repeatable. however difficult the individual tasks might be. Complex activities are unpredictable and we have to learn what will work. Cynefin has helped me to make sense of what I’ve had to do as a tester and an auditor. I’ve tried to explain here how I’ve found it relevant to both testing and auditing. https://clarotesting.wordpress.com/2014/09/11/cynefin-testing-auditing/
I strongly believe that auditors should call out spurious confidence in project managers and users. Unfortunately this goes against the culture of many organisations. They would never admit it, but in practice they often value confidence over competence. Realistic pessimism is discouraged, even punished. I dealt with it here. https://clarotesting.wordpress.com/2011/09/13/it-always-takes-longer/
Responsible developers should be able to look to auditors as allies in trying to make their companies accept realism. Agile doesn’t guarantee better results, but it is grounded in a more realistic understanding of the true nature of software development and so it does tilt the odds in our favour. But nothing is ever certain, and we have to be honest enough to admit that.
By the way, it looks a bit confusing referring to “structured methodologies” in the context of Agile. That’s a term usually reserved for traditional, formal, linear, document heavy methods.
LikeLike
Pingback: Agile, adaptive, serendipitous or out of control? | riskspeak