Efficiency audits: why you shouldn’t avoid them and how to tackle the objectionsJun 19, 2020
This article focuses on public sector performance auditors. However, it will also be useful for internal auditors who want to go beyond traditional conformance audits.
In a recent article, we discussed the importance of assurance leaders using crisis as an opportunity to reset strategic priorities.
But which audits will give you the most bang for your buck?
What does bang for the audit buck look like for your team or your audit office?
You need to deliver a program of audits that span your performance audit mandate. Balancing a spread of audits that cover the 3Es and 1C; efficiency, effectiveness, economy, and compliance.
Balancing your audit program
Traditionally, audit offices have steered towards assessing effectiveness. Asking whether an activity (e.g. program or scheme) has delivered what it was intended to deliver.
To demonstrate, we looked at performance audits from five Australian audit offices in 2019.
Efficiency was included in only 10% of the audit portfolio mix.
Source: Risk Insights, based on public reports from five audit offices
If the viz above doesn't load or takes too long, here is a static version:
To get the right balance in your audit program, you need to shift focus a bit.
Direct some of your effort towards efficiency audits.
Why is an efficiency focus important for auditors?
There are a few important matters of context.
- Stimulus measures and redirection of public funds to essential services.
These demand a heightened focus on probity, value for money and removing waste.Governments cannot afford to spend money on programs and services that are not maximising resources to achieve their objectives.
- Members of the public need assurance that their hard-earned tax dollars are not being wasted.
They also need confidence that any mistakes will be identified and rectified. That lessons have been learned and that those mistakes will not be made again.
- Governments are increasingly using data, technology and innovating.
The public expects to see this translate into more efficient government.
Citizens do not want to spend time on overly complicated processes.
Audit offices need to ask: can my government improve in these areas?
Efficiency audits can answer this question. They find opportunities for cost savings, minimising inputs. They also find ways to use existing resources to maximize outputs.
What holds us back from doing efficiency audits?
There is a perception that efficiency audits are more difficult to execute than effectiveness or compliance audits. The reasons for this perception include:
- Definitions of efficiency: Most countries, and even different state audit offices within each country, have not adopted a common definition.
- Auditee acceptance: Auditees may be resistant. This is often driven by fear – fear that an efficiency audit will focus on individual / team productivity. They think that this puts their jobs at risk (e.g. ‘a way for management to sack staff’).
- Isolating efficiency: It can be difficult to focus an audit on just efficiency. An individual performance audit may need to assess both effectiveness and efficiency. For example, when we want to test whether intended outcomes were achieved (effectiveness) and if resources were optimised (efficiency).
- Technical evaluation: New efficiency metrics can be difficult to establish and measure. They are often also subjective. For example, an ‘efficiency’ to some could be considered a ‘shortcut’ to others.
- Data quality and availability: Data about inputs and outputs is not sufficient or not appropriate. Sometimes there isn’t enough quality data for robust analysis to meet evidentiary requirements.
There’s an added complication:
- Typically, performance audits cannot assess and report on the merits of government policy. You cannot call out inherent inefficiencies in policy design. It’s unlikely that a policy will focus on minimising inputs. And that’s reasonable. It should focus on making sure that an outcome is achieved, regardless of the resources used.
None of these factors justify a reduced focus on efficiency audits. If it’s in your audit mandate, then it’s a function that your legislature considers important. You need to use that power as intended. The public expect it and they deserve it.
The good news: you can overcome these challenges.
- Definitions: It’s not a big deal that there are different definitions of efficiency. It is a relative measure that depends on the context. It’s more important to identify a suitable benchmark or comparator for your assessment. But remember: your definition needs to be simple enough for stakeholders to understand. This Canadian example, a summary definition, is easy to understand:
Efficiency is the optimal use of resources to achieve a given output or outcome.
(Canadian Audit and Accountability Foundation; Practice Guide to Auditing Efficiency; 2013).
- Auditee acceptance: It’s true that opportunities for efficiencies can result in activities being combined, reduced or stopped. But auditors never focus on individuals. It’s just not the way we operate. Reassure auditees that the audit focuses on evaluating how the intended policy outcome is being delivered.
- Isolating efficiency: You won’t have many audits that focus only on efficiency. They evaluate a discrete aspect of an activity, a confined part between policy implementation and its outcomes. In many cases, you need to know how a policy is implemented end-to-end. But you can combine effectiveness and efficiency to paint the broader picture. Or you could focus on the effectiveness aspects initially, then zone in on efficiency aspects.
- Technical evaluation: Your efficiency audit may be looking at an activity from a new angle. An outsider’s perspective, if you like. How you develop efficiency criteria is not different to how you would develop any other audit criteria. They should be well researched. You will discuss them with the auditee. They need to be reasonable measures to assess performance. Caution: if there is no comparator, and there’s disagreement about the metrics you’re using, you probably need to adjust them.
- Data: Data quantity issues can be overcome by using other sources of data as proxies. For example, you could use housing development applications data where you don’t have data on new houses being built. Or you can collect your own, through surveys for example. Data quality issues can be fixed by carefully adjusting the data. An added benefit is that you can give the more complete, adjusted dataset to the auditee. This can help them manage their own performance. A tangible benefit.
- Merits of policy: Auditors don’t make public policy and often don’t need to evaluate it. So then don’t comment on it. Focus on what you can do. Assess whether it’s being implemented Your efficiency metrics don’t need to deal with how a policy was designed, or its intent, only how it was implemented.
Make sure your team is aware of these common challenges. Importantly, make sure that they are aware of the solutions. This will help them map out their efficiency audits. No more excuses.
Prioritising your efficiency audits
Let’s assume you know how many efficiency audits you need to do next year. In an ideal world, budget would not be a constraint. But audit offices must be efficient themselves!
You need to prioritise those that will deliver the greatest impact.
Many audit offices use tried and tested models to prioritise audit topics.
Such models consider factors for each topic, including:
- relevance to various stakeholders.
Some other things you need to think about include:
- New initiatives – what are the programs or government investments in brand new initiatives? Where has public money been diverted from? How has the need or demand been met?
- Speed of implementation – which programs have had quick rollouts, with little time between policy announcement and service implementation? Does this make good governance and accountability more difficult to achieve?
- Targeted beneficiaries – is the program aimed at a single cohort or multiple cohorts? Or the public at large? Is it industry focused or aimed at a certain geographic region?
- Hidden incentives – does the design of the program perversely incentivise poor behaviour? Is funding linked to performance? How is that being monitored and assessed?
When deciding which efficiency audits you will prioritise, the most important thing is to involve people who can help you answer these questions.
Example topics for efficiency audit
Here are some examples of efficiency audits. They have helped to improve the use of resources for important public services. They are useful because you can potentially use or replicate them. They can be used to find efficiencies in similar contexts and sectors.
Administration of youth allowance and study support [Australia, Education]
- Equity of educational opportunity is fundamental to a modern society
- Financial support must be managed properly to promote equity
- Similar audits could target efficiency aspects of equity initiatives e.g. early years development, employment, gender.
Administration of justice in the Canadian Armed Forces [Canada, Defence]
- Timely access to justice is a foundation of a properly functioning justice system
- Delays can have detrimental impacts on individuals’ health and wellbeing and breach legal rights
- Similar audits could target efficiency aspects of the justice system e.g. remand, courts, tribunals, sentencing, parole.
Managing the Bank of England’s central services [UK, Central Agency]
- Central banks and other central agencies have traditionally been conservative bodies
- But they need to move with the times. The need to nsure that their internal functions can efficiently support them to deliver their national mandate
- Similar audits could target efficiency aspects of central agencies’ corporate functions e.g. HR, IT, procurement and finance.
Border security: using information to process passengers [New Zealand, Justice]
- Secure borders are important for a country’s security and economic and social prosperity
- Globalisation has increased the travel of people across borders which means managing border risks in an efficient way
- Similar audits could target efficiency in the operations of any regulatory agency balancing increased volume of activity with community safety e.g. licensing & compliance units, intelligence sharing, penalties and infringements.
Renewed focus on efficiency is here to stay
The public and parliamentarians are demanding more value from public services.
This means the focus on efficiency audits will only keep growing. They need to be prominent in your audit mix.
But you need to carefully consider which efficiency audits you do and why. You may have a standard way to prioritise audit topics. Or you may react to emerging priorities. It doesn’t really matter, as long as your selection process asks questions from differing viewpoints. Your selections must be defensible and focus on delivering the greatest impact.
More efficiency audits. Better economic and social outcomes.