Whats wrong with impact agenda?

Dr James Rees (@JamesRees_tsrc) from the Third Sector Research Centre examines the challenges of emerging 'impact agendas' with regard to funding community sector organisations. This is the second of three blog posts from James which tackle different aspects of this issue.Part one, Taming the monster', examined the value add of the community sector in service delivery and policy implementation. 

This is a follow-up to a previous blog post 'Taming the monster', in which I discussed what I think are two of the main reasons why the debate over the supposed distinctive contribution and role of the third sector has become rather confused. This is a rather shorter entry that concentrates in a bit more detail on the idea that a set of – admittedly rather diffuse – doubts have emerged about the wisdom of seeking to pin down the sector’s impact. They centre around the idea that ‘community sector organisations might be distinctive and make a difference but we’ll never prove it’ and furthermore, that there are dangers to going down this route.

In the previous blog post I alluded to the push in recent years to convince organisations in the ‘third sector’ to do better at demonstrating their impact. This has reflected a well-meaning and perfectly understandable sense that if only they could do a better job of identifying, recording and proving their efficacy in achieving socially desirable outcomes, they would be better placed to compete with other providers for government contracts, as well as in the more general ‘marketplace’ for funding from traditional funders (e.g. foundations), philanthropists, and increasingly perhaps, social investors.

After all, the reasoning goes, many people can instinctively sense the good that individual community organisations are doing at the local level for their core beneficiaries, so it should be possible to measure/capture that impact in a more rigorous manner and this can then be used to ensure those community organisations remain funded and are placed on a sustainable footing.

There has been a real push to win hearts and minds in the sector; it has been for instance at the centre of work by influential think tank New Philanthropy Capital. There has also been work by a number of sector bodies to create a code of good impact practice.

From a rather different angle has been the more widespread adoption by the Coalition government of ‘payment by results’, in which direct financial rewards go to providers when they deliver the pre-defined outcomes decided by commissioners. The highest profile UK example of this has been in the national Work Programme. PBR is, in effect, a real times system of impact measurement: the amount that is spent is related to the outcomes that are being achieved.

It seems to me that there have been a number of problems with the impacts agenda here in the UK, which are perhaps not so often acknowledged and discussed:

• Organisations do recognise that the status of outcomes and ‘impacts’ – especially when we’re dealing with the problems of real people – are all too often rather ambiguous. For one thing, can we be sure that outcomes are achieved when their measurement is inevitably a snapshot in time?

In the context of payment by results, for instance, it could lead to providers working with clients just enough to trigger an outcome payment, for example staying in work for x weeks or desisting from offending for y months. Yet does this mean that a real, lasting difference is made to the individual client? In other words, there is a possibility that a focus on outcomes can have perverse implications. It has led to considerable doubt about the outcomes focus and especially so-called ‘binary’ outcomes and calls for greater acceptance of ‘distance travelled’ measures.

• In my research and many conversations with community organisations, there is often a strong and vibrant sense that their services work and make a difference, but they are often at a loss to say how they could convincingly prove that difference. Maybe we need to accept that this form of impact remains essentially elusive. Maybe we lose something by trying to, as one of our interviewees once memorably put it, ‘pin jelly to the wall’.

• Many organisations have long histories of recording outcomes and have well developed data systems. These may be the sorts of community organisations most attractive to ‘Prime’ providers and public sector commissioners. But what is to say that an organisation that is well versed at demonstrating impact is really more valuable? Some social contexts lend themselves well to an outcome/impact gearing: those promoting recovery, or reintegrating individuals back into work. Could these crowd out organisations that provide something equally valuable: care and support for those who won’t recover, perhaps, or those who need services but won’t be measurably changed by that support?

• Some of the mechanisms designed to measure social impact have failed to convince. The most widely accepted method, social return on investment is still not seen universal adoption and is seen by some as complex and costly to implement. Impact measurement is in many ways still in its infancy… I will say more about this in the next blog.

As recent research by Jenny Harlock has shown, commissioners are still at an early stage of disentangling the idea of social value, as promoted by the Social Value Act that came into force in 2013, and more general concerns about outcomes-based commissioning. So even where the policy debate is relatively advanced, there have to be doubts about the capacity of the public sector to adequately operationalize measures of value, impact and outcomes.

In my view, all of these factors – as well as those discussed in the last blog – have contributed to the doubts currently hanging over the third sector about its ability to show that it has something distinctive to offer society, particularly in the context of public services that themselves are undergoing a period of transformation.

Yet my slightly more optimistic message is Don’t throw the baby out with the bathwater…

There are good theoretical reasons to think that even third sector organisations that do engage in public services and work with the state do remain distinctive and have something to offer. And it is perhaps from this more ‘positive’ standpoint that we should argue for a sense of perspective when thinking about the difficulties around the impact debate.

Moreover a wealth of empirical material collected by myself and Third Sector Research Centre colleagues do suggest that although they fully acknowledge the pressures and tensions they face, they don’t ‘sell out’ and lose all the elements that make them different. In short, I think those elements revolve around:

• A ‘philosophy’ that revolves around values, mission, and an understanding and a unique ‘take’ on special issues that the public sector for example can rarely match. Action is often rooted in a social justice motivation.
• An ethic of care rather than a profit motive; and closely linked to approaches like person-centred care, personalisation, user democracy, and advocacy.
• The genuine initiation of novel approaches, innovation, and lobbying to achieve change; trialling approaches before they become mainstream.

In the next blog, I leave these policy and theory-based concerns in the background and focus on a range of practical and practice-based approaches that are being developed in the UK to get to grips with measuring impact and social value.