Outcomes: who decides?

There is plenty of debate about outcomes in the social services sector at the moment: how to identify them, how to measure them, how to use them for continuous improvement, and how to report back on them. Recently, Susan Maury, Policy & Research Specialist with Good Shepherd Australia New Zealand @GoodAdvocacy @SusanMaury, presented on an unusual approach to developing outcomes. Here she expands on the possibilities.

Outcomes are higher-order effects from programs – the things we do that result in a significant change in the community. Selecting the right ones to track is important because it influences all of the decisions made for a program. While there is a great deal of interest in taking an outcomes-oriented approach to addressing social issues, it is perhaps telling that I had an exasperatingly difficult time finding a working definition of “outcome” as it is used in relation to social interventions. The best definition I could find is provided, perhaps not surprisingly, by the Department of Health. The health sector knows that definition is the first step to measurement.

Recently, Good Shepherd Australia New Zealand was invited to present a conference paper on an action research project that takes an unusual approach to identifying and tracking outcomes. The Uplift project was looking at ways to increase parent engagement at a primary school that services a low-SES community. The intent of the research was to not just collect information from the participants, but to provide a methodology that might increase engagement through active participation in the research project. We chose to use the concept of empowerment to develop the project. What could be more empowering than to facilitate the parents in selecting their own outcomes and indicators?

Three short workshops led the parents to create a vision for their children, identify actions that needed to be taken to achieve the vision, and then advocate for their plan – that is, get the right people committed to taking action. This led to five vision statements that could be used as outcomes, and a set of specific actions for the parents, the school, and the community to take in order for it to be achieved – serving as indicators. The method also allows for the parents to self-monitor progress on the indicators.

In two days of listening to presentations on identifying, tracking, and using outcomes data for program improvement, there were no other presentations on how outcomes might be generated by a client population. There was much reference, however, on the tension of incorporating client perspective and/or achieving client compliance in measurement processes.

Understanding where the concept of measuring outcomes has come from may explain this bias towards a top-down process. There is a close relationship between outcomes measurement and social impact investment. Social impact investment is part of a movement to align social services provision with a business model; governments or private investors fund social change which is likely to yield positive social benefits (see, for example, the Government of New South Wales’ web site on this topic, or the recently-established Impact Investment Australia). And, like a business model, investors want evidence that the approach will yield the anticipated benefit. And, like a business, it is predominantly a discussion between funders and funded.

There’s much hype over the potential for this new approach to address old problems. The introduction to the “visionary and compelling” Social Impact Investment Report developed for the G8 claims that “the world is on the brink of a revolution in how we solve society’s toughest problems. The force driving this revolution is ‘impact investing,’ which harnesses entrepreneurship, innovation and capital to power social progress.” The US-based Jason Saul, a leading expert on measuring social impact, is creating a “universal outcomes taxonomy” and further claims to have developed an algorithm that “can leverage predictive analytics to forecast a program’s efficacy in producing a desired outcome.” (It should be highlighted that many of these claims come from The Stanford Social Innovation Review, which is not peer-reviewed, but rather intended to bring “cross-sector solutions to global problems.” Additionally, some of these claims have been challenged in the same journal.)

I support systems change and collaboration. Identification of high-level social problems and a multi-dimensional response is often the key to creating lasting change (the dramatic drop in smoking rates in response to gradual changes in public policy is a stellar example). Much of Good Shepherd Australia New Zealand’s advocacy is focussed on creating structure change that advantages people who are marginalised, and we are known for our collaborative ethos.

However, it’s also important to consider the theories underlying new ways of working, which the social services sector is often forced to adopt through pressure from funding agencies. In this particular case there appears to be an implicit assumption that

1. businesses operate more effectively than not-for-profit organisations;
2. an economics approach is always the best approach;
3. when the “best minds” get onto this issue, it is at long last solved; and
4. the recipients of these services have little or nothing to add to the discussion.

My frame of operation, and many whom I know in the not-for-profit sector would agree, is that:

1. everybody is an expert in their own life experiences;
2. given the opportunity even highly disadvantaged and marginalised people are quite capable of identifying strengths, barriers, and possible avenues for positive change – and can articulate these eloquently;
3. the process of creating their own narrative is itself a positive force for change; and
4. organisational ways of working need to be congruent with what’s known to be best practice.

Perhaps another point could be added in here as well, about respecting the knowledge and practice base of the not-for-profit sector and recognising the incredible achievements that it by and large produces despite a range of systemic constraints.

I would argue that it’s not a common set of indicators that we need in order to better achieve high-level outcomes or societal impacts; rather, it is a common, shared, and promoted set of values. One of these values ought to be to ensure the dignity and active participation of our clients in identifying outcomes.

The not-for-profit sector carries different values than for-profits do, and those values are our key distinction. Working effectively with people is not a business, and a business model is seldom an appropriate response to the kinds of needs our clients face. When we appropriate models or responses from other sectors, we must ensure they align with our values of providing dignity, respect, compassion, and voice to those who have not had their share. My experience has been that it is through these values that problems are solved effectively.

Getting back to the research project I piloted with the parents at a disadvantaged school, the results were far beyond my expectations. The parents advocated for their plan with community leaders, staff at the school, and other parents. The few hours I spent with them resulted in a range of positive changes in the community. They are being supported, but it is their vision, their plan and their actions that are making a significant, lasting change in the community. It’s what we know works well. As a sector, we need to recognise and appreciate our strengths and our values. When we adopt a new way of working, processes must be tailored to ensure congruence with proven ways of working effectively. This is an ongoing negotiation and one we need to actively manage.

Posted by Kathy Landvogt