Those who deliver programs and services to the residents of remote Australia need to lift their game to ensure they are delivering outcomes that are meaningful and important to community members.
That’s according to Professor Tim Carey, Director of the Centre for Remote Health in Alice Springs, who is heading to the United States on a Fulbright scholarship to investigate how to improve the evaluation of health services and programs for remote communities.
He writes: “If systematic, routine, and ongoing monitoring and evaluation was a necessary feature of the conduct of health professionals and other human services providers, it is hard to imagine how a situation like Don Dale in the Northern Territory could ever have eventuated.”
Tim Carey writes:
A recent review advised that total spending on Indigenous programs is estimated to be at least $5.9 billion each year (Hudson, 2016). The review identified 1,082 current, Indigenous-specific programs being delivered as a result of this funding but, alarmingly, only 88 of these programs (less than 10 percent) had been evaluated.
Worse still, even when programs were evaluated, many could not provide evidence of the program’s effectiveness because of the evaluation methods used (Hudson, 2016).
This is an unacceptable situation and one the Federal Government is seeking to rectify with the recently announced plan to develop an Evidence and Evaluation Framework. The Government announced they will allocate $10 million a year for 4 years to establish the framework, which will ensure that reporting, monitoring, and evaluating will become a systematic and routine aspect of all phases of program planning and implementation.
Remarkably, about a month or so before the Federal Government made this landmark announcement, I discovered I had been awarded the 2017-18 Fulbright Northern Territory Senior Scholarship. This is remarkable because my Fulbright project dovetails perfectly with the Federal Government’s plan to establish their Evidence and Evaluation Framework.
I will spend 4 months at the Center for Behavioral Health Innovation at Antioch University in New Hampshire learning from the researcher clinicians at that Center about what they have found useful in their work with rural and under-served communities.
My plan is to develop training for health professionals and other people in human services organisations to engage in building practice-based evidence so that we can demonstrate that our programs and services are of meaningful benefit to the residents of remote communities.
Focus on meaningful outcomes
As a scientist-practitioner clinical psychologist I have spent many years discovering the benefits that are possible when clinicians use the data they are collecting in the course of their routine clinical practice to evaluate and improve the programs they offer for the benefit of the people they serve.
In one health setting, for example, through the systematic collection and analysis of practice-based data, we were able to demonstrate that a slight change in the way people were able to schedule appointments for psychological treatment almost doubled the capacity of the service (from 52 referrals to 93 referrals over a 6-month period) while simultaneously removing a 7-month waiting list (Carey & Spratt, 2009).
This meant that people seeking psychological assistance for their difficulties were able to access the service much more efficiently than they could before and had to wait much less time for the service.
Milton Friedman said “One of the great mistakes is to judge policies and programs by their intentions rather than their results” and it is disgraceful that, for many programs and services being provided to Aboriginal and Torres Strait Islander Australians, we are not even able to confidently say they are doing more good than harm. Hudson’s (2016) review highlighted the duplication and waste that are rife in many settings.
The work that we do must be evaluated and it must be evaluated in terms of the outcomes that are meaningful and important to the people with whom we work.
The perspectives of the people in the communities we serve must be used to guide the planning and conduct of the evaluations we undertake. If systematic, routine, and ongoing monitoring and evaluation was a necessary feature of the conduct of health professionals and other human services providers, it is hard to imagine how a situation like Don Dale in the Northern Territory could ever have eventuated.
Knowing the right thing to do is not as important as being prepared to evaluate what we do and make changes based on the evidence we continue to accumulate. Much of that evidence will consist of indicators that the people who are using our services are more able to live lives that they would describe as having meaning and value, as a result of their interactions with us.
Training for service providers
Clinical practice, and human services more generally, should involve an ongoing, dynamic, and systematic interplay between evaluation and service provision. Through my Fulbright Scholarship I plan to be able to offer training to providers of services so that they can efficiently assess the impact of their work in remote Australia for the benefit of the communities they serve.
Through the combined efforts of the Federal Government and the Australian-American Fulbright Commission we might finally begin to close the gap. At the very least, we’ll have an evidence-informed understanding of what’s happening to the gap. The most conservative assessment at the moment would be that the gap isn’t closing any time soon.
With an Evidence and Evaluation Framework and engaging and practical training for service providers there is an exciting and very real possibility that this situation will be corrected.
The residents of remote Australia have endured our inefficiencies for too long. We need to let them know we can do better and we need to demonstrate to them that we are doing better by the improvements they will recognise as they go about the business of creating the lives they would wish for.
• Professor Tim Carey is Director of the Centre for Remote Health, a joint centre of Flinders University and Charles Darwin University, Northern Territory
