The National Health and Hospitals Reform Commission’s final report made many suggestions requiring much more work if they are ever to have any chance of implementation. Consultancies must be rubbing their hands in anticipation of the business that health reform will generate.
Here at Croakey, we thought we’d do our bit to save the public purse by asking some relevant experts for their views about one particular recommendation, and the challenges surrounding its implementation.
This recommendation states (p97):
There should be a common national approach to evaluating all health interventions, with consistent evaluation of medical care, pharmaceuticals, prevention and population health interventions, medical devices and prostheses, allied health and complementary medicine. To use an example this might allow comparison of the relative efficacy of a medical intervention (gastric bypass), a pharmaceutical intervention (an anti-obesity drug), an allied health intervention (a structured program of exercises and diet management) and a population health intervention (a community walking program) in reducing obesity.
Of the experts contacted by Croakey, most liked the general thrust of the recommendation, although several cautions were sounded, and multiple barriers were identified.
Some of the cautions:
Jon Wardle (Qld Uni): The recommendation is certainly good in theory and worth pursuing – though is fraught in many practical ways. I see a well-intentioned measure heading for disaster.
Ben Harris-Roxas (University of NSW), when asked whether the recommendation should be progressed, gave “an emphatic ‘maybe’.” It sounds intuitively appealing but there will be substantial practical and methodological challenges.
Glenn Salkeld, School of Public Health, University of Sydney: It is worth agreeing upon a common set of principles for the evaluation of interventions and then consider the task of how best to evaluate the options. Without a clear articulation of the objectives of resource allocation (which must include the distributional consequences of resource allocation) we run the risk of measuring precisely the wrong thing in an evaluation.
***
Below, in no particular order, are the full responses to our informal survey:
1. Jon Wardle, School of Population Health, University of Queensland
What issues would need to be addressed to enable implementation? There are many barriers. The first is that there seems to be an underlying assumption that the generic nature of reactive pharmaceutical intervention for disease can immediately be transferred to the proactive/preventive treatment of disease. This casually ignores the many socio-cultural or other ‘non-health’ factors that affect the outcome of preventive health measures far more than acute treatments.
It also over-simplifies the methodological challenges in measuring the outcomes of interventions. Some may require a ‘whole-practice’ approach and not have readily identifiable markers of effect (I know of many menopausal interventions that have lower effect on single measures, but are superior on multi-factoral measures like ‘menopause scores’), or some may have patient-centred outcomes that may not be readily identifiable or comparable to clinical scores – but equally important in many aspects (for example the qualitative study of rheumatoid arthritis patients in Bristol that for the first time identified that fatigue associated with the condition, not pain, was the factor that most impacted their life – ‘clinical’ research had not previously explored this area). Of course people are individuals – some may respond better to diet than drugs, some vice versa.
Would a crude comparison make the intervention most effect for that individual unavailable as most other people did not see an effect? I would also suggest that it is not just health interventions that are measured. After all one of the most significant public health initiatives was seat belts – and that had bugger all to do with the health system!
The society in which we live predisposes the population to poor health outcomes (ie social isolation of suburbia; car-focused non-active transport; liquor licensing laws that encourage ‘super-pubs’ and subsequent binge drinking and violence over smaller, more social venues; urban planning that has reduced access to greengrocers, markets or other healthy and cheaper food choices in low SES areas; food safety laws that promote unhealthy options over fresher ones – for example national or processed over local or artisanal foods).
What impact might it have if it was implemented? The potential exists to stifle innovation in the development of health programs if the process is over-centralised or doesn’t deem it fit to focus on a number of similar interventions (ie why is the one ‘diet program’ chosen over another?). The bureaucracy of running it all may actually divert precious resources in the first place!
Do you think that it has any chance of being implemented? What might be some of the barriers? The barriers are too numerous to mention, and many of them involve internal politics, for example it assumes that one groups ‘walking program’ is the gold-standard, or which allied health profession will be chosen? Who will choose? In a sense this is being done already through broad health research funding agencies such as the NHMRC. A more prudent way to achieve this may be to develop research capacity in this new field through special funding rounds and later allow research to be funded on its merit. Although I would love to see the government fund research into each of those interventions my experience of grant rejection (we’ve all been there) makes me a little cynical and think there simply won’t be enough resources to do so.
Are you aware of any other countries that are already doing this? There are many attempts at doing similar things (NICE in the UK) but they are also share as many differences as they do similarities. Collaborations already compare such data already and we may be re-inventing the wheel.
Any other general issues that are worth raising? There are many factors that can affect health, in areas as disparate as transport or retail competition policy (for example it is pointless promoting to eat more vegetables if current retail policy competition means that the duopoly restricts access and affordability to nutritious foods – sorry a shameless plug for an article of mine coming up in ANZJPH). Many of the health issues of the future are particularly susceptible to the implementation of non-health policy and focus on this issue was sorely lacking. Some nations (including New Zealand) are investigating the role of “Health Impact Assessments” on new projects (such as housing estates) or policy and this is an area that should be explored further.
***
2. Chris Del Mar, Dean, Faculty Health Sciences and Medicine, Bond University
This recommendation is a worthy objective. In particular some treatments are currently measured with quite different standards (esp complementary and alternative medicines, where ‘traditional use’ which can be recent, is considered an OK to licence, while other drugs need endless costly evaluation)
What are some of the issues that will need to be addressed to enable implementation? We need to facilitate randomised controlled trials more easily. Currently these are difficult to get done. Multiple ethics committees etc. In fact an RCT needs a higher standard of care than ordinary treatment (even if unproved!)
What impact might it have if it was implemented? People (patients and their doctors) should be encouraged to partner more in RCT evaluation of novel treatments. Currently this is seen as a ‘predator-prey’ relationship, which the trial funders are trying to make a buck for the shareholders, which is OK sometimes, but often there is a value to society by the RCT taking place and we all benefiting for the knowledge.
Do you think that it has any chance of being implemented? Yes, if we can change attitudes here.
***
3. Michele Kosky, Health Consumers Council WA
Great idea, let’s have treatment based on best evidence and most cost effective for patient and the funder. I think it would reduce the competing interests and the territory wars between the tribes of health professionals that we often witness and it could be graded so that the gastric banding, for example, becomes treatment of last resort with all its inherent risks. I think that the industry groups in health and the pharmaceutical industry will ensure this proposal does not happen.
***
4. Carol Bennett, Consumers Health Forum of Australia
Is this recommendation worth pursuing? Yes, although it is obviously a goal or stretch target rather than something that can be quickly or easily achieved.
What are some of the issues that will need to be addressed? For common measures of efficacy, there will need to be common measures of patient outcomes which will need to include the experience of care – i.e. the degree to which health consumers felt the treatment matched their health needs rather than the degree to which the service provided matched some rigid standard of procedure.
What impact might it have if it was implemented? If health consumers were properly informed about likely outcomes from the full range of health interventions available, it would allow consumers to have greater control and ownership of their own health and well being. Obviously this would lead to a much more healthy community. At present, whichever procedural specialist you see, whether they be a surgeon or a gym instructor, they will be bias towards the efficacy of the services they provide.
Do you think that it has any chance of being implemented? What might be some of the barriers? This has very little chance of being implemented in the short to medium term primarily because existing health providers generally like to measure their performance against the efficacy of their agreed procedures rather than comparing real health outcomes for consumers. We need to make health outcomes for consumers the focus of evaluation, not the procedures that are currently used. A good model in the general practice setting is the Collaboratives program which focusses on health outcomes.
Are you aware of any other countries that are already doing this? Increasingly countries like the UK are looking to health consumer outcomes as the primary reference point for efficacy of any procedure or health intervention, but it still has a long way to go before it becomes a realised ambition as outlined in the recommendation of the NHHRC.
Any other general issues that are worth raising? While the goal may be a long way away, any step to provide better information to health consumers about the outcome of a range of possible interventions is to be encouraged and supported.
***
5. Public health physician George Rubin
What are some of the issues that will need to be addressed? National mechanism/s for conducting the reviews – a network of organisations under the auspices of NHMRC; methods agreed and developed …perhaps along the line of the evolution of systematic reviews
What impact might it have if it was implemented? Gradual shift of practice towards interventions of demonstrated effectiveness; hopefully a mechanism for ongoing assessment of interventions for which there is little or no evidence; reduction of interventions proven ineffective
Do you think that it has any chance of being implemented? What might be some of the barriers? Little chance. It will take a major organisational effort, will need a substantial budget (which could be saved by a successful program) and will involve substantial politics around what potentially gets dropped. More realistic to develop a program of comparative effectiveness measurement. Are you aware of any other countries that are already doing this? Health technology assessment organisations in NZ, Canada, US etc but unaware of comparative assessment programs
***
6. Michael Moore, Public Health Association
I think that there is some merit in pursuing the recommendation. However, I do have some reservations about how the comparative research might be conducted and the prism through which such research might be constructed. Over the last decades preventive and public health has been the poor cousin in research because it is not simply about a gold standard double blind study with all confounding factors eliminated.
The medical and the pharmaceutical research methodology is likely to be more easily understood and therefore carry a favourable bias. I am not sure how these concerns might be met. A sensible study that wrestled and dealt with those issues has the potential to provide some serious insights into the most effective methods of dealing with obesity providing opportunity for governments to determine the most effective allocation of resources.
Implementation will be difficult as I suspect the funding of broad ranging research across different disciplines as this will demand is likely to be very expensive. However, it is possible and could be achieved by letting a tender to a consortium capable of crossing the barriers.
The complexity of obesity, the structural issues that drive it and the role of junk foods is probably a higher priority for research funds.
***
7. Ben Harris-Roxas, Research Fellow, Healthy Public Policy Program, University of NSW
At time when we face increasingly complex health issues that are caused by multiple factors such as the way we live, work and take care of ourselves this may lead us further down the path to simple answers.
Also it’s unclear what methods might actually be used to compare interventions, e.g. cost-effectiveness studies, health technology assessment, etc. It’s not like issues of effectiveness will be examined independent of cost, so we should really be calling it what it is – a discussion about which intervention(s) should be funded based on their comparative cost effectiveness.
What impact might it have if it was implemented? I think it would tend to concentrate preventive activities even further on simple, well tested interventions that are closely tied to health outcomes, i.e. not complicated by other factors. This is not a bad thing in of itself, but it could stifle innovation and may only address the underlying issues at the margins.
Do you think that it has any chance of being implemented? What might be some of the barriers? It depends what the purpose of the “national approach” is. Will it be to ensure that only effective interventions get used or to decide that only a limited number of the most cost-effective interventions will be funded? Getting the states and the Commonwealth to agree on a common approach to this will be a minor miracle. The other issue is that the capacity to do health economic analysis is exceedingly weak at the state level and quite patchy at the Commonwealth level. The ability of drug manufacturers to demonstrate their products’ cost effectiveness is beter developed than, say, people developing population-based interventions, partly due to the nature of the interventions and outcomes but also because they are already doing cost-effectiveness studies for the PBAC/MSAC. The other problem is that it will favour one or two interventions as being the most cost-effective, when much of the emphasis in population health research is shifting to using complex interventions for complex problems.
Are you aware of any other countries that are already doing this? The UK’s National Institute for Health and Clinical Excellence (NICE) do this to a certain extent and the conventional wisdom is that they will only recommend that a drug or procedure gets funded if it costs less than £30,000/QALY.
If the “national approach” is only to set a minimum benchmark for cost effectiveness of interventions we may encounter a situation where
• costs grow rapidly (as currently occurs once drugs are apporved by PBAC or procedures by MSAC),
• there are increased inequalities in the health outcomes, and
• there will be concern about funding more and more interventions, suggesting that the benchmark threshold will have to be revised regularly.
A broader discussion is required about how much we are willing to pay as a nation and society for our health and how much we’re willing to invest in an average individuals’ health, but that’s too politically sensitive to contemplate in the forseeable future. If the purpose of the “national approach” is to ration services, and it would have to be at some level, this debate will have to occur.
Any other general issues that are worth raising? Transparency of the process will be important. Also, how do we circumscribe our consideration of effective interventions for health. For example, a recent meta-analysis of the lifetime effects of education on quality of life made a conservative cost estimate of between €98,000 and €168,000 for every QALY gained [Furnee CA, Groot W, van den Brink HM. The Health Effects of Education: A meta-analysis. European Journal of Public Health 2008;18(4):417-21.], and this didn’t quantify the effects such as the impact of education on mortality.
Education has a number of other positive impacts beyond direct health gains – should we be funding education as a cost effective health intervention?
***
8. Wayne Hall, School of Population Health, University of Queensland
What are some of the issues that will need to be addressed if this is to be implemented? Something along the lines of the Pharmaceutical Benefits Advisory Council would need to be set up to evaluate major interventions in fields of high public health priority. Given the demands on expertise and time to do these analyses such an approach would probably have to be phased in.
What impact might it have if it was implemented? It would put the onus of proof on advocates of new approaches to provide evidence not just of their safety and efficacy but their cost-effectiveness compared to other similar interventions directed at the same risk factor or disease. It would make government think more strategically about funding for health care interventions rather than make piecemeal decisions about things one at a time (as we do with drugs with PBAC with limited capacity to revisit past decisions in the light of new interventions)
Do you think that it has any chance of being implemented? What might be some of the barriers? Barriers will be many: special interest groups (e.g. food industry, diet industry, pharmaceutical industry, health professionals with an investment in particular approaches); the economic costs to govt of implementing and scaling up such a system of evaluation; lack of data for economic evaluations etc etc
Are you aware of any other countries that are already doing this? NICE in the UK attempts to do something along these lines in so far as data permit for major areas of public health concern. Rob Carter from Deakin and Theo Vos at UQ are undertaking this sort of project at present for an NHMRC funded grant. Rob has done something along the lines suggested on obesity interventions and Theo something similar on alcohol.
Any other general issues that are worth raising? It’s an idea that needs to be seriously discussed and trialed in a couple of key areas of current public health concern, e.g. obesity
***
9. Glenn Salkeld, School of Public Health, University of Sydney
What are some of the issues that will need to be addressed? Common issues – is there a common metric of health outcome that captures what matters? (ie the stated objectives of government health expenditure) Where does the information for the evaluation come from and is it valid and reliable? Who (an independent body or not) and what governs the assessment of the evaluation (is it regulation, legislation?) How does the process manage competing interests (of both buyers and sellers of health interventions/products)? What impact might it have if it was implemented? It would slow things down and allow time for proper debate on the best use of scarce resources. Depending on the responses to questions 1 & 2, it has the potential to shift resource allocation to a more transparent process (and hopefully move resource allocation to a more considered and defensible basis)
Do you think that it has any chance of being implemented? What might be some of the barriers? It will require a good deal of courage. It has worked in the pharmaceutical area because the only way to sustain large public expenditure that benefits the population and the sellers of medicines is to have an evidence-based approach to the selection and reimbursement of drugs. Without the public subsidy both sides lose. With a credible way to allocate money for medicines both sides gain. Are you aware of any other countries that are already doing this? The UK has the National Institute for Clinical Effectiveness (NICE) but that functions in the context of a more planned health service.
Any other general issues that are worth raising? Without a national approach to better resource allocation in health we will struggle to exert any control over who gets what health resources and how much. Doing nothing will only increase health inequalities.
***
10. Les Irwig, School of Public Health, University of Sydney
My quick reaction is:
• Great idea!
• will need massive Govt funding to avoid the problem of RCTs being more likely for new pharmaceuticals.
• non-drug interventions used in the trials will need good descriptions – see Paul Glasziou’s recent paper in the BMJ on how poorly they are described currently in the lit
• need better consumer/patient involvement to define what endpoints are meaningful to patients and therefore should be measured in the trials
• pity they didn’t include diagnostic technology, for which far more trials should be done
**
11. Lloyd Sansom, Chair, Pharmaceutical Benefits Advisory Committee
The Government is currently undertaking a Health Technology Assessment and one of the issues they are examining is the relationship between the various agencies particularly in regard to hybrid technologies and timeliness . With increasing applications of molecular targeted drugs, drug/device combinations it is important that we examine the situation to improve the efficiency of the process.
The PBAC submission to the review is here. In view of the current review it would be inappropriate for me to make any other comment other than the submission
***
12. Jon Jureidini, Adelaide psychiatrist and member, Healthy Skepticism
It could have a dramatic impact in some areas, but would require significant and often unpopular disinvestment. There would be some practical barriers, but the political ones will be stronger.
***
13. Libby Roughead, School of Pharmacy and Medical Sciences, University of SA
Very often in health care we need implementation of a mix of strategies, not just one, so there needs to be a context in which this recommendation is considered. That being said if we only evaluate pharmaceuticals against pharmaceuticals, then we might not understand how that compares with other interventions which may be less or more effective – which I suspect is the issue trying to be raised.
If you look at the US Institute of Medicine’s top 100 priorities for comparative effectiveness, you will see that many of those are about gathering evidence for the mix of health care interventions – although I gather the top 100 priorities were opinion-based supported by a nominal group process rather than based on other data.
It is certainly a good idea to think about how public health dollars and practice and primary health dollars and care may be better integrated and how the evidence may integrate – we will always need both strategies (not just one or the other).
The challenge is creating the incentive for developing the evidence – it is work rarely done and we already have a challenge of evidence just within areas of health care. I suspect the starting point is to create the dialogue about what this means – as Croakey is doing.
