Unlike some other countries, Australia does not have a large philanthropic medical research sector. This means that researchers rely heavily on government funding for their work, primarily via grants from the National Health and Medical Research Council (NHMRC).
All grant allocation processes are – to some extent – subjective. However, the central importance of the NHMRC in medical research in Australia highlights the need for its processes to be as fair, objective and free from discrimination as possible. Given that NHMRC grants involve public funds, it is also important that funding decisions aim to deliver maximum value to the Australian community.
The NHMRC has attracted criticism in the past about the inefficiency of its grant application process and the waste of valuable time and effort from Australian medical researchers who can spend hundreds of hours on grant applications only to be rejected.
Croakey has also published critical views of the NHMRC’s record on supporting Indigenous-led research.
These issues are now under consideration as part of the NHMRC’s review of its grant application processes.
This post from Professor Jon Karnon, from the School of Public Health, University of Adelaide, discusses the review and argues that if new drugs and medical services have to demonstrate their value before attracting a public subsidy then so too should medical research.
Jonathan Karnon writes:
Last month, the Australian Commission for Safety and Quality in Health Care (ACSQHC) published a report that estimated a huge return on 25 completed clinical trials conducted by the Australasian Stroke Trials Network (ASTN), the Interdisciplinary Maternal Perinatal Australasian Collaborative Trials (IMPACT) Network and the Australian and New Zealand Intensive Care Society Clinical Trials Group (ANZICS CTG).
Return on investment in clinical trials
The report estimated that if the results of the evaluated trials were implemented in 65% of the eligible Australian population, in one year $580 million of cost savings would be generated and a further $1.4 billion’s worth of improved health would be gained.
The 25 evaluated trials were selected from more than 460 trials that were conducted by the three networks. The National Health and Medical Research Council (NHMRC) has reported that Australian clinical networks conducted more than 1,000 trials between 2004 and 2014.
A proportion of those other trials likely produced valuable findings too and it is clearly not reasonable to expect every funded trial to generate significant health benefits or cost savings. As with all facets of life, there will be hits and misses.
The main funder of health research, the NHMRC has announced changes to its grant program and is consulting on a framework for the assessment and funding of clinical trials. However, the new grant program and consultation paper do not refer to the process by which decisions are made about which research will be funded.
Value for money?
Could greater consideration of value for money generate more positive outcomes from the funds allocated to the conduct of health research in general, and clinical trials in particular?
How are decisions made about which research to fund? In the case of research projects funded by the NHMRC, grant review panels are assembled each year to review, score and rank research applications. There are a wide range of panels and each panel focuses on a specific area of research, for example, research on cancer, cardiovascular disease, Indigenous populations, primary health care, health services research, etc. Panel members have expertise in the area of focus for their panel.
The panel is asked to score each application out of 7 with respect to three criteria: quality of the proposed study design, the experience and relevance of the proposed research team and the significance of the proposed research. A weighted average score out of 7 is then generated for each application. The quality of the proposed study design is assigned a weight of 0.5 and the other two criteria are assigned weights of 0.25 (i.e. quality of study design is twice as important as the proposed research team and the significance of the project).
These weighted scores essentially determine the allocation of funds. The scores are heavily weighted to the funding of research with a high probability of completion – 75% of the panel’s score reflects study design and research team.
The cost of the proposed research is excluded from direct consideration. Such criteria incentivizes high-cost research, but a larger number of lower-cost research studies may increase the chances of a ‘hit’ and produce greater aggregate returns.
A new criterion
In a recent call for research proposals around dementia (the Boosting Dementia Research Initiative), the NHMRC introduced a new criterion of value for money to the assessment of research proposals, which made up 15% of the aggregate score for each proposed research study. This is a move in the right direction, but really value for money should be the sole criterion.
Pharmaceuticals and new medical services must demonstrate value for money to receive public subsidy, why shouldn’t research be subject to the same criterion. The Pharmaceuticals Benefits Advisory Committee (PBAC) firstly assess the evidence on the clinical effectiveness of a new pharmaceutical.
If there is deemed to be sufficient evidence that the new drug will address a significant level of unmet need, the committee than assess whether the drug provides value for money at the price requested by the manufacturer. Value for money of new pharmaceuticals is a function of the severity of the condition being treated, the effectiveness of the drug, the likelihood that only indicated patients will use the drug and they will take the drug as prescribed and the cost of the drug.
Cost benefits of research
A similar approach could be applied to health research, noting that value is a function of the probability that the proposed study design and research team will generate a robust answer to the proposed research question, that a robust answer will translate into significant health benefits (or cost savings) and of course the cost of the proposed research.
Analogous to submissions made by pharmaceutical companies to the PBAC, grant applicants could be asked to estimate the potential health benefits/cost savings associated with a positive research outcome. Grant review panels could adjust these estimates based on the presented evidence and estimate the probability of a positive research outcome. Combining these two parameters will inform the expected benefits of the proposed research, which can further combined with the cost of the proposed research to rank research applications.
Clearly some very high value research is being funded and the announced changes to the NHMRC grant program should further increase returns to research funding. However, without substantial changes to the way in which grant applications are assessed, the process will continue to incentivize high-cost research. A focus on value for money, analogous to the processes by which new technologies are assessed, could support more research and even higher returns to research funding.
