Filter by Categories
Aged care
Budgets
Federal Budget 2024-25
Federal Budget 2023-2024
Federal Budget October 2022
Federal Budget 2022-23
Federal Budget 2021-22
Climate and health
#HealthyCOP29
Climate emergency
Healthcare sustainability
Heatwaves
National Health and Climate Strategy
#HealthyCOP28
#HealthyCOP27
#HealthyCOP26
#CoveringClimateNow
COVID collection
COVID-19
Long COVID
COVIDwrap
COVID SNAPS
#JusticeCOVID
Caring for the Frontline
COVIDglobalMHseries
Croakey Conference News Service
#WICC2024
#HealthierSocieties
#DigitalDeterminants24
#iDEA24
#ASMIRT2024
#EACH24
#CHFtalks
#SARRAH2024
#GreenHealthForum24
Croakey Professional Services
Sponsored content
NHLF series
#KidneyCareTogether
ACSQHC series
ACSQHC series 2022
ACSQHC series 2021
ACSQHC series 2020
ACSQHC series 2019
CATSINaM 25 Years
Croakey projects
@WePublicHealth 2024
#ChurchillDeepEnd
Planetary Health Equity Hothouse 2024
Summer reading 2023-2024
The Zap
#CroakeyLIVE #DigitalNationBuilding
#CroakeyLIVE #VoiceForHealth
#PHAAThinkTank2023
The Health Wrap
ICYMI
@WePublicHealth
@WePublicHealth2023
#CroakeyVOICES
#SpeakingOurMinds
Croakey longreads
#CroakeyREAD
CroakeyEXPLORE
CroakeyGO
#CroakeyGO #NavigatingHealth
#GamblingHarms
#HeatwaveHealth
Mapping CroakeyGo
Determinants of health
Environmental determinants of health
Social determinants of health
Education
Discrimination
Housing
Internet access
Justice and policing
Justice Reinvestment
Newstart/JobSeeker
Poverty
Racism
Social policy
Commercial determinants of health
Alcohol
Arms industry
Digital platforms
Food and beverages
Fossil fuels
Gambling
Pharmaceutical industry
Plain packaging
Sugar tax
Tobacco
Vaping
Disasters and extreme weather events
Disasters
Extreme weather events
Bushfires
Bushfire-emergency 2019-2020
Floods 2023
Floods 2022
Floods 2021
Floods 2011
Donor-funded journalism
Donor-funded journalism – 2024
Donor-funded journalism – 2023
Donor-funded journalism – 2022
Donor-funded journalism – 2021
Donor-funded journalism – 2020
Elections
lutruwita/Tasmania 2024 election
NT election 2024
Queensland election 2024
United States election 2024
Federal election 2025
#NSWvotesHealth2023
Victorian election 2022
Federal Election 2022
The Election Wrap 2022
SA election 2022
WA election 2021
Tasmanian election 2021
First Nations
Indigenous health
Community controlled sector
Cultural determinants of health
Cultural safety
Indigenous education
Social and emotional wellbeing
Uluru Statement
The Voice
Lowitja Institute
NT Intervention
WA community closures
Acknowledgement
#CTG10
#NTRC
#RCIADIC30Years
General health matters
Abortion
Cancer
Cardiovascular disease
Child health
Chronic conditions
Consumer health matters
Death and dying
Diabetes
Disabilities
Euthanasia
Fetal Alcohol Spectrum Disorders (FASD)
Genetics
HIV/AIDS
HRT
Infectious diseases
Influenza
LGBTQIA+
Medical marijuana
Men's health
Mental health
Mpox
Non communicable diseases
Oral health
Organ transplants
Pain
Pregnancy and childbirth
Sexual health
Suicide
Swine flu
Trauma
Women's health
Youth health
Global health matters
Asylum seeker and refugee health
Conflict and war
Gaza conflict
Global health
WHO
Ebola
NHS
#WorldInTurmoil
Health policy and systems
Co-design
Health financing and costs
Health reform
Health regulation
Medicare 40 Years
MyMedicare
National Health Performance Authority
Pharmaceutical Benefits Scheme
Private health insurance
Royal Commissions
TGA
Workforce matters
Strengthening Medicare Taskforce 2022
National Commission of Audit 2014
Healthcare
Adverse events
Allied healthcare
Australian Medical Association
Choosing Wisely
cohealth
Complementary medicines
Conflicts of interest
Co-payments
Digital technology
E-health
Emergency departments and care
Equally Well
General practice
Health Care Homes
Health ethics
Hospitals
International medical graduates
Medicare Locals
MyHospitals website
Naturopathy
NDIS
Nursing and midwifery
Out of pocket costs
Palliative care
Paramedics
Pathology
Pharmacy
Primary healthcare
Primary Health Networks
Rural and remote health
Safety and quality of healthcare and aged care
Screening
Social prescribing
Surgery
Telehealth
Tests
Media and health
Media-related issues
Health & medical marketing
Misinformation and disinformation
Public interest journalism
Social media and healthcare
The Conversation
Media Doctor Australia
News about Croakey
Croakey Funding Drive June 2024
Public health and population health
Air pollution
Artificial intelligence
Australian Centre for Disease Control
Government 2.0
Gun control
Health communications
Health impact assessment
Health in All Policies
Health inequalities
Health literacy
Human rights
Illicit drugs
Injuries
Legal issues
Marriage equality
Nanny state
National Preventive Health Agency
Obesity
Occupational health
Physical activity
Prevention
Public health
Road safety
Sport
Transport
Vaccination
VicHealth
Violence
Web 2.0
Weight loss products
#PreventiveHealthStrategy
#UnmetNeedsinPublicHealth
Research matters
Cochrane Collaboration
Evidence-based issues
Health and medical education
Health and medical research
NHMRC
#MRFFtransparency
The Croakey Archives
#cripcroakey
#HealthEquity16
#HealthMatters
#IHMayDay (all years)
#IHMayDay 2014
#IHMayDay15
#IHMayday16
#IHMayDay17
#IHMayDay18
#LoveRural 2014
Croakey Conference News Service 2013 – 2023
2023 Conferences
#GreenHealthForum23
#hpsymposium2023
#NMS23
#HEAL2023
#ASMIRT2023
#NSPC23
Our Democracy Forum
#AskMSF
#Lowitja2023
2022 Conferences
#16nrhc
#GreenHealthForum22
#Heal2022
#ICEM22
#NAISA22
#NNF2022
#RANZCP2022
#RethinkAddiction
#RTP22
GiantSteps22
Equally Well 2022 Symposium
Choosing Wisely National Meeting 2022
2021 conferences
#21OPCC
#BackToTheFire
#FoodGovernance2021
#GiantSteps21
#GreenHealthForum21
#HealthClimateSolutions21
#HearMe21
#IndigenousClimateJustice21
#NNF2021
#RANZCP2021
#ShiftingGearsSummit
#ValueBasedCare
#WCepi2021
#YHFSummit
2020 conferences
#2020ResearchExcellence
#Govern4Health
#HealthReImagined
#SAHeapsUnfair
2019 Conferences
#ACEM19
#CPHCE19
#EquallyWellAust
#GiantSteps19
#HealthAdvocacyWIM
#KTthatWorks
#LowitjaConf2019
#MHAgeing
#NNF2019
#OKtoAsk2019
#RANZCOG19
#RANZCP2019
#ruralhealthconf
#VMIAC2019
#WHOcollabAHPRA
2018 conferences
#6rrhss
#ACEM18
#AHPA2018
#ATSISPC18
#CPHCE
#MHED18
#NDISMentalHealth
#Nurseforce
#OKToAsk2018
#RANZCOG18
#ResearchIntoPolicy
#VHAawards
#VMIACAwards18
#WISPC18
2017 conferences
#17APCC
#ACEM17
#AIDAconf2017
#BTH20
#CATSINaM17
#ClimateHealthStrategy
#IAHAConf17
#IDS17
#LBQWHC17
#LivingOurWay
#OKtoAskAu
#OTCC2017
#ResearchTranslation17
#TheMHS2017
#VMIACConf17
#WCPH2017
Australian Palliative Care Conference
2016 conferences
#AHHAsim16
#AHMRC16
#ANROWS2016
#ATSISPEP
#AusCanIndigenousWellness
#cphce2016
#CPHCEforum16
#CRANAplus2016
#IAMRA2016
#LowitjaConf2016
#PreventObesity16
#TowardsRecovery
#VMIAC16
#WearablesCEH
#WICC2016
2015 conferences
#CPHCEforum
#CRANAplus15
#HSR15
#NRHC15
#OTCC15
Population Health Congress 2015
2014 conferences
#IPCHIV14
AIDA Conference 2014
Congress Lowitja 2014
CRANAplus conference 2014
Cultural Solutions - Healing Foundation forum 2014
Lowitja Institute Continuous Quality Improvement conference 2014
National Suicide Prevention Conference 2014
Racism and children/youth health symposium 2014
Rural & Remote Health Scientific Symposium 2014
2013 conferences
Australian Centre for Health Services Innovation Forum 2013
Australian Health Promotion Association Conference 2013
Closing the Credibility Gap 2013
CRANAplus Conference 2013
FASD Conference 2013
Health Workforce Australia 2013
International Health Literacy Network Conference 2013
NACCHO Summit 2013
National Rural Health Conference 2013
Oceania EcoHealth Symposium 2013
PHAA conference 2013
Croakey Professional Services archive
#CommunityControl
#CommunityControl Twitter Festival
#COVIDthinktank21
Lowitja Indigenous knowledge translation series
Croakey projects archive
#PHAAThinkTank 2022
Summer reading 2022-2023
#CommunityMatters
#CroakeyFundingDrive 2022
#CroakeyLIVE #Budget2021Health
#CroakeyLIVE #USvotesHealth
#CroakeyLIVE Federal election 2022
#CroakeyYOUTH
#HousingJusticeAus
#IndigenousHealthSummit
#IndigenousNCDs
#JustClimate
#JustJustice
#LookingLocal
#OutOfPocket
#OutOfTheBox
#RuralHealthJustice
#TalkingTeeth
@WePublicHealth2022
@WePublicHealth2021
@WePublicHealth2020
AroundTheTraps
Croakey register of influence
Croakey Register of Influencers in Public Health
Croakey Register of Unreleased Documents
Gavin Mooney
Inside Story
Journal Watch
Naked Doctor
Poems of Public Health
Summer reading 2021-2022
Summer reading 2020-2021
Summer Reading 2019-2020
Summer Reading 2017-2018
Summer Reading 2016-2017
The Koori Woman
TOO MUCH of a Good Thing
Wonky Health
CroakeyGO archive 2017 – 2018
CroakeyGo 2018
#CroakeyGO #QuantumWords 2018
#CroakeyGO #VicVotes 2018
#CroakeyGO Albury 2018
#CroakeyGO Callan Park 2018
#CroakeyGO Carnarvon 2018
#CroakeyGO Marrickville 2018
#CroakeyGO Palm Island 2018
CroakeyGo 2017
#CroakeyGO Adelaide 2017
#CroakeyGO Melbourne 2017
#CroakeyGO Newcastle 2017
#CroakeyGO Sydney 2017
Elections and Budgets 2013 – 2021
#QldVotesHealth 2020
Budget2020Health
Federal Budget 2020-21
Federal Budget 2019-20
#AusVotesHealth Twitter Festival 2019
#Health4NSW
Federal Election 2019
NSW Election 2019
Federal Budget 2018-19
Federal Budget 2017/18
NZ Election 2017
Federal Budget 2016-17
Federal Election 2016
#HealthElection16
NT Election 2016
Federal Budget 2015-16
Qld Election 2015
NSW Election 2015
Federal Budget 2014-15
Victorian Election 2014
Federal Budget 2013-14
Federal Election 2013
Federal Budget 2012-2013
Federal Budget 2011
Federal Budget 2010
Federal Election 2010
Federal Budget 2009-2010
Support non-profit public interest journalism
Filter by Categories
Aged care
Budgets
Federal Budget 2024-25
Federal Budget 2023-2024
Federal Budget October 2022
Federal Budget 2022-23
Federal Budget 2021-22
Climate and health
#HealthyCOP29
Climate emergency
Healthcare sustainability
Heatwaves
National Health and Climate Strategy
#HealthyCOP28
#HealthyCOP27
#HealthyCOP26
#CoveringClimateNow
COVID collection
COVID-19
Long COVID
COVIDwrap
COVID SNAPS
#JusticeCOVID
Caring for the Frontline
COVIDglobalMHseries
Croakey Conference News Service
#WICC2024
#HealthierSocieties
#DigitalDeterminants24
#iDEA24
#ASMIRT2024
#EACH24
#CHFtalks
#SARRAH2024
#GreenHealthForum24
Croakey Professional Services
Sponsored content
NHLF series
#KidneyCareTogether
ACSQHC series
ACSQHC series 2022
ACSQHC series 2021
ACSQHC series 2020
ACSQHC series 2019
CATSINaM 25 Years
Croakey projects
@WePublicHealth 2024
#ChurchillDeepEnd
Planetary Health Equity Hothouse 2024
Summer reading 2023-2024
The Zap
#CroakeyLIVE #DigitalNationBuilding
#CroakeyLIVE #VoiceForHealth
#PHAAThinkTank2023
The Health Wrap
ICYMI
@WePublicHealth
@WePublicHealth2023
#CroakeyVOICES
#SpeakingOurMinds
Croakey longreads
#CroakeyREAD
CroakeyEXPLORE
CroakeyGO
#CroakeyGO #NavigatingHealth
#GamblingHarms
#HeatwaveHealth
Mapping CroakeyGo
Determinants of health
Environmental determinants of health
Social determinants of health
Education
Discrimination
Housing
Internet access
Justice and policing
Justice Reinvestment
Newstart/JobSeeker
Poverty
Racism
Social policy
Commercial determinants of health
Alcohol
Arms industry
Digital platforms
Food and beverages
Fossil fuels
Gambling
Pharmaceutical industry
Plain packaging
Sugar tax
Tobacco
Vaping
Disasters and extreme weather events
Disasters
Extreme weather events
Bushfires
Bushfire-emergency 2019-2020
Floods 2023
Floods 2022
Floods 2021
Floods 2011
Donor-funded journalism
Donor-funded journalism – 2024
Donor-funded journalism – 2023
Donor-funded journalism – 2022
Donor-funded journalism – 2021
Donor-funded journalism – 2020
Elections
lutruwita/Tasmania 2024 election
NT election 2024
Queensland election 2024
United States election 2024
Federal election 2025
#NSWvotesHealth2023
Victorian election 2022
Federal Election 2022
The Election Wrap 2022
SA election 2022
WA election 2021
Tasmanian election 2021
First Nations
Indigenous health
Community controlled sector
Cultural determinants of health
Cultural safety
Indigenous education
Social and emotional wellbeing
Uluru Statement
The Voice
Lowitja Institute
NT Intervention
WA community closures
Acknowledgement
#CTG10
#NTRC
#RCIADIC30Years
General health matters
Abortion
Cancer
Cardiovascular disease
Child health
Chronic conditions
Consumer health matters
Death and dying
Diabetes
Disabilities
Euthanasia
Fetal Alcohol Spectrum Disorders (FASD)
Genetics
HIV/AIDS
HRT
Infectious diseases
Influenza
LGBTQIA+
Medical marijuana
Men's health
Mental health
Mpox
Non communicable diseases
Oral health
Organ transplants
Pain
Pregnancy and childbirth
Sexual health
Suicide
Swine flu
Trauma
Women's health
Youth health
Global health matters
Asylum seeker and refugee health
Conflict and war
Gaza conflict
Global health
WHO
Ebola
NHS
#WorldInTurmoil
Health policy and systems
Co-design
Health financing and costs
Health reform
Health regulation
Medicare 40 Years
MyMedicare
National Health Performance Authority
Pharmaceutical Benefits Scheme
Private health insurance
Royal Commissions
TGA
Workforce matters
Strengthening Medicare Taskforce 2022
National Commission of Audit 2014
Healthcare
Adverse events
Allied healthcare
Australian Medical Association
Choosing Wisely
cohealth
Complementary medicines
Conflicts of interest
Co-payments
Digital technology
E-health
Emergency departments and care
Equally Well
General practice
Health Care Homes
Health ethics
Hospitals
International medical graduates
Medicare Locals
MyHospitals website
Naturopathy
NDIS
Nursing and midwifery
Out of pocket costs
Palliative care
Paramedics
Pathology
Pharmacy
Primary healthcare
Primary Health Networks
Rural and remote health
Safety and quality of healthcare and aged care
Screening
Social prescribing
Surgery
Telehealth
Tests
Media and health
Media-related issues
Health & medical marketing
Misinformation and disinformation
Public interest journalism
Social media and healthcare
The Conversation
Media Doctor Australia
News about Croakey
Croakey Funding Drive June 2024
Public health and population health
Air pollution
Artificial intelligence
Australian Centre for Disease Control
Government 2.0
Gun control
Health communications
Health impact assessment
Health in All Policies
Health inequalities
Health literacy
Human rights
Illicit drugs
Injuries
Legal issues
Marriage equality
Nanny state
National Preventive Health Agency
Obesity
Occupational health
Physical activity
Prevention
Public health
Road safety
Sport
Transport
Vaccination
VicHealth
Violence
Web 2.0
Weight loss products
#PreventiveHealthStrategy
#UnmetNeedsinPublicHealth
Research matters
Cochrane Collaboration
Evidence-based issues
Health and medical education
Health and medical research
NHMRC
#MRFFtransparency
The Croakey Archives
#cripcroakey
#HealthEquity16
#HealthMatters
#IHMayDay (all years)
#IHMayDay 2014
#IHMayDay15
#IHMayday16
#IHMayDay17
#IHMayDay18
#LoveRural 2014
Croakey Conference News Service 2013 – 2023
2023 Conferences
#GreenHealthForum23
#hpsymposium2023
#NMS23
#HEAL2023
#ASMIRT2023
#NSPC23
Our Democracy Forum
#AskMSF
#Lowitja2023
2022 Conferences
#16nrhc
#GreenHealthForum22
#Heal2022
#ICEM22
#NAISA22
#NNF2022
#RANZCP2022
#RethinkAddiction
#RTP22
GiantSteps22
Equally Well 2022 Symposium
Choosing Wisely National Meeting 2022
2021 conferences
#21OPCC
#BackToTheFire
#FoodGovernance2021
#GiantSteps21
#GreenHealthForum21
#HealthClimateSolutions21
#HearMe21
#IndigenousClimateJustice21
#NNF2021
#RANZCP2021
#ShiftingGearsSummit
#ValueBasedCare
#WCepi2021
#YHFSummit
2020 conferences
#2020ResearchExcellence
#Govern4Health
#HealthReImagined
#SAHeapsUnfair
2019 Conferences
#ACEM19
#CPHCE19
#EquallyWellAust
#GiantSteps19
#HealthAdvocacyWIM
#KTthatWorks
#LowitjaConf2019
#MHAgeing
#NNF2019
#OKtoAsk2019
#RANZCOG19
#RANZCP2019
#ruralhealthconf
#VMIAC2019
#WHOcollabAHPRA
2018 conferences
#6rrhss
#ACEM18
#AHPA2018
#ATSISPC18
#CPHCE
#MHED18
#NDISMentalHealth
#Nurseforce
#OKToAsk2018
#RANZCOG18
#ResearchIntoPolicy
#VHAawards
#VMIACAwards18
#WISPC18
2017 conferences
#17APCC
#ACEM17
#AIDAconf2017
#BTH20
#CATSINaM17
#ClimateHealthStrategy
#IAHAConf17
#IDS17
#LBQWHC17
#LivingOurWay
#OKtoAskAu
#OTCC2017
#ResearchTranslation17
#TheMHS2017
#VMIACConf17
#WCPH2017
Australian Palliative Care Conference
2016 conferences
#AHHAsim16
#AHMRC16
#ANROWS2016
#ATSISPEP
#AusCanIndigenousWellness
#cphce2016
#CPHCEforum16
#CRANAplus2016
#IAMRA2016
#LowitjaConf2016
#PreventObesity16
#TowardsRecovery
#VMIAC16
#WearablesCEH
#WICC2016
2015 conferences
#CPHCEforum
#CRANAplus15
#HSR15
#NRHC15
#OTCC15
Population Health Congress 2015
2014 conferences
#IPCHIV14
AIDA Conference 2014
Congress Lowitja 2014
CRANAplus conference 2014
Cultural Solutions - Healing Foundation forum 2014
Lowitja Institute Continuous Quality Improvement conference 2014
National Suicide Prevention Conference 2014
Racism and children/youth health symposium 2014
Rural & Remote Health Scientific Symposium 2014
2013 conferences
Australian Centre for Health Services Innovation Forum 2013
Australian Health Promotion Association Conference 2013
Closing the Credibility Gap 2013
CRANAplus Conference 2013
FASD Conference 2013
Health Workforce Australia 2013
International Health Literacy Network Conference 2013
NACCHO Summit 2013
National Rural Health Conference 2013
Oceania EcoHealth Symposium 2013
PHAA conference 2013
Croakey Professional Services archive
#CommunityControl
#CommunityControl Twitter Festival
#COVIDthinktank21
Lowitja Indigenous knowledge translation series
Croakey projects archive
#PHAAThinkTank 2022
Summer reading 2022-2023
#CommunityMatters
#CroakeyFundingDrive 2022
#CroakeyLIVE #Budget2021Health
#CroakeyLIVE #USvotesHealth
#CroakeyLIVE Federal election 2022
#CroakeyYOUTH
#HousingJusticeAus
#IndigenousHealthSummit
#IndigenousNCDs
#JustClimate
#JustJustice
#LookingLocal
#OutOfPocket
#OutOfTheBox
#RuralHealthJustice
#TalkingTeeth
@WePublicHealth2022
@WePublicHealth2021
@WePublicHealth2020
AroundTheTraps
Croakey register of influence
Croakey Register of Influencers in Public Health
Croakey Register of Unreleased Documents
Gavin Mooney
Inside Story
Journal Watch
Naked Doctor
Poems of Public Health
Summer reading 2021-2022
Summer reading 2020-2021
Summer Reading 2019-2020
Summer Reading 2017-2018
Summer Reading 2016-2017
The Koori Woman
TOO MUCH of a Good Thing
Wonky Health
CroakeyGO archive 2017 – 2018
CroakeyGo 2018
#CroakeyGO #QuantumWords 2018
#CroakeyGO #VicVotes 2018
#CroakeyGO Albury 2018
#CroakeyGO Callan Park 2018
#CroakeyGO Carnarvon 2018
#CroakeyGO Marrickville 2018
#CroakeyGO Palm Island 2018
CroakeyGo 2017
#CroakeyGO Adelaide 2017
#CroakeyGO Melbourne 2017
#CroakeyGO Newcastle 2017
#CroakeyGO Sydney 2017
Elections and Budgets 2013 – 2021
#QldVotesHealth 2020
Budget2020Health
Federal Budget 2020-21
Federal Budget 2019-20
#AusVotesHealth Twitter Festival 2019
#Health4NSW
Federal Election 2019
NSW Election 2019
Federal Budget 2018-19
Federal Budget 2017/18
NZ Election 2017
Federal Budget 2016-17
Federal Election 2016
#HealthElection16
NT Election 2016
Federal Budget 2015-16
Qld Election 2015
NSW Election 2015
Federal Budget 2014-15
Victorian Election 2014
Federal Budget 2013-14
Federal Election 2013
Federal Budget 2012-2013
Federal Budget 2011
Federal Budget 2010
Federal Election 2010
Federal Budget 2009-2010

Beyond the p(ale) — statistics and significance in clinical research

With a boom in predatory journals and “publish or perish” pressures more real than ever, truly significant findings are becoming ever-more difficult to discern, prompting proposals for a rethink on how we define new discoveries.

In an upcoming piece for Nature Human Behaviour, published on preprint this month, a global transdisciplinary group of researchers have called for the threshold on a common benchmark of significance — the p-value — to be dropped from 0.05 to 0.005, arguing that the former allows too many false positives, even in the most robust of studies.

Put simply, in research the p-value is an expression of how likely it is that your results are due to chance. Traditionally, p=<0.05 is said to be statistically significant, putting this likelihood at less than 5%, though some authors prefer p=<0.01 (less than 1%) as a threshold.

The Nature authors propose tossing these markers in favour of p=<0.005 (less than 0.5%) as a new benchmark in the social and biomedical sciences.

But will this address what’s been described as a “reproducibility crisis” in science?

ben1

ben2ben3ben4Shlomo Argamon, Professor of Computer Science and Director of the Master of Data Science at the Illinois Institute of Technology, argues that it may in fact entrench woes within the current system, and has called for a rethink on how we approach the publication of research, in a series of blog posts republished with permission for Croakey readers.

(And if stats and bias is your thing, you might be interested in a recent proposal in the Annals of Internal Medicine introducing an ‘e-value’ to measure causality and confounding factors in observational studies).


Shlomo Argamon writes:

Several days ago, an article was posted on PsyArXiv (to appear in Nature Human Behavior), arguing that a significant step towards solving the reproducibility crisis in many fields will be to reduce the standard significance threshold to require p < 0.005 (rather than the current standard of 0.05) for a result to be termed “statistically significant”. Results with higher p values could only be termed “suggestive” under this proposal.

I, along with a number of other scientists, was recently interviewed for a brief article in Nature | News which clearly, and concisely, sets out arguments for and against this proposal as a way to address the reproducibility crisis. I’d like to expand here on my very compressed comments as quoted in that article.

On the one hand, the proposal to reduce the “significance threshold” to 0.005 will indeed be helpful, in that the change will reduce the occurrence of false positives, and thus enhance the likelihood that effects termed “significant” will be reproducible, and so reduce the number of non-existent effects reported in the literature as real.

However, the proposal does nothing whatsoever to change the fundamental problem, which is the entire framework of judging experimental results by a binary criterion of “significance”. There are two key problems with the regnant view of “statistical significance”:

  1. It does not account for effect size – a large effect with less-significant p may have much more “real-life significance” than a tiny effect with more-significant p, and
  2. It is very easy to (even inadvertently) “p-hack” and find putatively significant effects due to the very large number of possible parameter settings for any experiment in a sufficiently rich and theoretically-underdetermined domain.

Given that, this new recommendation may even do significant harm, in that improving research outcomes somewhat (by lowering false positive rates) without addressing the fundamental problem could lead to a false sense of security and entrench bad methodological attitudes.

Same old song

To expand on this, the problem I have with the new recommendation is that it maintains the exact same conceptual framework of statistical significance testing that has been a major driver of the replicability crisis.

In statistical significance testing, a level of confidence (previously 0.05, per the new proposal 0.005) is defined, and then the question asked is: Is the probability that the null hypothesis produced the experimental results less than that confidence level? If it is, the null hypothesis is rejected, and the hypothesis is considered statistically significant.

This framework does not account at all for the question of effect size – an effect may be statistically significant, but practically speaking negligible and irrelevant.

For example, often the null hypothesis is of the form “x = 0” for some model parameter x. Any such null hypothesis is almost certainly false, since the actual value of x is probably at least slightly different from 0, and so some statistically significant result can nearly always be extracted with enough data and ingenuity, even though the result may not be significant in any meaningful way in the real world.

Forking paths and the file drawer effect

More fundamentally, no matter what confidence level you choose, if there are enough different ways to design your experiment, it becomes highly likely that at least one of them will give a statistically significant result just by chance. And unless every single detail of the experiment is decided and registered ahead of time, it is virtually impossible to ensure, except by care and good judgement by the researchers, that no choices are made along the way that could (unwittingly) enhance the significance of the final experimental results, since there are so many different choices.

This phenomenon was termed the “garden of forking paths” by Gelman and Loken, and illustrated nicely by XKCD. (This problem is inherent in any metric-based decision criterion for identifying effects, and is not specific to statistical significance testing.)

Finally, the goal and the strength of the “p < 0.005” proposal is that it will reduce the number of false positive results in the literature.

However, by the same token, there will be fewer reports of “less statistically significant” results, which will cause overall problems for the science, due to the “file-drawer” effect.

The problem is that the vast majority of negative results are never reported, which gives a biased picture of the state of knowledge. By lowering the p-threshold, the file-drawer bias will be even greater, since fewer results, with only lower p-values, are published.

So how should we proceed?

There is no simple answer to this question, as there are several interrelated problems: p-hacking, effect sizes, and literature bias, as I’ve mentioned above, but also the question of how the incentive structures of modern science* may bias the creation and evaluation of new knowledge (the file-drawer problem is an example of this).

All of these factors contribute to the “reproducibility crisis”, and all need to be addressed. Any simple answer to one of these, such as changing the threshold p-value, will not solve the problem, and may in fact make things worse by making it seem solved when it is not.

Any meaningful solution must involve a combination of new methodological standards, institutional structures, and incentives.

Reducing the p threshold is simple and could be one useful methodological standard, though it would perpetuate problematic concepts, as I’ve said. Another would be introducing some notions from Bayesian statistics, such as

  1. Using Bayes factors, which are more easily interpreted in most situations than a p-value, as a standard reported metric, or
  2. The use of explicit priors, to make assumptions clearer, or
  3. Explicit loss functions, to account explicitly for the importance of effect size and the varying weights of different kinds of errors.

It is essential that the scientific community have good incentives for publishing replications, especially those that fail, and for publishing negative results.  (And there cannot be penalties for early-stage researchers for doing so!)

There need to be more and better institutional structures for preregistering studies, like clinicaltrials.gov, but set up with the appropriate community standards and consequences, so that they actually work.

Just as during the early and laborious development of effective scientific practices and institutions several hundred years ago, there is no royal road to solving the crisis.

A great deal of hard work lies ahead of us, but past experience has shown that it will (eventually) pay off in more solid understanding of our world.

*See a follow-up post on perverse incentives of the new proposal here

Shlomo Argamon is a Professor of Computer Science and Director of the Master of Data Science at the Illinois Institute of Technology

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search by: Categories or tags

Filter by Categories
Aged care
Budgets
Federal Budget 2024-25
Federal Budget 2023-2024
Federal Budget October 2022
Federal Budget 2022-23
Federal Budget 2021-22
Climate and health
#HealthyCOP29
Climate emergency
Healthcare sustainability
Heatwaves
National Health and Climate Strategy
#HealthyCOP28
#HealthyCOP27
#HealthyCOP26
#CoveringClimateNow
COVID collection
COVID-19
Long COVID
COVIDwrap
COVID SNAPS
#JusticeCOVID
Caring for the Frontline
COVIDglobalMHseries
Croakey Conference News Service
#WICC2024
#HealthierSocieties
#DigitalDeterminants24
#iDEA24
#ASMIRT2024
#EACH24
#CHFtalks
#SARRAH2024
#GreenHealthForum24
Croakey Professional Services
Sponsored content
#IndigenousEvaluation
NHLF series
#KidneyCareTogether
ACSQHC series
ACSQHC series 2022
ACSQHC series 2021
ACSQHC series 2020
ACSQHC series 2019
CATSINaM 25 Years
Croakey projects
@WePublicHealth 2024
#ChurchillDeepEnd
Planetary Health Equity Hothouse 2024
Summer reading 2023-2024
The Zap
#CroakeyLIVE #DigitalNationBuilding
#CroakeyLIVE #VoiceForHealth
#PHAAThinkTank2023
The Health Wrap
ICYMI
@WePublicHealth
@WePublicHealth2023
#CroakeyVOICES
#SpeakingOurMinds
Croakey longreads
#CroakeyREAD
CroakeyEXPLORE
CroakeyGO
#CroakeyGO #NavigatingHealth
#GamblingHarms
#HeatwaveHealth
Mapping CroakeyGo
Determinants of health
Environmental determinants of health
Social determinants of health
Education
Discrimination
Housing
Internet access
Justice and policing
Justice Reinvestment
Newstart/JobSeeker
Poverty
Racism
Social policy
Commercial determinants of health
Alcohol
Arms industry
Digital platforms
Food and beverages
Fossil fuels
Gambling
Pharmaceutical industry
Plain packaging
Sugar tax
Tobacco
Vaping
Disasters and extreme weather events
Disasters
Extreme weather events
Bushfires
Bushfire-emergency 2019-2020
Floods 2023
Floods 2022
Floods 2021
Floods 2011
Donor-funded journalism
Donor-funded journalism – 2024
Donor-funded journalism – 2023
Donor-funded journalism – 2022
Donor-funded journalism – 2021
Donor-funded journalism – 2020
Elections
lutruwita/Tasmania 2024 election
NT election 2024
Queensland election 2024
United States election 2024
Federal election 2025
#NSWvotesHealth2023
Victorian election 2022
Federal Election 2022
The Election Wrap 2022
SA election 2022
WA election 2021
Tasmanian election 2021
First Nations
Indigenous health
Community controlled sector
Cultural determinants of health
Cultural safety
Indigenous education
Social and emotional wellbeing
Uluru Statement
The Voice
Lowitja Institute
NT Intervention
WA community closures
Acknowledgement
#CTG10
#NTRC
#RCIADIC30Years
General health matters
Abortion
Cancer
Cardiovascular disease
Child health
Chronic conditions
Consumer health matters
Death and dying
Diabetes
Disabilities
Euthanasia
Fetal Alcohol Spectrum Disorders (FASD)
Genetics
HIV/AIDS
HRT
Infectious diseases
Influenza
LGBTQIA+
Medical marijuana
Men's health
Mental health
Mpox
Non communicable diseases
Oral health
Organ transplants
Pain
Pregnancy and childbirth
Sexual health
Suicide
Swine flu
Trauma
Women's health
Youth health
Global health matters
Asylum seeker and refugee health
Conflict and war
Gaza conflict
Global health
WHO
Ebola
NHS
#WorldInTurmoil
Health policy and systems
Co-design
Health financing and costs
Health reform
Health regulation
Medicare 40 Years
MyMedicare
National Health Performance Authority
Pharmaceutical Benefits Scheme
Private health insurance
Royal Commissions
TGA
Workforce matters
Strengthening Medicare Taskforce 2022
National Commission of Audit 2014
Healthcare
Adverse events
Allied healthcare
Australian Medical Association
Choosing Wisely
cohealth
Complementary medicines
Conflicts of interest
Co-payments
Digital technology
E-health
Emergency departments and care
Equally Well
General practice
Health Care Homes
Health ethics
Hospitals
International medical graduates
Medicare Locals
MyHospitals website
Naturopathy
NDIS
Nursing and midwifery
Out of pocket costs
Palliative care
Paramedics
Pathology
Pharmacy
Primary healthcare
Primary Health Networks
Rural and remote health
Safety and quality of healthcare and aged care
Screening
Social prescribing
Surgery
Telehealth
Tests
Media and health
Media-related issues
Health & medical marketing
Misinformation and disinformation
Public interest journalism
Social media and healthcare
The Conversation
Media Doctor Australia
News about Croakey
Croakey Funding Drive June 2024
Public health and population health
Air pollution
Artificial intelligence
Australian Centre for Disease Control
Government 2.0
Gun control
Health communications
Health impact assessment
Health in All Policies
Health inequalities
Health literacy
Human rights
Illicit drugs
Injuries
Legal issues
Marriage equality
Nanny state
National Preventive Health Agency
Obesity
Occupational health
Physical activity
Prevention
Public health
Road safety
Sport
Transport
Vaccination
VicHealth
Violence
Web 2.0
Weight loss products
#PreventiveHealthStrategy
#UnmetNeedsinPublicHealth
Research matters
Cochrane Collaboration
Evidence-based issues
Health and medical education
Health and medical research
NHMRC
#MRFFtransparency
The Croakey Archives
#cripcroakey
#HealthEquity16
#HealthMatters
#IHMayDay (all years)
#IHMayDay 2014
#IHMayDay15
#IHMayday16
#IHMayDay17
#IHMayDay18
#LoveRural 2014
Croakey Conference News Service 2013 – 2023
2023 Conferences
#GreenHealthForum23
#hpsymposium2023
#NMS23
#HEAL2023
#ASMIRT2023
#NSPC23
Our Democracy Forum
#AskMSF
#Lowitja2023
2022 Conferences
#16nrhc
#GreenHealthForum22
#Heal2022
#ICEM22
#NAISA22
#NNF2022
#RANZCP2022
#RethinkAddiction
#RTP22
GiantSteps22
Equally Well 2022 Symposium
Choosing Wisely National Meeting 2022
2021 conferences
#21OPCC
#BackToTheFire
#FoodGovernance2021
#GiantSteps21
#GreenHealthForum21
#HealthClimateSolutions21
#HearMe21
#IndigenousClimateJustice21
#NNF2021
#RANZCP2021
#ShiftingGearsSummit
#ValueBasedCare
#WCepi2021
#YHFSummit
2020 conferences
#2020ResearchExcellence
#Govern4Health
#HealthReImagined
#SAHeapsUnfair
2019 Conferences
#ACEM19
#CPHCE19
#EquallyWellAust
#GiantSteps19
#HealthAdvocacyWIM
#KTthatWorks
#LowitjaConf2019
#MHAgeing
#NNF2019
#OKtoAsk2019
#RANZCOG19
#RANZCP2019
#ruralhealthconf
#VMIAC2019
#WHOcollabAHPRA
2018 conferences
#6rrhss
#ACEM18
#AHPA2018
#ATSISPC18
#CPHCE
#MHED18
#NDISMentalHealth
#Nurseforce
#OKToAsk2018
#RANZCOG18
#ResearchIntoPolicy
#VHAawards
#VMIACAwards18
#WISPC18
2017 conferences
#17APCC
#ACEM17
#AIDAconf2017
#BTH20
#CATSINaM17
#ClimateHealthStrategy
#IAHAConf17
#IDS17
#LBQWHC17
#LivingOurWay
#OKtoAskAu
#OTCC2017
#ResearchTranslation17
#TheMHS2017
#VMIACConf17
#WCPH2017
Australian Palliative Care Conference
2016 conferences
#AHHAsim16
#AHMRC16
#ANROWS2016
#ATSISPEP
#AusCanIndigenousWellness
#cphce2016
#CPHCEforum16
#CRANAplus2016
#IAMRA2016
#LowitjaConf2016
#PreventObesity16
#TowardsRecovery
#VMIAC16
#WearablesCEH
#WICC2016
2015 conferences
#CPHCEforum
#CRANAplus15
#HSR15
#NRHC15
#OTCC15
Population Health Congress 2015
2014 conferences
#IPCHIV14
AIDA Conference 2014
Congress Lowitja 2014
CRANAplus conference 2014
Cultural Solutions - Healing Foundation forum 2014
Lowitja Institute Continuous Quality Improvement conference 2014
National Suicide Prevention Conference 2014
Racism and children/youth health symposium 2014
Rural & Remote Health Scientific Symposium 2014
2013 conferences
Australian Centre for Health Services Innovation Forum 2013
Australian Health Promotion Association Conference 2013
Closing the Credibility Gap 2013
CRANAplus Conference 2013
FASD Conference 2013
Health Workforce Australia 2013
International Health Literacy Network Conference 2013
NACCHO Summit 2013
National Rural Health Conference 2013
Oceania EcoHealth Symposium 2013
PHAA conference 2013
Croakey Professional Services archive
#CommunityControl
#CommunityControl Twitter Festival
#COVIDthinktank21
Healthdirect Australia series 2019
Lowitja Indigenous knowledge translation series
Croakey projects archive
#PHAAThinkTank 2022
Summer reading 2022-2023
#CommunityMatters
#CroakeyFundingDrive 2022
#CroakeyLIVE #Budget2021Health
#CroakeyLIVE #USvotesHealth
#CroakeyLIVE Federal election 2022
#CroakeyYOUTH
#HousingJusticeAus
#IndigenousHealthSummit
#IndigenousNCDs
#JustClimate
#JustJustice
#LookingLocal
#OutOfPocket
#OutOfTheBox
#RuralHealthJustice
#TalkingTeeth
@WePublicHealth2022
@WePublicHealth2021
@WePublicHealth2020
AroundTheTraps
Croakey register of influence
Croakey Register of Influencers in Public Health
Croakey Register of Unreleased Documents
Gavin Mooney
Inside Story
Journal Watch
Naked Doctor
Poems of Public Health
Summer reading 2021-2022
Summer reading 2020-2021
Summer Reading 2019-2020
Summer Reading 2017-2018
Summer Reading 2016-2017
The Koori Woman
TOO MUCH of a Good Thing
Wonky Health
CroakeyGO archive 2017 – 2018
CroakeyGo 2018
#CroakeyGO #QuantumWords 2018
#CroakeyGO #VicVotes 2018
#CroakeyGO Albury 2018
#CroakeyGO Callan Park 2018
#CroakeyGO Carnarvon 2018
#CroakeyGO Marrickville 2018
#CroakeyGO Palm Island 2018
CroakeyGo 2017
#CroakeyGO Adelaide 2017
#CroakeyGO Melbourne 2017
#CroakeyGO Newcastle 2017
#CroakeyGO Sydney 2017
Elections and Budgets 2013 – 2021
#QldVotesHealth 2020
Budget2020Health
Federal Budget 2020-21
Federal Budget 2019-20
#AusVotesHealth Twitter Festival 2019
#Health4NSW
Federal Election 2019
NSW Election 2019
Federal Budget 2018-19
Federal Budget 2017/18
NZ Election 2017
Federal Budget 2016-17
Federal Election 2016
#HealthElection16
NT Election 2016
Federal Budget 2015-16
Qld Election 2015
NSW Election 2015
Federal Budget 2014-15
Victorian Election 2014
Federal Budget 2013-14
Federal Election 2013
Federal Budget 2012-2013
Federal Budget 2011
Federal Budget 2010
Federal Election 2010
Federal Budget 2009-2010

Search the Croakey Archives

Archives
Facebook
X/Twitter
LinkedIn