CE Workshop 09: Cast Aside Traditional Notions of Statistical Significance, and Focus Instead on Characterizing the Magnitude of Effects that are Clinically or Scientifically Relevant
Robert Ploutz-Snyder, PhD, PStat(r)
Research Professor, Interim Associate Dean of Research and Scholarship
University of Michigan, School of Nursing
Credit Hours: 1.5
Level of Instruction: Introductory
CE Workshop 09: Cast Aside Traditional Notions of Statistical Significance, and Focus Instead on Characterizing the Magnitude of Effects that are Clinically or Scientifically Relevant
Abstract & Learning Objectives:
There is an ongoing debate among statisticians and discipline scientists about the consequences of our persistent, dogmatic reliance on evaluating all statistical results as meaningful if and only if "p<0.05," regardless of context. This was never the intended goal of Ronald Fisher, nevertheless scientists have adopted it as a convenience, and the decades long dependence on "p<0.05" has had important negative consequences. In this presentation, I review common misconceptions about interpreting p-values, why we should consider de-emphasizing p-values, and why scientists should rely more on practical, clinical, or scientifically meaningful differences over arbitrary cut-offs. I will present several different metrics for evaluating and reporting effect magnitude, and whether or not data support the null vs. alternative hypothesis, under the frequentist paradigm, how Bayesian methods can augment or replace frequentist analyses, and a few options that help to clarify how important a finding may be. Throughout this talk, I advocate that discipline scientists take charge of sharing scientific results that are not based merely on arbitrary p-value cutoffs and other default logic, but instead based on their content expertise, in light of all of the specific relevant aspects of experimental design and experimental data, balancing the consequences of Type I vs Type II errors appropriately, and focusing on characterizing effects, rather than dichotomizing research into only two categories of importance (significant vs. not).
Upon conclusion of this course, learners will be able to:
1. Discuss what p-values mean and how they are commonly misinterpreted.
2. Explain the leading arguments promoted by the American Statistical Association with regard to why science should carefully reconsider if and how p-values should continue to dominate our decisions about what research should be published, and how scientists should be evaluating its worth.
3. Apply new practices in how to evaluate and publish their own research, as well as how to evaluate research appearing in peer-reviewed journals, whether as consumers, reviewers, or editors.
Speaker Biography:
Dr. Robert Ploutz-Snyder is a Research Professor, Interim Associate Dean for Research and Accredited Professional Statistician through the American Statistical Association. He also directs an Applied Biostatistics Lab and Data Management Core within the School of Nursing at the University of Michigan. Prior to this appointment, he was a Senior Biostatistician supporting the NASA Human Research Program at the Johnson Space Center, and prior to that an Associate Professor of Medicine at SUNY Upstate Medical University. Dr. Ploutz-Snyder is a strong advocate of Team Science, where scientists from diverse fields work closely together to answer complex problems. His personal interests are in methods appropriate for getting the most information out of prospective small-n research, often involving longitudinal experimental designs, and typically with experimental constraints that contribute to missing data or other methodological issues.