Statistical Inference as Severe Testing How to Get Beyond the Statistics Wars

Mounting failures of replication in social and biological sciences give a new urgency to critically appraising proposed reforms. This book pulls back the cover on disagreements between experts charged with restoring integrity to science. It denies two pervasive views of the role of probability in inference: to assign degrees of belief, and to control error rates in a long run. If statistical consumers are unaware of assumptions behind rival evidence reforms, they can't scrutinize the consequences that affect them (in personalized medicine, psychology, etc.). The book sets sail with a simple tool: if little has been done to rule out flaws in inferring a claim, then it has not passed a severe test. Many methods advocated by data experts do not stand up to severe scrutiny and are in tension with successful strategies for blocking or accounting for cherry picking and selective reporting. Through a series of excursions and exhibits, the philosophy and history of inductive inference come alive. Philosophical tools are put to work to solve problems about science and pseudoscience, induction and falsification.

Preface; Excursion 1. How to Tell What's True about Statistical Inference: Tour I. Beyond probabilism and performance; Tour II. Error probing tools vs. logics of evidence; Excursion 2. Taboos of Induction and Falsification: Tour I. Induction and confirmation; Tour II. Falsification, pseudoscience, induction; Excursion 3. Statistical Tests and Scientific Inference: Tour I. Ingenious and severe tests; Tour II. It's the methods, stupid; Tour III. Capability and severity: deeper concepts; Excursion 4. Objectivity and Auditing: Tour I. The myth of 'the myth of objectivity'; Tour II. Rejection fallacies: whose exaggerating what?; Tour III. Auditing: biasing selection effects and randomization; Tour IV. More auditing: objectivity and model checking; Excursion 5. Power and Severity: Tour I. Power: pre-data and post-data; Tour II. How not to corrupt power; Tour III. Deconstructing the N-P vs. Fisher debates; Excursion 6. (Probabilist) Foundations Lost, (Probative) Foundations Found: Tour I. What ever happened to Bayesian foundations?; Tour II. Pragmatic and error statistical Bayesians; Souvenir (Z) farewell; References; Index.

Deborah G. Mayo is Professor Emerita in the Department of Philosophy at Virginia Tech. Author of Error and the Growth of Experimental Knowledge (1996), she won the 1998 Lakatos Prize for an outstanding contribution to philosophy of science. She directed the NEH Summer Seminar (1999) on Philosophy of Experimental Inference. She co-founded, with G. W. Chatfield, the Fund for Experimental Reasoning, Reliability and Objectivity and Rationality (E.R.R.O.R) in 2006 which has co-sponsored 10 conferences, workshops and distinguished lecture series. She's a visiting professor at the London School of Economics and Political Science, Centre for the Philosophy of Natural and Social Science (CPNSS) (2007–present).