The epistemic and pragmatic function of dichotomous claims based on statistical hypothesis tests

Uygun Tunç D., Tunç M. N., Lakens D.

Theory and Psychology, vol.33, no.3, pp.403-423, 2023 (SSCI) identifier identifier

  • Publication Type: Article / Article
  • Volume: 33 Issue: 3
  • Publication Date: 2023
  • Doi Number: 10.1177/09593543231160112
  • Journal Name: Theory and Psychology
  • Journal Indexes: Social Sciences Citation Index (SSCI), Scopus, Academic Search Premier, IBZ Online, International Bibliography of Social Sciences, Periodicals Index Online, Educational research abstracts (ERA), Linguistics & Language Behavior Abstracts, Psycinfo, Sociological abstracts
  • Page Numbers: pp.403-423
  • Keywords: falsification, inference, philosophy of science, statistical decision, testing, P-VALUES, DECISION-THEORY, INFERENCE, PEARSON, NEYMAN
  • Middle East Technical University Affiliated: Yes


Researchers commonly make dichotomous claims based on continuous test statistics. Many have branded the practice as a misuse of statistics and criticize scientists for the widespread application of hypothesis tests to tentatively reject a hypothesis (or not) depending on whether a p-value is below or above an alpha level. Although dichotomous claims are rarely explicitly defended, we argue they play an important epistemological and pragmatic role in science. The epistemological function of dichotomous claims consists in transforming data into quasibasic statements, which are tentatively accepted singular facts that can corroborate or falsify theoretical claims. This transformation requires a prespecified methodological decision procedure such as Neyman-Pearson hypothesis tests. From the perspective of methodological falsificationism these decision procedures are necessary, as probabilistic statements (e.g., continuous test statistics) cannot function as falsifiers of substantive hypotheses. The pragmatic function of dichotomous claims is to facilitate scrutiny and criticism among peers by generating contestable claims, a process referred to by Popper as “conjectures and refutations.” We speculate about how the surprisingly widespread use of a 5% alpha level might have facilitated this pragmatic function. Abandoning dichotomous claims, for example because researchers commonly misuse p-values, would sacrifice their crucial epistemic and pragmatic functions.