Open Science in Higher Education: Replication, Reproducibility, and Research Policy
48th Annual EAIR Forum 2026, Vilniaus, Litvanya, 26 - 28 Ağustos 2026, (Özet Bildiri)
- Yayın Türü: Bildiri / Özet Bildiri
- Basıldığı Şehir: Vilniaus
- Basıldığı Ülke: Litvanya
- Orta Doğu Teknik Üniversitesi Adresli: Evet
Özet
Concerns about research credibility have become more visible in the age of generative artificial intelligence, as tools can lower the barrier for problematic practices such as fabricated references or misrepresented analyses. Yet, high-profile misconduct cases in the social sciences existed before recent technologies. For example, the Stapel case illustrated how limited transparency and weak verification practices can allow systemic problems to persist across a large body of published work raising questions about how research policies and incentives shape integrity and misconduct (Markowitz & Hancock, 2014; Stanford University, n.d.). Related debates have linked these risks to publication systems that reward statistically significant results and novelty which potentially amplify publication bias and discouraging null or complex findings (Francis, 2012; Pashler & Wagenmakers, 2012). At the same time, calls for organized openness and institutionalized verification have also faced resistance including arguments that transparency requirements may be demanding or could destroy public trust (Vazire, 2017).
Across the social sciences, approaches to replication and reproducibility have developed unevenly. Some fields such as economics and finance have moved further toward formalized expectations for sharing data and code, including journal-level practices intended to support cumulative and credible research (Christensen, 2016). However, higher education research occupies an interdisciplinary space and has often been described as fragmented and weakly cumulative with diverse epistemic traditions and limited consolidation of evidence across studies (Macfarlane & Grant, 2012; Tight, 2012). Within this context, clarification of core concepts is important. Replication can be understood as directly testing an original study’s claims using new data and the same methodological specifications while reproducibility concerns reexamining results using the same data and procedures (Dreber & Johannesson, 2025). Large-scale discussions in the broader social and behavioral sciences have emphasized that even studies that appear methodologically rigorous may fail to reproduce due to random or systematic error highlighting replication as one mechanism for independent verification (Open Science Collaboration, 2015). Initiatives such as the Transparency and Openness Promotion work have similarly framed replication and analytic transparency as complementary pathways to improving credibility (Nosek et al., 2015). At the same time, replication can generate cultural and epistemological tensions within disciplines, sometimes producing defensive reactions or concerns about policing rather than knowledge building (Peterson & Panofsky, 2021).
Against this background, this study examines how replication and reproducibility are institutionalized, enabled, and framed within higher education research. It adopts a qualitative, multisource document and discourse analysis of three interconnected layers as publication practices, journal transparency policies, and editorial discourse. The overarching question is how leading higher education journals publish, enable, and frame replication and reproducibility as components of cumulative science. Three subquestions guide the analysis by investigating what is labelled or treated as replication in published work, what is enabled through data and code sharing policies, and how these practices are justified and interpreted in editorial and policy texts. Methodologically, the publication mapping component of this study replicates established procedures used to investigate replication work in education research (Makel et al., 2012; Makel &Plucker, 2014). This paper extends the approach to more recent years and complementing it with an analysis of journal policies and editorial discourse. This design allows the study to move beyond counting replication outputs by showing how journals and other intermediary actors define, legitimize, or marginalize replication and reproducibility in practice.
Findings suggest that replication remains a marginal and unevenly institutionalized practice in higher education research. Descriptively, across four leading journals (2014 to 2025), very few papers display characteristics consistent with replication, and even fewer conduct systematic tests of prior findings. Policy texts tend to emphasize transparency, rigor, and ethical standards, but rarely articulate replication as a valued scholarly contribution. Editorial discourse often highlights diversity, context dependence, and conceptual innovation, shaping a field level understanding of knowledge production in ways that may leave replication conceptually underspecified and institutionally unsupported. Taken together, the analysis speaks to how research governance in higher education can strengthen credibility and trust by clarifying expectations, incentives, and infrastructures for cumulative evidence.