Causal discovery for automated real-world educational evidence extraction

Abstract

There is increasing demand to shift from intuition- and experience-based practices to evidence-based education. However, extracting meaningful evidence from real-world educational data poses significant challenges. Traditional approaches to evidence generation, such as randomized controlled trials and systematic reviews, face limitations in both the medical and educational domains due to high costs and ethical constraints. In response, the concept of real-world evidence has emerged as a promising alternative, particularly in medicine and, more recently, in education. Although this approach may be less robust than traditional methods, it offers the potential to uncover broad and practical insights from naturally occurring data. This study explores the use of deep learning for causal discovery in real-world educational data. Specifically, we apply Structural Agnostic Modeling, a method previously validated in biological datasets, to identify underlying causal relationships. In Study 1, we compare this data-driven approach to a traditional hypothesis-driven method. The results demonstrate that this technique can generate both interpretable and novel causal hypotheses, although it occasionally produces plausible relationships in the reverse direction. To address this limitation, we propose an enhanced model, SAM+, in Study 2. Our findings indicate that SAM+ effectively mitigates the identified shortcomings. This research contributes a new methodology for leveraging large-scale educational data and opens new possibilities for advancing evidence-based education.