Evidential Pluralism Mitigates Epistemic Injustice in Evidence-Based Evaluation

Today's post comes from Dr Alexandra Trofimov and summarises her talk at part of the Manchester Online Seminars on Evidential Pluralism titled "How Evidential Pluralism mitigates epistemic injustice in evidence-based evaluation", which will take place on the 14th of April, 2025, at 3pm BST. You can register and see the rest of the talks in the series here. 

Evidence-based evaluation aims to ensure that interventions, such as medical treatments or public policies, effectively achieve the ends at which they aim. But what kind of evidence should be considered? Orthodox evidence-based evaluation advocates an exclusive focus on comparative studies. In contrast, Evidential Pluralism advocates an inclusive approach to evidence and has been defended on the grounds that it enables more robust evaluations. In an upcoming talk, I offer a new line of argument: that Evidential Pluralism helps to mitigate epistemic injustices that result from orthodox evidence-based evaluation.

Orthodox evidence-based evaluation

Orthodox evidence-based evaluation has created an influential stereotype of ‘good evidence’ that, I argue, results in epistemic injustice.

Orthodox evidence-based evaluation is characterised by evaluation methods and evidence hierarchies that focus almost exclusively on comparative studies, especially randomized controlled trails (RCTs). This is most explicitly captured in the evidence hierarchies of evidence-based medicine, which place systematic reviews and meta-analyses of RCTs are at the top of the hierarchy, followed by individual RCTs and then observational studies. All other kinds of studies are relegated to a lower position on the hierarchy and, in practice, typically excluded from consideration. Similarly, the UK Government’s Magenta Book, which provides evaluation guidance for all Government policy and regulatory interventions, prioritises the use of experimental or quasi-experimental studies and advocates the use of other kinds of study only when these are unavailable.

Consider, for example, the controversy surrounding the effectiveness of face mask mandates during the Covid-19 pandemic. A highly influential Cochrane Collaboration systematic review that included only RCTs found no clear reduction in infections from face masks. The authors of the review concluded that, due to the various limitations of the included RCTs, firm conclusions could not be drawn. They called for further, well-designed RCTs to address the uncertainty. Many, however, mistakenly interpreted the review to mean ‘masks did nothing’. There were also dismissals of non-RCT evidence. A review that included a wide range of evidence found face masks can effectively reduce transmission. The review was criticised for being non-systematic and recommending public policies in the absence of RCT evidence.

The extent to which RCT evidence was prioritised and other kinds of evidence were devalued or dismissed is, I argue, indicative of prejudice and results in both testimonial injustice and hermeneutical injustice.

Evidential Pluralism

Addressing the epistemic injustice that results from orthodox evidence-based evaluation is crucial; it is a significant wrong that can have disastrous practical consequences. In the case of Covid-19 face masks, it led to policy delays that likely cost ‘thousands of lives’. 

Evidential Pluralism offers a solution. According to Evidential, establishing a causal claim requires establishing that the putative cause and effect are both correlated and connected by an appropriate mechanism. This, in turn, will typically require consideration of a broad range of evidence types. Evidential Pluralism takes each item of evidence on its own merits. It does not commit to pre-determined rankings of evidence or preclude certain types of studies from providing high quality evidence simply in virtue of their methodology. 

The benefits of Evidential Pluralism are illustrated in a proof-of concept evaluation of Covid-19 face mask mandates. The evaluation included a broad range of evidence types and concluded that Covid-19 face mask mandates were effective at reducing infections, hospitalisations and deaths.

By advocating a more inclusive approach to evidence, Evidential Pluralism enables more robust evaluations. It also avoids evidence being prejudicially devalued or dismissed and therefore avoids the epistemic injustice that results from orthodox evidence-based evaluation’s stereotype of ‘good evidence’.

The fact that Evidential Pluralism is able to mitigate epistemic injustice supports its adoption as a new methodology for evidence-based evaluation.

Alexandra Trofimov is a research associate in philosophy at the University of Manchester. Her research interests are in moral philosophy, legal philosophy and normative epistemology. She is currently working with Professor Jon Williamson on a Leverhulme Trust funded project ‘EBL+: New Philosophical Foundations for Evidence-Based Law’. Through an application of Evidential Pluralism to evidence-based law, the project aims to provide a new methodology for evidence evaluation to improve the effectiveness of legislative and regulatory interventions. Alexandra is also interested in improving evidence evaluation more broadly, including in public policy and public health.