Séminaire du 23 février 2023

Flora Lysen (Maastricht University STS research Group, Maastricht University)

“Examining patient perspectives on artificial intelligence in health care”

23 February 2023

---

---

During the seminar, Flora Lysen presented her contribution to the RAIDIO project. The first part of her presentation aimed at introducing the project. The second part focused on her historical research. The third part focused on patient engagement and artificial intelligence.

---

The RAIDIO project (2020-2024) aims at examining how artificial intelligence can be responsibly used and integrated in image-based medicine (radiology and pathology). It includes a focus on the issues of expertise, responsibility, trust and patient perspectives.

This project studies artificial intelligence in image-based medicine through an interdisciplinary methodology:

• • Medical ethics, bioethics and philosophy
  • Social studies of science and medicine (qualitative interviews, film ethnography of the digitalization of pathology)
  • Narrative medicine and discourse analysis
  • Historical research (a history of automation and computation in medical imaging)

At the beginning of the project (2020), the team mapped the field of artificial intelligence in image-based medicine through discourse analysis. They were able to highlight two trends: discourses framing artificial intelligence as a competitor to medical professionals, and a critique of these overpromises of artificial intelligence. This critique increased with the Covid-19 pandemic. Indeed, at that time, artificial intelligence was hailed as a crucial support tool for overburdened health care workers. However, many reports quickly showed that this could not be fulfilled. Artificial intelligence might not work, and when it does, it could be highly biased.

Recently, an ethnographic film called Samples to Slide: labor and craft in pathology was created by Megan Milota and Jojanneke Drogt. It analyzes workflows in the field of digital pathology. Digital pathology is a recent subfield of pathology focusing on data management based on information generated from digitized pathological slides. The filming was done in Nijmegen, in Radboudumc’s pathology laboratory. 

Firstly, the film highlights the importance of manual craft in digital pathology (cutting specimens, putting them in paraffin, cooling paraffin blocks, making microtomic slides out of them). As a result, it conveys a representation of artificial intelligence richer than existing stereotypical images. Secondly, it stands against the exceptionalism of artificial intelligence. Indeed, in this laboratory, artificial intelligence forms part of a much longer history of automation. Thirdly, it questions the idea that artificial intelligence could replace tedious elements of labor in the laboratory. Indeed there is a long history of alternating tasks in medical laboratories. What does it mean that artificial intelligence needs to replace some of these practices? Fourthly it questions the gendered aspects of medical laboratory work (most of the people working in the laboratory are women).

Flora Lysen’s current research on the history of the first mass screening of Pap-smear in the Netherlands in the 1970s relates to some points raised by the film. Indeed, before the campaign started, there was a huge political effort to think about the benefits of mass screening. In particular, there was a fear within the Dutch government that such mass screening would increase tedious work. As a consequence, the discussion fell into a broader debate about automation in the Netherlands. Was bulk image reading the kind of profession that one wanted to create?

However, people working on these mass campaigns pointed out that this was an emancipation gesture. In fact, it could create work for women who would otherwise not work. Thus, in the early literature, bulk image reading was designated as a female profession. These women reading images also divided tasks in the laboratory to avoid making their work tedious.

---

Flora Lysen’s historical work within the RAIDIO project focuses on radiology. She recently submitted it to the British Journal for the History of Science (“The fallible expert radiologist: X-ray reading and the ‘error problem,’1947-1960”). Her research work seeks to trace the history of an argument often used to justify the deployment of artificial intelligence in radiology. This argument is that artificial intelligence is needed because medical professionals make errors.

According to Flora Lysen, the idea of the fallible expert radiologist emerged at the end of the 1940s. This was the time of mass X-ray screening campaigns for tuberculosis and pneumoconiosis. In 1947, the United States Department of Veterans Affairs commissioned an evaluation of the efficiency of these campaigns. Shortly after, in 1949, observer variability and observer error were brought to the debate by various reports showing that radiologists often disagreed with each other or with their own previous evaluations (sometimes up to 20 and 30%). Therefore, it appeared that health care efficiency could only be improved by mitigating these errors.

Various solutions were developed for this purpose, but one became prominent in the 1950s: the rationalization of clinical decision-making. In a Science article from 1959, Robert S. Ledley (a radiologist) and an engineer (Lee B. Lusted) were one of the first to propose a formalization of the diagnostic reasoning process. It was inspired by game theory and decision-making theory. These works on the rationalization of medical decision-making reframed the fallible observer as a suboptimal decision maker. The minds of doctors were thought to work similarly to a logical model. However, doctors were perceived as underachieving in their logical capacities and thus needed to be helped by rationalizing procedures.

At that time, radiology was seen as a ‘testing ground’ for the search for an optimal procedure. These researches eventually led to the first attempts at automating image reading (e.g., coded analysis, pattern-recognition machines). Technologies developed for this purpose after the 1950s, such as computer-aided detection and diagnosis (CAD and CADx, 1960s) and artificial neural networks (1970s-today) are tied to this discourse. 

Another solution proposed in the 1950s to mitigate for the human factor which received equal attention was to restructure the reading labor, in particular through double-reading of X-rays by lower-skilled readers, mostly women (a cost-effective alternative to double reading by radiologists). Because of mistrust of lower-skilled readers as part of a demarcation of radiological expertise, this solution was rarely implemented.

Flora Lysen’s historical analysis of the human factor problem in X-ray image reading reveals a rhetorical association between image reading technologies and the idea of the fallible expert. According to her, this pairing took a new turn in the 1990s. This is when CAD started to be proposed as a way to realize double-reading procedures in the context of mass screening. The fact is that CAD was not presented as a way to replace radiologists in image reading, but as a not-yet-perfected assistant to the imperfect X-ray reading expert. Artificial intelligence is presented today in this very way. According to Flora Lysen, this idea of computer-assisted procedures as second readers helps sustain interest in these technologies when evidence that they actually improve clinical accuracy is still pending.

---

Flora Lysen’s current research questions patient engagement in artificial intelligence research. The publicizing of patient engagement is part of that of the participatory turn in medicine. This movement denotes a shift from a paternalistic relationship between patients and doctors to a more egalitarian one where patients participate in care practices.

For this movement, patient participation often goes hand in hand with patient empowerment. According to Barbara Prainsack, patient empowerment has various meanings, goals and values (Prainsack 2017):

• • Individualistic empowerment (focus on individual autonomy)
  • Instrumental empowerment (helps innovation)
  • Democratic empowerment (value of democracy for equality)
  • Emancipatory empowerment (liberating people from oppressive structures)

There is, however, a discrepancy between the rhetoric of a patient participation ideal and how it is actually achieved. Critics are as follows:

• • “Participation washing”
  • Patient participation is often ill-defined
  • Patient participation rhetoric may lead to misleading expectations in patients
  • Often no financial resources to compensate for patients’ time
  • Efficacy and outcomes of patient participation are not consistently assessed
  • Participation discourses cause “responsabilisation” of patients in health
  • Importance of rethinking common assumptions about participation (e.g., patients may not want to be in control of their health)

Despite these criticisms and due to policies, the participation field is undergoing professionalization and infrastructurization. The field of artificial intelligence in health care is no exception to this trend: there is a growing literature focusing on patients’ perspectives, views or even attitudes towards artificial intelligence.

According to Flora Lysen, these studies are mostly quantitative studies using survey methods to assess patients’ agreement with certain statements. She also observed that:

• • Some studies address particular respondent groups (e.g., diabetes patients) while others address broader groups (e.g., adult patients) and that it is not always clear why a specific group was chosen
  • The purpose of the research and the use of its outcomes are most often left implicit
  • Often, research is meant to measure the acceptability of artificial intelligence: there seems to be a conflation between acceptability and trustworthiness and patients should not be turned into ethical traffic controllers

She then exposes her preliminary directions for future research on patient participation in artificial intelligence:

• • Qualitative research may supplement quantitative research to better understand how patients relate to a complex issue such as artificial intelligence
  • Asking patients about artificial intelligence may be a subtopic in asking patients about their experiences of care
  • Include a consideration of the broader social, political and economic project that shapes artificial intelligence, as well as a more explicit engagement with values that are enacted through co-design practices
  • Focus on the “perceptions and attitudes” of developers and physicians as well as on their interactions, because this is where significant decisions are made
  • Investigate patient engagement “in action” by examining case studies of patient advocacy in relation to developments in big data and artificial intelligence

---

Published results of the RAIDIO projet (March 2023)