“Algorithm Auditing? Meet Design Thinking!” – Daniël Everts

Right now, I am just finishing up on a data-driven research project on algorithmic bias in the Dutch job seeking process. Recently, a question arose amongst the members of the research team: how might such a project actually yield the knowledge necessary to exact any real change in practice? It is a question that had been occupying my mind for some time when Jon McKenzie of Cornell University presented his Design Thinking (DT) pedagogy during the latest Transmission in Motion seminar and, in doing so, revealed to me the way forward.

Data-driven research

The research project I refer to was aimed at exploring to what extent certain Dutch job-listing websites discriminate against so-called ‘protected classes’ – groups constituted by characteristics that employees are by law prohibited to let weigh in on their recruitment process; characteristics such as ethnicity, sex, gender, and age (d’Alessandro, O’Neil, and LaGatta 2017). The focus of our research project was the extent to which different applicants might be fed different information on the basis of protected attributes, even as they would search using the same terms.

In an attempt to find, we provided the websites’ search engines with specific search terms using accounts defined by varying protected attributes. Subsequently, we examined the output provided by the search engines. Eventually, we found no evidence to support the idea that these search algorithms changed their output on the basis of protected attributes linked to the profiles.

You know nothing…    

So far, good news. I wonder, however, whether it would have mattered if we had found any such evidence. Yes, we now knew a bit about the exact functionality of specific algorithms, but we knew little, for instance, about how said algorithms had come into existence. What considerations had gone into the creation of these algorithms? How did considerations of fairness and algorithms’ obvious requirement of effectiveness (Kearns and Roth 2020) play a role? If we did not know anything about the circumstances of the algorithms’ genesis, how could we ever hope to exact any kind of meaningful change if we would have deemed it necessary?

Design Thinking  

Enter Jon McKenzie and his Design Thinking (DT) pedagogy, which demands of its academically trained practitioners to ‘interface’ with the outside world – in McKenzie’s words, it is a pedagogy that puts the episteme into contact with the doxa (McKenzie 2019, 110). In simpler terms, it means taking the world outside of academia into account in such a way that, during research, one treats the outside world not as an audience, but as a stakeholder; a partner. Those subscribing to the DT method are not just involved in research, but rather in a comprehensive design process (Ibid., 127) that – to put it very briefly – includes conducting ethnographic work to properly gauge the practical field which one normally only researches from afar (Ibid., 128), defining problems based on that first exploration (Ibid., 129), conducting research and basing solutions on those problems, which one then also actually tests in the field (Ibid., 130–132).

To me, this seems like a perfect way to make any future research I conduct on algorithmic bias relevant and useful where it counts; there where they are created. Thus, to my fellow data-driven researchers I say: let us take McKenzie’s DT pedagogy to heart. Today, algorithms are intertwined with nearly all other aspects of our everyday lives and can no longer be understood as isolated pieces of computer code (Seaver 2017, 5). Let us too leave our data-driven, algorithmically focused isolation. Let us begin to ‘interface’ with everyday life as well.

References

• Alessandro, Brian d’, Cathy O’Neil, and Tom LaGatta. 2017. “Conscientious Classification: A Data Scientist’s Guide to Discrimination-Aware Classification.” Big Data 5 (2): 120–34. https://doi.org/10.1089/big.2016.0048.
• Kearns, Michael, and Aaron Roth. 2020. “Algorithmic Fairness: From Parity to Pareto.” In The Ethical Algorithm: The Science of Socially Aware Algorithm Design, 57–93. Oxford: Oxford University Press.
• McKenzie, Jon. 2019. “Becoming Cosmographer: Co-Designing Worlds.” In Transmedia Knowledge for Liberal Arts and Community Engagement, by Jon McKenzie, 109–45. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-20574-4_4.
• Seaver, Nick. 2017. “Algorithms as Culture: Some Tactics for the Ethnography of Algorithmic Systems.” Big Data & Society 4 (2): 205395171773810. https://doi.org/10.1177/2053951717738104.

*Image credits: Markus Spiske via Pixabay