“The SEMIA Revolution: Denaturalizing Our Conceptions of Knowledge and Discovery” – Laura Jimenez Rojas

The metaphors we use to describe our thoughts, practices and projects matter. In this TiM session, Eef Masson presented the Sensory Moving Image Archive -SEMIA-. This is an interdisciplinary project that challenges our traditional conceptions about what a digital archive is and what could it be. Their point of departure is the distinction between searching and browsing.

Searching refers to the semantic descriptors (search terms) that we use to navigate collections, it is a practice that comes from a paradigm in which it is assumed that answers are shown from specific questions. In contrast, browsing is a practice of exploration, it is the experience of experimentation, it is the “screwing around”[1].

SEMIA is in itself an exploration of other ways of knowing, the software that is being developed -through AI deep learning techniques- is based on taking as the reference, not the physical world (object recognition), as it is traditionally done from the AI practice, but rather from representational objects: moving images archives.

This focus may not seem relevant for the unsuspecting reader, but it is a great paradigmatic shift. The fact that these techniques find new fields of possible implementation allows, on one hand, technical developments towards new objects. On the other hand, and this is very important, it starts to change stabilized notions of knowledge and discovery. This last point makes me think of the SEMIA project as a revolutionary attempt to challenge both the field of AI on a technical level and said naturalized terms.

The introduction of a new paradigm through the practice of “screwing around” the moving image archive let to position SEMIA as a perfect illustration of an “interface” framed from an approach of the humanities. SEMIA is indeed a “space that supports interpretative events and acts of meaning production.”[2] In this case the interpretative events are mediated by descriptive sensory metadata, namely: movement, color, and shape. These categories are very different from the traditional metadata which focuses mainly on single word items.

The displacement from semantic descriptors to sensory metadata was conceptualized by Masson, following Julia Flanders, as the transition from the “network” model to the “patchwork” model. The former assumes that things have a prior connectability, while the last one does not take for granted a supposed connectability, but rather acknowledges the agency involved in the act of collection, it entails the idea of being a crafted assemblage[3].

In this sense, knowledge is not of a static nature which can be grasped, rather it emerges from the practice. The affordances of the digital platform allow the emergence of the constructivist subject[4] of the humanist interface theory, thus also changing the notion of discovery. Here, to discover is not positioned in an analytical framework, it is rather a performative thought, a craft practice. SEMIA is then an effect of the metaphors we think with and at the same time, it is an opening to new practical and theoretical paths.

Notes

[1] Concept from Ramsay, Stephen. (2010). ‘The Hermeneutics of Screwing Around; or What You Do with a Million Books’. Cited in Julia Flanders, ‘Rethinking Collections’, in Advancing Digital Humanities: Research, Methods, Theories, Paul Arthur Longley and Katherine Bode (Houndmills: Palgrave Macmillan, 2014), pp. 163–74.

[2] Johanna Drucker, ‘Humanities Approaches to Interface Theory’, Culture Machine, 12 (2011), p. 3.

[3]“A patchwork or mosaic (…) connects pre-existing informational nodes while acknowledging them to be incommensurable: taken from different semantic spaces, from different contexts of purposiveness and original utility, not automatically the right size and shape.”  Flanders, p. 172.

[4]“(…) the constructivist subject of the digital platform emerges in a codependent relation with its affordances. This is the ‘subject of interface’ when interface is conceived as a dynamic space of relations, rather than as a ‘thing’.”  Drucker, p. 3.

References

• Drucker, Johanna, ‘Humanities Approaches to Interface Theory’, Culture Machine, 12 (2011)
• Flanders, Julia, ‘Rethinking Collections’, in Advancing Digital Humanities: Research, Methods, Theories, Paul Arthur Longley and Katherine Bode (Houndmills: Palgrave Macmillan, 2014), pp. 163–74