“Robot Opera: Performance, Embodiment, Vocality” – Evelyn Ficarra

This is a report on the Robot Opera research initiative by the Centre for Research in Opera and Music Theatre (CROMT) at the University of Sussex. Using commercially available robots, we explore issues of performance, embodiment, and vocality through scholarly investigation and creative production. The report will show highlights from the documentation of our recent one-day symposium, which featured talks and performances of two five minute operas for two Nao robots and two human musicians. What does it mean for a robot to sing? How might generative music software be impacted by robotic physicality? Can a robot perform, or is it always performing? This project explores the meanings and challenges of involving robots in operatic performance, in particular, highlighting the potential for robotic musicality and vocal expression.

Evelyn Ficarra (Ph.D., UC Berkeley) is a composer and sound artist. She writes music for dance, music theatre, multimedia, experimental film, installation, and the concert hall. Evelyn is Senior Lecturer in Music at the University of Sussex where she helps to run the Centre for Research in Opera and Music Theatre. www.evelynficarra.net

“Cooperation of robots and humans in recent operas” – Sabina Macioszek

In Information Arts, published in 2001, Stephen Wilson asserts that robots are the quintessence of contemporary times. Wilson defines them as “intelligent machines”, which not only gain more and more importance in everyday life but also provoke questions about potential limits in constructing AI and robots’ embodiment. Moreover, he states that nowadays it is often not possible to establish an explicit border between science and different types of art. It appears that similar rules should be adopted for recent operas. Opera is treated by some scientist, research institutes, engineers and composers themselves as an excellent form for testing different ways of communication between robots and human performers and opera audience. Therefore, this article applies to performances in which robots are active and (semi-)autonomous opera performers. I will explain what constitutes the cooperation between nonhumans and humans and what characterizes individual robots. Moreover, selected opera works, created among others by Tod Machover, Gob Squad, Wade Marynowsky or Matthew Sleeth, pretend to be performances that indicate many elements of cooperation between people and robots which may also exist outside “the operatic stage”. They suggest how to improve the cooperation between people and various technologies in so-called “everyday life”. According to researchers – among other from the Massachusetts Institute of Technology in Boston or the Neurorobotics Research Laboratory in Berlin – actually, such operas can be defined as “operas of the future”

Sabina Macioszek is working on a dissertation about strategies of the cooperation between bodies and digital technologies in recent operas, created in the 21st century. She is interested in new technologies which affect the bodily perception of performances, and in collaboration between people and technologies which can be treated as autonomous actants.

“Creative AI and Human-Centred Robotics in Music Performance” – Craig Vear

In this Performance/ Paper, I will outline and demonstrate an approach to performative robotics that is cooperative and creative. Augmenting theories from Nouvelle and Human-Centred Ai I will present Dexter 2: a robotic musician who co-creates inside the flow of live music. Drawing on nearly 15 years’ worth of research with robotics, autonomous performers and Creative AI I will argue for considering cooperative robotic musicians/ performers to be a different thing from an autonomous performing machine. I will expand on this through a framework of Creative AI based on principles of Coping, Belief and Flow in order to highlight that which is important to the robot, and to the humans who a) construct performance with robotics and AI and b) co-create inside the flow. At the center of this paper is a desire to explain how robotic musicians can understand humans, and how the humans can understand the robots inside the domain of performance, and for creativity to be shared across these relationships. If we want robots to join us – I mean truly join in with us – inside the creative acts of music and performance then it is the relationships that bind us together inside the flow that should be our priority.

Craig Vear’s research is naturally hybrid as he draws together the fields of music, digital performance, creative technologies, AI, gaming, mixed reality and recently robotics. He has been engaged in practice-based research with emerging technologies for over two decades. His software score is published internationally by Composers Edition, and he is deputy editor-in-chief of Springer’s Cultural Computing Series. His recent monograph The Digital Score: creativity, musicianship, and innovation, will be published by Routledge NY in spring 2019.

“Can theatre be a site to practice in the future?” – Marco Donnarumma and Kristina Andersen

We would like to propose a conversation between design (Andersen) and theater making (Donnarumma), as it exists in opposition and convergence between our own practices. Our combined bodies of work lie between integrating machines into what we consider human, and understanding them as autonomous others, which can be appended, engaged and collaborated with. The format could be a turn taking between the imagery of Donnarumma’s dance theater performances with AI prostheses and Andersen’s experiments of designing alongside robotic systems. Through this lens, terms such as ‘tool’ and ‘prostheses’ take on new meanings. In performance, a robot can be an inherent and agential part of the human body and therefore affect the way we think, move and feel; as incorporation, rather than an extension. In a similar manner, things are proposed and designed, but through their presence design us in turn. This continuous becoming is a process of emergence, where the notion of “human” is neither defined nor static, but changes by incorporating affordances and agencies of the things we make. At the same time, technological progress – and with it, the propaganda machine of techno-capitalism, solutionism and the desire for the “new”, drives our understanding of technology towards the “normalized”, conformed, and shallow; while stranger, unfamiliar configurations of human and machines are relegated to the margin of the discourse. Our own practices converge in a shared insistence on performing that marginal area. How can we create a theater that pays and allows attention to machines on stage and in the workshop as neither aid nor replacement, but things/bodies that are sometimes us, and sometimes autonomous others? And what are the implications of such a performative practice for culture at large?

Marco Donnarumma is a performance artist, director, composer and scholar combining contemporary performance, media art and computer music since the early 2000s. He creates performances probing the body through sound physicality, technological engineering, and movement research. His latest cycle of works, “7 Configurations” (2014-19), focuses on the conflictual relations between AI robotics and body politics.

Kristina Andersen is a designer and researcher based at TU/e. Her work is concerned with how we can allow each other to imagine our possible technological futures through digital craftsmanship and collaborations with semi-intelligent machines in the context of a material practice of soft fiber-based things.

“The posthuman stage: Thinking humanness through robotics” – Julie-Michèle Morin

The robotic presence in the theatre field informs us about many aspects of the human-machine interactions but also on our sole conception of the living. The way we design and think robotics is shaped by the notion of human familiarity. Thus, these theatrical relations help us reflect on the way we export our view of humanness and what we consider to be familiar, or not, on figures that are non-human. Therefore, the robotic imaginary is based on the way we project human normativity on the non-human. Jennifer Rhee underlines these power dynamics in the form of a question: “Who gets humanized and how? Who gets dehumanized and why?” (2018, p. 11). This approach allows us to reflect on the robotic imaginaries that has consolidated around robots but also to reject the self-evident characteristic of our conception of the human being in the theatrical space.

During this conference, I will analyze how the use of robotic in theatre mobilize and reshape the sole concept of the human being in a post-anthropocene era. I will investigate how the robotic imaginary materialize itself on stage and how theatre-makers can unfold certain presumptions we have on robots in order to propose new non-normative interactions between humans and robotic figures. The goal of this presentation is to share ideas regarding the way we can observe power dynamics operating between human and robots in the theatrical contexts and unfold the way we think of humans. Observing the human through the robots could enlight us on the way we perceive the living forms on stage. I will refer to Jennifer Rhee’s work on the robotic imaginary but also to Karen Barad’s theories regarding the interactions and intra-actions between human and non-human in order present broadly the early stage of my doctoral project.

Julie-Michèle Morin is a second-year Ph.D. student in Litterature. She is pursuing her doctoral research at the University of Montreal and the Antwerpen University. Her main research interest concerns robots and machines in the theatre field and the way they reshape our conceptions on human and non-human interactions. She takes a multidisciplinary approach that encompasses neo-materialist theories and techno-ethics matters in order to analyze the work of many artists such as Kris Verdonck, Oriza Hirata, Bill Vorn, Louis-Philippe Demers and Martin Messier.

“Robot Performance and the Staging of the ‘Techno-vulnerability’” – Ruowen Xu

This paper will discuss the staging of the collapsed, interrupted, failed, and malfunctioning episodes in many contemporary theatrical robot performances, and look specifically into the case of the Japanese Android Theatre Sayonara (2011-2012,) as an example to examine the presentation of the “techno-vulnerability” and how this notion becomes adopted from the theatrical practice to the field of social robotics to enhance human-robot interaction. Through the concept of “techno-vulnerability,” this paper not only unveils the transformation of the human physical and affective vulnerabilities with the interference of technological interfaces, but it also accounts for the alternative subjects and technological bodies as being involved in the re-framing of the “shared vulnerability” in the post-anthropocentric biopolitics. As performance scholar Jon McKenzie denotes, along with the body-culture performance that shapes our cultural identity, “techno-performance” conceptualizes another stratum of the socio-technological imperative to proceed with effectiveness over technical challenges. In this light, however, the paper elucidates the robot’s theatrical performance of the counter-effective “techno-vulnerability,” as first demonstrated through the robot’s sensing of its loss in the cultural and normative dimension, and then in the technological sense, accounting for the robot’s status as being impaired to a malfunctioning device; meanwhile, it also manifests the human vulnerabilities enacted by and intermingled with technology (in this case, the hospice care and the nuclear disaster.) Highlighting the cross-materiality of the bodies experiencing loss, as well as the interwoven of technological performance in the social construction of vulnerability, “techno-vulnerability” theorizes the posthuman subjectivity as always imbricated with heterogeneous bodies, technology and the discursive nexus in practice; meanwhile, it strives to render the social critique of the non-discriminative object-targeting of the human and robot’s “shared vulnerabilities,” which accelerate the assimilation of robots in the post-anthropocentric politics and global economy, displaying the ethical dynamics of the human-technological assemblages which are exposed to existing and emerging paradigms of power relation formation.

Ruowen Xu is a first-year PhD candidate in Media and Performance Studies at the Institute of Cultural Inquiries, Utrecht University. Focusing on robotic art and robot performances from the fields of artistic experimentation, theatrical staging to the social implementation, her research“Techno-vulnerability: Affective Interactions in Robot Performances”investigates robot technology’s mediation, transformation, disposal and production of a range of vulnerabilities throughout the process of human-robot interaction.

“Life Illusion: Animating the Machine”-Thomas Riccio

An overview examination of social robot creation and animation applied by Hanson Robotics. Referencing the presenter’s fourteen years of working recurrently with Hanson Robotics as Character Writer, Narrative Engineer and most recently, Creative Director, the presentation will outline three distinct character creation approaches. The first character building method is based on historical figures and draws on the writings, and the biographical and historical record of Einstein and P.K. Dick to shape the robot personality. This method of rendering is at times more speculative in regards to personality and animations, and lives more symbolically, sharing a social/cultural context of recognition and expectation. The second method based on a living person, whereby interview transcriptions, video reference, and personal interaction provide character input data. With this method, the robot personality has a direct reference to the human personality, providing nuanced facial viseme animations, specific and idiosyncratic language and syntax, and content specificity. The presentation will outline the process and development of Bina 48, a social robot rendering of Bina Rothblatt, commissioned by the Terasem Movement Foundation. The third method is the creation of a unique and individual robot character. This method draws on data and content developed for other robot characters which are then re-shaped and augmented to serve commercial needs. This method will examine the robot Sophia and how she has and continues to, evolve as both a research platform and robot personality. Created by a committee and shaped and reshaped in reaction to need and expectation, Sophia will be the paradigm for social robot creation in the future.

Thomas Riccio, Professor of Performance and Aesthetic Studies, the University of Texas at Dallas, concurrently Visiting Professor, Jishou University, China. Artistic Director, Dead White Zombies, Dallas. Previous: Professor, University of Alaska Fairbanks. Research areas: ritual, shamanism, indigenous performance, and robot characterization and theatricalization. Recent: Creative Director, Hanson Robotics, thomasriccio.com, deadwhitezombies.com

“Characterising Robots: Some Ways in which Theatre might Support the Creation of Sociable Robots” – Louise LePage

I start from the premise that the robot is inherently performative. The robot has no essential character to express, enact, or originate speech or action. Drawing on the cultural theorist, Judith Butler’s, work on gender performativity, but substituting humanness for gender, I propose that performativity of a human-like character is a stylized repetition of acts, an imitation of the dominant conventions of humanlike characters. I propose that if a sociable robot is to be compelling in social interactions, it needs to appear to have a character. Setting out a number of theatrical elements that might be particularly significant in promoting the imaginative transformation of a robot into a character, I will argue that its characterization must be aesthetically coherent, taking account of identity, genre, and style. I will draw upon Masahiro Mori’s theory of the uncanny valley for its insights into how affinity is important to and may be engendered in humanlike robots. Building from David V. Lu and William D. Smart (‘Human-Robot Interactions as Theatre’, 2011), I will also consider ways in which a Stanislavskian-type approach to acting might provide insights into ways in which robots might be given, and develop characters. I will also draw upon the performances of robots in plays and performances, for example, Pipeline Theatre’s Spillikin: A Love Story (2015), Seinendan Theatre Company’s Three Sisters: Android Version (2012), as well as my own playful performance projects with Rethink Robotics’ Baxter robot and, more recently, a Nao robot.

Louise LePage is a Theatre lecturer at the University of York. She teaches acting and technology in theatre. Her research is focused on robots in theatre, robots as performative entities, and the creation of robots we can ‘believe’ in. (See www.robottheatre.com.) Louise has a monograph forthcoming: Theatre and the Posthuman: A Subject of Character (Palgrave) and co-edited Twenty-First Century Drama: What Happens Now (Palgrave, 2016).

“How modeling various levels of a human interaction on stage can inform the development of social robots for theatre” – Helen Root & Irene Echeverria-Altuna

Theatre exposes human emotion and social interaction and, as such, provides a natural setting for experimenting with the development of human-computer interactions and showcasing the results. We will argue that the psychological and neuroscientific attempts to quantify emotional states in humans could be combined with the methods that human performers use to develop relationships between characters to inform the development of algorithms for social robots. Research into human-VP interactions has demonstrated that simple behaviors of the VPs, such as imitation, can lead to participants assigning them agency, intentionality and even emotional states. This is a potentially fruitful line of investigation into a set of fundamental reactionary behaviors, discovered through various measurements of the interaction between two human performers, that could be embedded into a robotic actor and will, in turn, be interpreted as personality by human performers. By using scripts written for theatre, we can be freed from the notoriously complex tasks of natural language processing and generation in order to focus on examining how the dynamics between a human and a robotic actor shift as the two are given character notes from their director.

Helen Root is a software engineer who graduated in 2017 from Imperial College London, where she discovered a love for directing plays.

Irene Echeverria-Altuna is a neuroscientist who graduated from University College London in 2017 and who is currently studying her Master in Brain and Mind Sciences in Paris.

In recent years robots have started to learn new tricks. They have to because their task description no longer contains things that are Drugerous, Dangerous or plain Dull (for humans), but now includes tasks for Care, Cure, and even Companionship. This might make them even more interesting for the stage – or vise-versa: Theatre forms an excellent playground to test robot capabilities and their interactive performance. Usually, the robots deployed on stage are ‘puppeteered’. The goal of this presentation is to sketch an outline of what is necessary to turn them from puppets into actors, making robots even better performers, both on stage as well as in real life. The outline will be illustrated with a large number of own research and art projects including a robot Facebook, robotic desk-lights, emotional robot cleaners, a furby retirement home, puppeteering interfaces, animatronic toolkits, interactive robot heads, a ballet for robot vacuum cleaners, the Dancing White Man robot, the crying robot ‘Annelies’ and Astrobot (both featuring at the recent RobotLove exhibition in Eindhoven). Also, the results of a recent summer school workshop on improv theatre as an evaluation tool of social robot application scenarios will be discussed. The paper and presentation will dive into the platforms and technologies which power these projects and thoughts on the performative aspects. (and will also shed some light on why the Robotics and Mechatronics group’s famous gas-pipe inspection robot responds to MIDI commands.)
short bio:

Edwin Dertien holds an MSc degree in Electrical Engineering and a Ph.D. in robotics. He also is a lifelong maker, designer, and tinkerer and works as an artist / art engineer. Accidentally he became director of a care institute for people with autism, as well as co-host of an art gallery. Besides his work with the technology he works as a musician and has over 20 years of experience in making music for improv theatre. Currently, he tries to teach these skills to (and with) robots as well.

“When Robots Breathe: Dislocation and Nonhuman voices” – Yaron Shyldkrot

This performative presentation seeks to examine how robotic voices, as nonhuman performers, trouble the human/nonhuman binary and unsettle the boundaries between the natural and the artificial, the organic and the technological. Robotic voices are all around us. From Siri to our satnav, voices on phone operations, in supermarkets, as well as in popular music or as actors in performances. While acknowledging the current discourses around posthumanism, cyborgs, and ecology – and putting aside technical pragmatism – this presentation will study the performance of robots through sound. Specifically, the dislocation of the human voice. Informed by my practice research, I will demonstrate (in practice) how robotic voices can sing, breathe, shout or stutter in order to consider the voice nonhumans might have both physically and politically. Conceptualizing sound as a performative tool which might alter what is perceived, this presentation will highlight how robotic voices articulate (metaphorically and literally) the blurriness between human/nonhuman and destabilize fixed ontologies about humans and machines. At a time when robots tell jokes, sweat or even gain citizenship, I will propose that Robotic voices can reveal our coexistence with the other-than-human and challenge, to paraphrase Dixon (2009), both what counts as bodily as well as which bodies count.

Yaron Shyldkrot is a practitioner-researcher in the final stages of a Practice Research Ph.D. at the University of Surrey (UK), exploring dramaturgies of uncertainty and performance in the dark. Yaron served on the Executive Committee of the Theatre and Performance Research Association (TaPRA) and the editorial board for The Journal of Arts Writing by Students (JAWS). http://www.yaronshy.com

“APPENDIX MACHINA” – Dani Ploeger
APPENDIX MACHINA is an extension package for the domestic robot Keecker, consisting of a modified medical anal electrode and custom software, developed in collaboration with V2_Lab for the Unstable Media in Rotterdam. The package will be used for biosignal performances that explore intimate relationships between human and robot bodies, in the context of gadget commodity culture. The robot will project medical video recordings from the inside of a human rectum on the outside of my body. The video projections, based on found medical footage, will be modified and controlled in real-time by the sensor data from the electrode in my anus. The human rectum is an ambiguous place: often tabooed, it can both be a site of pleasure (sex) or disgust (shit; in very rare cases the two coincide). By using a biosensor that registers contractions of the human anal sphincter to establish a connection with a domestic robot – a stereotypical product of start-up culture – APPENDIX MACHINA transgresses the smooth and polished ways in which IoT commodities are usually marketed. On one hand, the connection of the robot with the pleasures of anal penetration heightens a supposedly harmonious human-machine interaction into the realm of explicit sexualization. On the other hand, the use of a medical sensor for the treatment of faecal incontinence places the robot in the context of inevitable decay, both of the fleshy human body and the plastic and metal components of the robotic body. APPENDIX MACHINA cherishes the imperfect, fragile and dirty sides of intimate relationships with technological devices.

Dani Ploeger is an artist and researcher who combines performance, video, computer programming and electronics hacking to investigate and subvert the spectacles of techno-consumer culture. He holds a Ph.D. from the University of Sussex and is currently a Research Fellow at The Royal Central School of Speech and Drama (London) and Leiden University.

Forced Labor: A choreography for robots

During this session, you’ll get a sneak peek of the robots choreographer Ugo Dehaes is developing for his upcoming dance performance Forced Labor (2020).
Ugo has been working with dancers for over 15 years. For each show, he looked for the specific dancers that could help him create the dance he imagined. Most of the time he worked with professional dancers, but for some performances, he brought together groups of older dancers, children, urban dancers, real couples, or amateurs.

Forced Labor is about our society, and how we developed robots and artificial intelligence to replace us at our jobs. Originally the robots got the heavy, repetitive and dangerous jobs. The artificial intelligence helped to recognize patterns or learned how to drive a car, but it felt safe to say that all creative jobs would still require a real human being.
But now we passed the point where computers are just executing what they are programmed for, as they start to think for themselves, and become smart and creative to do their job.

As a choreographer, Ugo has always been interested in movement, no matter how or by whom it was created. For Forced Labor, he set himself the challenge to create a dance performance where as many artistic jobs as possible would be executed by robots.

At this point, he replaced all dancers by moving objects, but hopefully, by 2020 he will be able to replace himself, the choreographer, by artificial intelligence that will create the final choreography.

“More than an Object”

This demonstration includes the presentation of the robotic object discussed during the keynote lecture on Saturday. For more information about the keynote lecture click here. 

Petra Gemeinboeck is an artist and researcher exploring our entanglements with machines. She is a Senior Researcher at the Creative Robotics Lab, National Institute for Experimental Arts, UNSW | Art and Design, AU; and holds a Senior Research Fellow position at the MetaMakers Institute, Falmouth University, UK.

Rob Saunders is a researcher working across creative AI and creative robotics. He is Associate Professor of Computational Creativity at the Games Academy, Falmouth University, UK; and a Senior Research Fellow at the Design Lab, School of Architecture, Design, and Planning, University of Sydney, AU.

Pottery Bot

Do you remember the famous pottery scene from the 90s movie ‘Ghost’, in which Demi Moore and Patrick Swayze are molding their love together into a slowly coalescing pot? Whether you think this scene is the climax of nineties romance, or find it to be shallow drivel, the scene is easily recognizable by an entire generation. SETUP is going to put this famous scene into action in our next installation.

How will we do this? Pottery 2.0 – traditional pottery, together with a robot. The user either takes the role of Demi or Patrick, and will learn from the robot how to mold a vase, or will teach the robot to do this. Not only will they make something together, but the process will also redefine the relationship between human and machine.

The fusion between human and machine has become more fact than fiction, for example through companies such as Elon Musk’s Neuralink that are building a future as wild as famous sci-fi stories from the past. Most of the media attention seems to be about the assumption that machines are becoming more like us. The often ignored antithesis to this is that we slowly become more and more like machines.

This installation will be created in conjunction with Casper de Jong, an artist who specializes in installations and robotics, and who tries to discuss heavy subjects through theatrical narratives that raise a smile.

SETUP is an Utrecht-based media lab, established in 2010. Combining academic background with artistic practice, we strive to strengthen a more enriched, critical stance towards emerging technologies. Our main goal is to educate a wide audience, providing them with the tools necessary to design this brave new world and infuse it with human values and new-found creativity.