yuxi liu

Designer and artist based in Munich. Recent work explores designing with machine intelligence through critical, philosophical, and poetic lenses.

01 Serendipity Searcher
02 Five Machines
03 Cooking with Miso
04 no machine is an island
05 Poet on the Shore
06 The Other Actor


The Other Actor (2018)

The Other Actor is a concept created together with Jing Yu in response to FRAME magazine’s invitation for its design challenge – The Future of Shows.

The Concept

We hypothesize that in response to the advancement of tecnical sophistication in digital entertainment, physical shows will be driven to commit and further invest in the properties unique to the physical world. Multisensory, embodied, irreproducible and contextual experiences are already surfacing as counter–points to the always- and anywhere- accessibility of the digital show. Increasingly, coveted experiences demonstrating the desirability of encounters unobtainable online are available to the public – Epicurean Nights, Sleep No More and Pop-Up Magazine are a few examples of such shows that dig deeper into wealth of the physical, analog world has to offer. 

Even as phyiscal shows redouble in their physicality, the frontier of digital shows grows ever further – pushing the boundaries and imagination of what is possible to experience in art, music and entertainment. As the two mediums develop, we may see the spectrum of shows, from the most traditional physical to the newest fashionable digital experience begin to widen and fractalize. What is of particular interest to us is where the two domains meet and begin to blend. In that liminal space, the strengths particular to digital or physical combine for a show that is generative, reactive, and wholly exclusive. 

Within this space we focus on how the collaboration with artificial intelligence (AI) could bring an immersive, hybrid experience. In a world of human-machine cohabitation, we envision a future show in which the AI senses, learns, generates, and together with its human collaborators, it expresses its latent dreams, with words, with matters, with movements; to perceive, to converse, to surprise. The AI + Human collaborative show is one whose content transforms not only in response to the human actors but the human audience – a show never finished, constrantly evolving from day to day, city to city, culture to culture. Within the boundaries of the stage, however it may be defined, the AI actor-container is liberated from the traditional organic limitations of its human counterparts. What could a comedy show, an intelligent improvisation, an emotive dance – feel like from an actor who embodies an ungendered, amorphous shape, an industrial robotic arm, a 50 meter screen?

In this human-machine circus, the boundary between performer and audience, subject and object, seeing and being seen, dissolves, and we become one, dreaming the same dream. Perhaps, on this stage, the collaborative show will hold a mirror up to the question of what it is to be, as performances of the past have done for centuries. 

The Other Actor is a concept for future shows in which humans and AI create and perform together, in a never-ending cybernetic loop – the performer and audience (the actor) interact with the sophisticted AI system (the other actor), influencing how the AI senses and generates, and therefore what they see in the next moment; these changes in turn will influence how the actor interacts with the other actor. The intervention of and collaboration with the AI actor will facilitate a paradigm shift whereby shows will be increasingly dynamic, evolving with time, space, and context. 

The visualization will examine the interplay between the performer, the audience, and the AI system within a defined space, represented by a cube – how the AI reacts to the perfomer and audience, how the performer acts upon the mediated space and affections, and how the audience, being seen, sensed, and analyzed, perceives the evolving show of which they become an indispensable part. To illustrate this ongoing construction, we will embark upon defining different levels of granularity. Through the change of perspective, we will reveal different facets of the system, from different angles. 

The Visualization

To visualize the concept, we present three illustrations: the overview of the system, the zoomed-in views of the audience and the performer, from the other actor’s perspective. 

The amorphous AI actor is embedded in the space itself, with cameras capturing affective reactions, sensors sensing realtime locations. Therefore, on the system level, the space itself can be viewed as a data container (fig. 1), in which movements (fig. 2 and fig. 3) and emotions (fig. 2) are collected and converted into data, analyzed and translated by the AI, and eventually manifested through visual and auditory medium. In this immersive space, the aesthetic experience of the audience – the output of the AI-mediated performance, becomes part of the input of the experience itself. 

︎ Fig. 1. The overview

︎ Fig. 2. The audience – the other actor’s point of view

︎ Fig. 3. The performer – the other actor’s point of view

The Conversation

Say someone has just been to see The Other Actor show. How would they describe what they actually saw to a friend who wasn’t there?

We enbision The Other Actor as a new paragigm rather than a specific show. We believe with AI being the other actor, shows in the future will be generative, more dynamic and immersive.  For the visualization, we imagine a show in which a panoramic screen displays genetive visuals along with sounds based on the performer’s improvised movements and the audience’s reaction, which are captured by embedded cameras and sensors. Thus, what the viewer experiences is a conversation between the performer, the audience, and the AI, in s multisensory way.

Can you describe the ‘neverending’ aspect of the performance a bit more clearly?

What we mean in regards to ‘neverending’ is that the show of the future may not be written before the show is produced – the result is typically a production with a defined story/aesthetic/prognosis. An Other Actor show is a show undone, the story is written with the understanding that the show will continue to evolve once it is put into the world and comes in contact with its context. 

What kind of spaces do you envisage your project being used?

Theoretically any kind of space that can be equipped with data capturers could be used for a The Other Actor show. Here space itself becomes an entity, its characteristics can provide different possibilities of input that can affect what is perceived, by the performer, the audience, and more importantly the AI.

How does the environment impact the experience?

As the show is highly improvised, the environment impacts directly how the performer interacts with it and how the audience perceives it. The AI system would then capture this perception as well as environmental data, generating content accordingly. The space also defines the skin that the AI could inhabit, one can imagine a large theather that enables the Other Actor to embody the entire screen or in an outdoor space the AI as a fleet of small drones.