In order to get to know our commissioned artists for Experimenta Life Forms a little better we are publishing a series of interviews about their works. First up are Uyen Nguyen, Max Piantoni and Matthew Riley who’s work ‘You, Me, Things’ will push their ‘Yomeci’ series into new territory.
Experimenta: What inspired you to create the concept behind You, Me, Things?
Matthew, Max and Uyen: The intention of You, Me, Things is to create an ever-evolving world that is assembled by sounds made by audience members, who as participants, become part of the existence of the world itself. Inviting both listening and performing, the work will be a playful experience made up of a complex set of relations between the works digital entities, the participants and its virtual world.
You, Me, Things is the latest in the Yomeci series of works first instigated by Uyen in the Master of Animation, Games and Interactivity program at RMIT. These works explored listening and making sounds as a form of play – investigating how our sonic world can be harnessed creatively to construct new perceptions and interpretations. The first project of this series involved Uyen walking the streets, listening to the city and recording its sounds and voices. These collected everyday sound moments were then reimagined as animations in different contexts to their original meanings.
This interest in what a city sounds like then led to a prototype for an experimental location-based mobile game called YomeciLand. Uyen and Matt shared this at the DIGRA 2019 conference in Kyoto, Japan and the Playable City Melbourne symposium and this prototype is still a project we all want to develop. Players walked the city locales and public spaces of Melbourne using their mobile phones to listen and record found sounds that ‘fed’ a virtual world. These urban sounds (vehicles, traffic lights clicks, construction noise etc) were used as input and abstracted into the game as various animated ‘creatures’ in its world. Given louder urban sounds tend to dominate more subtle and quieter ones, these were translated as dark ‘predator’ creatures while gentle sounds were represented as ‘prey’. However You, Me, Things moves beyond this hierarchical approach and brings together sounds and visuals in a more ‘flatter’ and relational way.
While the early work in the Yomeci series focused on listening, for You, Me, Things we are interested in the ‘sounding body’ at play. We explored this notion in an installation version of YomeciLand at Bunjil Place in 2019 where participants interacted with the work by making sounds into a mic. These were largely non-verbal sounds where participants used their bodies, actions and gestures and so on to make sounds which would ‘call’ digital entities into a world or artificial ‘ecology’.
In You, Me, Things these audio inputs will form the ‘building blocks’ of a complex virtual world by inviting digital entities to emerge and form it. These will reflect the tone of the sound uttered or made by the participant – a whisper will make a light and floating entity while a harder sound might make a solid clump of something. These ‘things’ will form the ‘world’ – they are part environment and part inhabitant. These lumps, forms and pieces that make the digital entities can change and assemble and this will create a lot of diversity and variation. Ideas about coexistence began as Uyen thought about and drew symbiotic pairs of animals, plants and trees from different species’ relating or having a rapport with one another.
You, Me, Things will try to establish some playful and emergent relationships, both within it’s virtual world and between audience members interacting with the work. As its own unique lifeform, it’s ‘world’ is an ongoing ‘dance’ between the digital entities and participants. The gallery space where the work will be situated will oscillate, move and sway with all of these different beings performing and playing.
Experimenta: Tell us more about working with animation, sound & A.I. technology within this artwork.
Matthew, Max and Uyen: You, Me, Things uses a sound recognition system powered by artificial intelligence in order to recognise audio input as sounds. It encourages participants to perceive that they are being ‘listened’ to by the ‘world’ and it understands the sounds they make.
The A.I. technology allows us a much more complex system for recognising and classifying the sounds participants make. Previous iterations of this work detected pitch and volume as input which were less accurate. Using A.I. gives us more control in how we will design the interaction of You, Me and Things. Uyen’s brother Duy Phuong Nguyen is an engineer and has developed the audio recognition system. Responding to this model, Max’s systems are able to propagate the world with ‘living’ digital entities.
The digital entities of You, Me, Things have different identities expressed through the various sounds, forms and movements they have. Rod Price, the sound designer on YomeciLand at Bunjil Place, carefully considered how the sonic identity of each entity would express itself individually but also when they came together as a group. This created a cohesive composition, no matter what entities were making sounds in the world.
As we mentioned, with the entities made up of different ‘building blocks’ they will be constructed according to relationships to other things, such as each other and the participants’ actions. That is, the digital entities will be rarely static or fully formed. Although there are parameters and ‘rules’, the work’s artificial ‘life’ will emerge from forces and relationships rather than being predetermined.
Experimenta: How have you developed the interactivity of the artwork and how have audiences responded to this?
Matthew, Max and Uyen: While the prototype for location-based mobile game of YomeciLand focused on listening and ‘collecting’ urban sounds, the installation adaptation invites people to make sounds and is a different experience. There is a shift from listening to performing. The installation work responds to non-verbal sounds, e.g. clapping, humming, stomping, whistling, chuckling etc., and encourages everyone of any age, gender and language to play. Participants of the last installation version of YomeciLand at Bunjil Place in 2019 often interacted with the work collaboratively, harmonising in the gallery space as they made the same sound together. Adults and children played it together or enjoyed it as an individual experience.
Some participants were very attached to specific entities and purposefully ‘called’ them into existence to influence the YomeciLand world’ towards their sensibilities. For example, we saw parents encouraging their children to laugh to help them call out an entity that responded to laughter. People enjoyed composing the experience through sound and we could see this encouraged a kind of connection and joy with the world and it’s entities.
Experimenta: What are you looking forward to about presenting and touring the work as part of Experimenta Life Forms exhibition?
Matthew, Max and Uyen: We have always followed and supported Experimenta and the themes of this Experimenta Life Forms show are particularly compelling and pertinent given the impact of non-human activity recently. We are very interested to see how the other commissioned artists are exploring the questions and ideas of this show.
Given it’s a touring national exhibition, it’s great that audiences in regional communities and other locations can experience this playable installation. It’s why we are so enthused to be commissioned, it’s important people of different communities and ages are presented with opportunities to experience work of this kind.
Experimenta Life Forms International Triennial of Media Art
Exploring how biological and artificial life are challenging human-centric thinking, Experimenta Life Forms will be Experimenta’s 8th national touring exhibition. It premieres in 2021 and will tour nationally until 2023.
‘You, Me, Things’ by Uyen Nguyen, Max Piantoni and Matthew Riley is an Experimenta Commission.