Cooperation

QUEUE

XR (Extended Reality)

Interactive

AV (Audiovisual Design)

2025

I FEEL YOU

VR (Virtual Reality)

Hybrid Spaces

Liis Vares (choreographer) and Taavet Jansen (interdisciplinary artist and researcher) integrate movement, digital technologies, and interactive performance, inviting the audience into a space where instinctive physicality meets linguistic thinking, using mixed reality.

, part of the duo's wider Held in Human series, explores the meaning of presence and "being in a body" in a contemporary world where technology significantly extends and mediates human experience. It invites performer-audience members to enter a dynamic "thought-space" and lend their bodies to its dramaturgy. Equipped with a see-through VR headset that allows for both physical and digital navigation, each participant brings their own unique physicality, imagination, and interpretation, contributing to an evolving choreographic language and inclusive dramaturgy.

Held in Human is an ongoing research project that we began in 2023, exploring how physical and virtual spaces can be combined to create hybrid artistic environments. Central to the project are interaction, improvisation, embodied participation, and the evolutionary nature of the work itself.

The first iteration, Held in Human (2023), took place simultaneously in a gallery space in Tallinn and a surrounding digital layer. Participants could influence a hyper-performative installation through text contributions, manipulate projections and screens, and contribute to sound recordings. Over the course of three weeks, thoughts, emotions, and desires accumulated and interacted with one another, allowing people to encounter each others presence without ever meeting face to face.

This was followed by Held in Human II: Rose in Your Brain, a single-user mixed reality performance built for Meta Quest 3. The work focused on reading as a performative act, inviting participants into an intimate MR experience where physical and virtual dramaturgies intertwined.

The third iteration, Still Moving, was developed within the MODINA project and shifted the focus toward multiplayer experiences in a staged theatre environment. Participants shared a mixed reality space supported by a composed soundscape, lighting design, and video elements. They could create textual environments, exchange headsets, observe each others actions, and collectively shape the unfolding experience. Structured as a 45-minute performance, Still Moving became an important step toward understanding shared presence and collective dramaturgy in mixed reality.

Building on these iterations, we became increasingly interested in developing a dramaturgy of shared mixed reality experiences. The residency in Dortmund provided an opportunity to continue this research and investigation into the deeper relationship between physical and virtual performance spaces.

As we both have a background in theatre practice – Taavet experimenting with technological aspects, Liis having collaborating with differently abled bodies – it was quite strange to witness how, after the pandemic, people wanted to get back to normality. This meant cutting out the technological connections and online-platforms that helped us to survive, stay sane and connected, and to feel the presence of the others, during the times when physical presence was constrained. In response to this, we started to treat the web space not merely as a tool but as a stage in its own right, using it to extend our practice and to inve stigate how our work could expand through digital presence rather than retreat from it.

There are these famous Shakespeare words:

And all the men and women merely players;

In Held in Human, we ask: What is a stage in todays world? And who are the players… or actors?

What is the 'Held in Human' project?

In Dortmund, our main focus was on the relationship between physical and virtual space. We explored how stimuli in the physical environment shape the audiences experience of a virtual world. This time, we deliberately focused on artistic and dramaturgical questions. Since our developer, Norbert Pape, was not with us, we shifted our attention from building new technical features to investigating staging strategies for mixed reality performance.

At the moment, we are particularly interested in developing multiplayer MR experiences that allow different bodies to meet, create images and spaces together, and share realities that are not limited by physical proximity. Participants may be located in entirely different physical environments, yet encounter one another within a shared virtual space. The coexistence of multiple spaces, the challenge of synchronising them, and the artistic possibilities that emerge from such arrangements feel almost endless to us.

One of the most exciting discoveries in Dortmund was finding ways to synchronise events across both layers of the experience. For example, a text disappearing in the virtual environment could simultaneously trigger a moving-head light and a sound cue in the physical space. Moments like these revealed new dramaturgical possibilities for mixed reality performance and gave us a glimpse of what a truly integrated MR language might look like.

It was also invaluable to test the work with a relatively large number of participants. This allowed us to identify both the strengths and the weaknesses of our system. In the end, the performance structure concentrated a great deal of control in the hands of one facilitator, while another person participated as a performer. This exposed a challenge we had not fully anticipated: when something goes wrong in a mixed reality performance, such as headset desynchronisation, the event often has to pause while devices are restarted, recalibrated, and brought back into alignment. Beyond the technical disruption itself, the facilitator must communicate this process clearly, playfully, and without breaking the experience. We realised that this creates a unique kind of performative pressure that is as much a dramaturgical challenge as it is a technical one.

How has 'History. Her story. Hi, story?' been developed at the Akademie?

For us, mixed reality begins with a very simple yet profound shift in perception. We use VR headsets in passthrough mode, which creates the impression that you are looking directly at the physical world and the people around you. We call it an impression because, technically speaking, you are actually looking at camera feeds displayed on screens just centimetres from your eyes, with latency so low that the brain accepts it as reality. What fascinates us is how quickly this technological mediation disappears from awareness. Within moments, people begin to trust the space, move naturally, and behave as though the virtual and physical layers belong to the same world. It is important to us that participants feel free in their bodies and able to move safely through their surroundings, guided by curiosity rather than instruction. Mixed reality becomes interesting when attention can shift fluidly between what is physically present and what exists only in the virtual layer.

In our work so far, the virtual layer has been primarily text-based. Rather than filling the space with realistic 3D environments, we have been interested in what happens when words themselves become spatial material. Text can appear, disappear, accumulate, transform, and be shared between participants. Meaning begins to occupy space, and language becomes a performative medium through which worlds can be built.

What excites us most about mixed reality is that it allows multiple realities to coexist without forcing one to replace the other. The physical environment continues to carry its own textures, sounds, and social dynamics, while the virtual layer introduces additional meanings and possibilities for interaction. Our use of text is one way of exploring this relationship: language becomes in a way a bridge between realities, connecting bodies, spaces, and imaginations. Performance emerges in the negotiation between what is physically present and what is virtually constructed, creating experiences that neither layer could produce on its own.

It is like stepping physically inside a thought-space that is usually invisible. We say: you cannot see what is going on inside someone's head -- but in our practice you kind of can. We call it a choreography of thoughts.We dont know any better technology to be able to create that kind of environment and sensation than a VR headset which fully immerses you and your attention but keeps it real, meaning: where you physically are.

Every body is a separate reality. This means we are already living in a mixed-reality, but we all have something in common. Liis has a background in mixed-abilities dance and for her it is mixed-realities dance because different body-realities meet. Mixed-reality technology enables us to merge together the inner and outer layers of a human. It helps to bring invisible visible.

What does Mixed Reality mean to you? What does it make possible?

It is not just the language and meaning but also shape, form, body, architecture.

Simply said, we have observed that there are two types of people: those who search for the structure and those who search for meaning. The first want to use the words to build sentences, poems, a story or an image. The second type are those who go in more physically, who approach the words and symbols as objects, building blocks; they start to build and play with physical structures, scaling, relation to other bodies, they dont care so much about the meaning.

Text in a three-dimensional space is not only read but entered. It is no longer just meaning but also scale, form, colour, position, and movement. Words gain a body. They can be monumental or intimate, static or dynamic, and their relationship to the surrounding architecture changes according to where the reader stands and how they move. In this situation, reading becomes both interpretative and physical. Participants engage with the semantic content of words, but they also encounter them as spatial objects to move among, rearrange, activate, or avoid. Words can appear, disappear, confront the reader, or withdraw. Meaning remains important, but it emerges through embodied experience rather than preceding it. This creates a fundamentally different relationship to text, one in which reading becomes a form of navigation and movement becomes a form of interpretation.

One question that continually returns in our work is how to choose the words themselves. If the text is too semantically loaded, the dramaturgy becomes overly determined and participants lose some of their freedom to create meaning. If the words are too arbitrary or abstract, the opposite can happen: the sense of engagement and play begins to disappear. Finding the balance between openness and direction remains one of our central artistic challenges.

For this research, we drew inspiration from two texts written from different perspectives but emerging from the same historical period: 

 by Willem Frederik Hermans. Both deal with the profound psychological transformations brought about by the Second World War. We were interested in what would happen when such politically and emotionally charged material was placed within a playful mixed reality environment. Would the text retain its weight? Would it provoke discomfort, fear, or resistance? Or would the playful framework dissolve its meaning?

In a way, both things happened. The atmosphere that emerged was undeniably dark, melancholic, and inescapable. Yet the playful and interactive nature of the experience introduced moments of lightness and curiosity that allowed participants to approach the material from unexpected angles. Rather than diminishing the subject matter, the mixed reality setting seemed to create a different mode of engagement with it.

This leaves us with a question that feels increasingly relevant. Could this combination of technology and artistic practice offer new ways of approaching difficult historical and political realities? Could playful interaction become a means of engaging with subjects that are otherwise overwhelming, especially in a region where the consequences of war no longer feel distant or abstract?

What have you discovered by watching audiences interact and play with moveable texts?

We explore the atmospheres, the textures, smells, etc, that can trigger your imagination and creative mind, orimmerse you into someone else's story.

It gives back the ability to play and imagine, dream, go deeper than the surface that is so much missed those days. It activates audiences' bodies and frees people from this constant how do i lookmode, the constant outer gaze. It gives you the freedom to follow your own attention and make your own choices…

The VR headset works as a mask, but not as a mask behind which to hide or become someone else, but maybe more and truly you.

In an area which is so fully visual and thats why there is not much room for imagining… With one word or symbol, adding color and size, we can awaken so many images and memories, travel into past, present and future.

For this iteration, we worked with a range of physical materials, including paper, charcoal, and fresh flowers. A television set occupied a central place in the space, functioning almost like a window into a third reality. Since text was also present in the virtual layer, we wrote directly onto the walls with chalk. By illuminating these texts and adding further textual elements in the virtual environment, we created moments where the boundary between the physical and the virtual became difficult to locate.

One of the biggest surprises for us has been how important the physical space actually is. At the beginning, we imagined that mixed reality performances would be highly portable: put the headsets in a backpack and touring is solved. In reality, the physical environment turns out to be one of the key elements of the experience. If a space feels inviting, encouraging, and artistically meaningful, people participate with much greater openness and joy.

For this reason, I increasingly find black box theatres among the least interesting spaces for mixed reality performance, even though they remain extremely useful during development and research. Much of our work at the academy took place in a black box because technical setup is straightforward and everything can be changed quickly. But when presenting performances, we are often drawn to semi-public spaces such as libraries, where the existing atmosphere becomes part of the dramaturgy. In November, for example, we will perform in a crypt. These kinds of spaces do not simply host the work, they actively contribute to it.

What is your approach to scenography, across physical and digital realities?

One of the most important things we have learned is that technology should not be the starting point. Even though we work with advanced tools, our thinking begins elsewhere. Both of us, as well as our developer Norbert Pape, come from backgrounds in dance and choreography. We tend to approach mixed reality from inside the experience, from the body. We ask how attention moves, how people relate to one another, how space is perceived.

Of course, the technical side matters. We build our mixed reality environments in Unity and currently work with Meta Quest 3 headsets. One thing we learned very quickly is that these devices are not neutral. They respond to the qualities of a space: light conditions, reflective surfaces, room layout, and internet connectivity all influence what is possible. In our multiplayer setup, every headset is connected to a server, which means a stable network becomes part of the dramaturgy whether we acknowledge it or not. Testing is therefore not simply a technical procedure. It is a way of understanding the relationship between technology, bodies, and space.

If I were to give one piece of advice to artists working with emerging technologies, it would be to involve people as early as possible. Test with different audiences. Observe carefully. Ask questions. Listen to what participants actually experience rather than what you hope they experience. New technologies can easily become surrounded by mystery, but we have found the opposite approach to be much more productive. Make the technology visible. Explain how it works. Invite people into the research process.

The most valuable conversations often happen when audiences feel comfortable enough to be honest. Shared experimentation, shared curiosity, and even shared confusion can become part of the artwork itself. If we want audiences to engage with unfamiliar artistic forms, we need to create spaces where people feel welcome to explore, question, and play. The future of the field will not be shaped by technology alone, but by the communities that grow around it.

What did you learn through the process of creating 'History. Her story. Hi, story!'? 

Dortmund brought us closer to some of our central questions, while simultaneously revealing new ones. We are increasingly interested in how dramaturgy can function in a multiplayer environment where participants are free to move, make choices, and shape the unfolding experience. What holds such a space together? What kind of stories belong to an extended reality? What is the role of the performer?

We dream of a performative situation where physical space plays one existential part of dramaturgy and the virtual space another – and they merge perfectly from one to another, very poetical, meditative, physically moving and playful piece of art, space to be.

Which new questions do you hope to answer with future works?

Liis Vares

Taavet Jansen

In History. Her story. Hi, story!, Liis Vares and Taavet Jansen explore dramaturgically meaningful ways of connecting physical and virtual space within mixed reality performance. The virtual layer of a staged MR experience is something the artists already had strong understanding of, but the integration of the physical environment provided an exciting area of exploration. They investigated how materials, spatial qualities, lighting, projections, sound design, and meaningful objects can be woven into a coherent MR dramaturgy. Their primary research method was bodystorming: creating mixed reality compositions and repeatedly testing them through embodied performance and experimentation.
___________________________________________________________
Supported by the Estonian Academy of Arts, Vares.digital, and the University of Tartu Viljandi Culture Academy

RESEARCH PLATFORM

Academy for Theatre and Digitality

History. Her story. Hi, story!

Liis Vares Taavet Jansen

Mixed Reality Movement Research Hybrid Spaces 2025
completedCooperation

Introduction

Similar Projects

History. Her story. Hi, story!

Liis VaresTaavet Jansen

Mixed RealityMovement ResearchHybrid Spaces2025completedCooperation

Introduction

Similar Projects

Liis Vares Taavet Jansen

Mixed Reality Movement Research Hybrid Spaces 2025
completedCooperation