is the knowledge-sharing platform for the 

. Explore the experiments and bold ideas behind the Academy's Fellowship programme, delve into the science behind the projects of our HIDA researchers, and explore the results of international collaborations.

is a platform for discovery and documentation. Each 

showcases a unique artistic-technological research project, with topics as diverse as the use of AI in theatrical design, choreography for and with robotics, and the possibilities for theatre in virtual spaces. Each 

 offers the chance to learn more about the members of our artistic research network. By pooling our knowledge in such a way, creative research into theatre, performance and digital technologies can become more accessible, sustainable, and artistically innovative.

Funded by the Federal Cultural Foundation, the Academy will award a total of 54 fellowships between 2019 and 2022, each lasting for five months. The 

 are designed for people interested in artistic-technical research which explores the possibilities for digital narratives in theatrical spaces, as well as their practical implementation. Fellows have researched a variety of digital methods (motion capture, coding, XR, VR, AR, sensor technologies, robotics, and more) across artistic fields including opera, dance, orchestra and drama. 

Helmholtz Information & Data Science Academy

(HIDA), the Academy for Theatre and Digitality has offered joint research fellowships at the interface of digital art and cutting edge scientific research since 2021. The HIDA fellowships are kindly supported by the 

Artificial Intelligence

Installation

VR (Virtual Reality)

Puppetry

Interactive

Stage Design

Dance

Sound Design

Theater Dortmund
Theater Karree 1-3
44137 Dortmund
Germany
Fon: +49 (0) 231 226 14330

Address and location of the Academy for Theater and Digitality:
Speicherstrasse 17
44147 Dortmund
Germany

E-Mail: akademie@theater.digital
Web: theater.digital

Tax number: 314/5702/9367
Sales tax identification number: DE 124643894

Management Team:

Executive Director
Tobias Ehinger

Administrative Director
Martin Lizan

Director of the Opera
Heribert Germeshausen

Director of the Ballet
Xin Peng Wang

General Music Director
Gabriel Feltz

Artistic Director of the Schauspiel
Julia Wissert

Director of the Children's and Youth Theater
Andreas Gruhn

Director of the Academy for Theatre and Digitality
Marcus Lobbes

Please also note our privacy policy.

 as the sixth division of Theater Dortmund (hereinafter referred to as "Theater", information about Theater Dortmund can be found 

), is pleased that you are visiting our website. Data protection and data security when using our website are very important to us. We would therefore like to inform you at this point, in accordance with Article 13 of the EU General Data Protection Regulation (DSGVO) and Section 13 of the German Telemedia Act (TMG), which of your personal data we collect when you visit our website and for what purposes it is used. Since changes in the law or changes in our internal company processes may make it necessary to adapt this data protection declaration, we ask you to read through this data protection declaration regularly. The data protection declaration can be accessed, saved and printed out here at any time.


§ 1 Scope of application

This data protection declaration applies to the Internet offering of Theater Dortmund, which can be accessed under the domain www.theaterdo.de and the various subdomains (hereinafter referred to as "our website").


§ 2 What is personal data?

Personal data is individual information about personal or factual circumstances of a specific or identifiable natural person (data subject). This includes, for example, information such as your name, address, telephone number, date of birth or e-mail address. Information for which we cannot establish a reference to your person, e.g. by anonymizing the information, is not personal data.


§ 3 What personal data do we collect/process and use?

Access data without personal reference is stored, such as the name of your Internet service provider, the page from which you visit us, the names of the requested files and their retrieval date. These data are evaluated exclusively for the improvement of our offer and do not allow any conclusions to be drawn about your person. 

If you wish to make use of services offered by us on our website, such as purchasing tickets via the ticket store or ordering a newsletter, etc., it is necessary for you to provide further data. This is the data that is required for the respective processing, such as your payment information when purchasing tickets or your e-mail address when ordering a newsletter. 

We process the data we collect about you for the provision of the services we offer, to answer your questions and to operate and improve our website and applications. 

We process your personal data exclusively to provide you with a comprehensive service via our website or, if you use our contact form or, to process your ticket purchase or, to process your application or, to provide you with requested information about our offers.

Fellowship

HIDA

discover cutting-edge digital research in theatre and performance 
meet our research fellows and collaborators in the 

Portal: Academy for Theatre and Digitality

Besides the aesthetic implementation, we are interested in two main elements:

1. How can non-verbal communication between space (built from sound and scenography) and spectators imply a plot line or narrative?

2. How can the environment's own characteristics play a role, so that they can allow conclusions to be drawn about motivations and correlations? 

Jakob Lorenz

Stella Lennert

'PALIMPSEST' engages with virtual space as an interactive stage, and explores narration through interaction instead of representation. For this, the project takes theatrical literature as an anchor and combines it with elements of gaming. The dramatic texts act as a common ground, a guideline through unfamiliar virtual environments that follow strange rules. PALIMPSEST is experienced via VR goggles and headphones. 

PALIMPSEST

Drama

2022

How to depict violence in theatre?           

Without reproducing violence?
Without aestheticizing it?
Without trivializing it?
Without inflicting violence on others while trying to find answers?

>>>>>>> Thoughts and insights from this search <<<<<<<


In the beginning of 2023 there is this vague vision for a setup which gives me the freedom to experiment with different aspects of the depiction of violence:

a human-sized puppet | its digital face-double | a green screen studio

___________________________________________________________________________________________________


I knew right from the start that I want to work with a puppet.              Firstly, because I think that this is a logical approach to the question of how to depict violence in theatre without inflicting violence on others while trying to find answers. The puppet itself feels nothing. And since I am going to animate the puppet myself, I am autonomous - and bypass putting others in situations of violence.
But above all, I choose the puppet because I am obsessed with the effect it has on the viewer.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

RELATIONSHIP BETWEEN SPECTATOR AND PUPPET

THE SPECTATOR`S (LIFE-BRINGING) GAZE
Although the puppet is obviously artificial (manufactured & dead), the subconscious draws a line from the viewer's body to the puppet's body and brings it to life. 

The uncanniness inherent in the puppet is partly due to its resemblance to humans.

THE PUPPET AS 'DESTROYER' OF THE BODY ORDER
This process creates a kind of feedback: we are thrown back on ourselves when looking at the puppet - the sight of it destroys the illusion of the perfect, complete body.

>>> The act itself is already a violent one.

>>> The phenomenon of the puppet lies in the bodily consternation it triggers in us.

What is the puppet's digital face-double?

I am driven by the idea to combine two components - analog theater work and digital innovation - to create a character that serves as a test subject for me in an attempt to research violence.
I asked myself how I could enhance the strange liveliness of the puppet.
Then I had the idea: what if the puppet can speak for itself?
What if I 3D-scanned the puppet's face and then animated it, let it talk?

The original goal at the time was to merge these two components - and create a hybrid character.
An analogue puppet with a digital face - united in one body within a video work.


By working in the green screen studio, I can put the puppet into different local contexts which gives me a endless freedom.
I´m wearing a green morphsuit so I can erase myself from the material.
The puppet moves as if by itself.

In the end, I want to have a character which is half analog and half digital puppet. It moves and speaks by itself, acts in different "worlds" - which consist of videos, images, different sounds and noises that I collect.
_____________________________________________________________________________________________________

What should the puppet I will be researching with look like?
Do I want to set a clear frame with its appearance? Prescribe interpretations?
What type of material do I want to use?
What must the puppet be able to do - how should it move?

//collaboration with puppeteer and puppet maker Emilien Truche


Since I've never built a human-sized puppet that can move like a human and be animated by just one person, I worked with a professional. Emilien was there to advise me (I had the ambition to really build the puppet myself). He was recommended to me by a friend who works at the 

.
His first questions were mainly about what postures the puppet should adopt and what expressions it should master. In order to be able to communicate more clearly with each other, I looked for sculptures, paintings, and photographs from the news and social media that tell a story of violence in their posture. 

//studying Sicilian puppet theatre - opera dei pupi


Before I applied for the fellowship, I had planned a short trip to Sicily in October. To study and research Sicilian puppet theater there.                   It was important to me to first study a traditional form of puppet theater intensively in order to incorporate what I had learned into my work. What do I take away from what I have seen? What do I explicitly want to do differently?                                                                                               The Sicilian tradition is one that I can experience and explore in life - although of course it is by no means the oldest form of puppetry (as it developed in the first half of the 19th century). A major component of the Opera dei Pupi is the violence: swordfights, tournaments and battles with monsters, sorceresses and enemies.
This research trip was really insightful and inspiring for my work on „Researching Violence“ - and perfect for this point in the work process. I had the chance to dive deep into the portrayal of violence through (traditional) puppet animation and to get involved with puppet making and puppetry.

I had the opportunity to see several performances, paying particular attention to the study of the gestures and postures of violence.

In addition, I was lucky to visit various studios. I was able to talk to the puppet makers themselves, ask questions and even try out the puppets myself and study them up close.
For example, when a character strikes with his sword, the opponent can be split in half, fall apart or lose his head. In the puppet maker's (Italian: oprante) workshop I was able to study the mechanisms and construction of the puppets at close quarters.

// building up my 'archive' (by collecting sounds, texts, interviews, images and video material of places)


From the very first day, I collected material for my archive for the experimental setup.
I was interested in building up a broad spectrum and I mainly used my iPhone for the documentation. Regardless of whether it was video recordings of
- places and spaces that are commonly associated with violence
or as a counterpart to it
- places and spaces that are commonly associated with peace
- I also tried to find 'neutral' places
- as well as places that I personally associate with violence and peace

examples of sound recordings I recorded:  

- Munich Central Station during the Oktoberfest
- bee colony in Düsseldorf
- shopping mall in Dortmund city center
- eerie sounds at night on the way home
- cicadas by the wayside in Palermo                                                             - sounds of the sword fight of the dolls in Sicily


- I was lucky to be able to attend a Blender workshop (by Elisabeth  Drache, 3D artist and a member of the duo The Ruhrgebieterinnen) at the Academy, which proved useful for the next steps in animating the 3D scan.

NOVEMBER+DECEMBER 2023// ANIMATION + COMPOSITION                         

At the beginning of November I took part in a motion and face capture workshop as part of the DIVE FESTIVAL OF IMMERSIVE ARTS (In cooperation with the Figurentheater-Kolleg Bochum). Irina Shutova and Warja Rybakova (both working for DOCKdigital BERLIN) drew my attention to the plug-in Faceit from Blender after I told them what I was planning to do with my puppet. The workshop also gave me a better idea of what I need to pay attention to when animating the handmade puppet in the green screen studio.

//animating the puppet in the green screen studio

3 days of intensive exchange, reviewing the video material, discussing.

WORKING THESES 
a) the puppet itself feels nothing
b) the puppet is not able to play in comparison to the actor or actress
>>> Which means e.g. that when an actor dies on stage, they are only pretending. This can still touch us, but we know that they don't really die. The puppet dies the moment it dies. Right before our eyes.<<<
c) the puppet is familiar to every human being and therefore accessible (in the sense of not elitist)

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++



With this first material of the human-animated puppet in the green screen in front of me, I began to go through my archive (with sounds, videos etc.) and experiment with putting the individual components together.


In order to be able to act as quickly and flexibly as possible, I worked with Final Cut. On the one hand, to be able to cut out the puppet with one click and, on the other, to be able to put my video and sound material together as simply as possible.                                                    At the same time I am working with Blender and the plug-in Faceit to animate the puppet´s face. Since I had hardly any Blender experience, I always used apps (with AI-based animation tools for faceplay and deep fake, like Tokking-Heads or Reface) in between to be able to check ideas quickly.


The composition of the individual parts allows contrasts to be created between the puppet's expression, physical movements, spoken text, sound and setting.
How do expressions of horror that really reach us come about?
When does empathy arise, when does alienation?

We have access to so many images of atrocities that the individual images lose their impact. My working hypothesis is that both a puppet and a completely artificial person (avatar) have the power to sensitize us as viewers to the experience of visual and linguistic violence. The puppet and the avatar feel nothing; we create their suffering, pain and exhaustion solely through our imagination. This creates a direct connection between the viewer and the representation.

// REALIZATION OF THE IMPOSSIBLE
The days of trial and error made me rethink my approach completely.   

                                                                                                                      I had the feeling that - with everything I tried - I was aestheticizing. And thus trivializingviolence.


>> You can't depict violence without reproducing it in some way. <<

I also couldn't get out of the vicious circle of thinking that everything is just my personal view. How do I get to the point where something becomes more universal? To become a little more detached from myself and my feelings?

And who am I to talk about this?
What position do I have within the "narrative of violence"?

These doubts made me take a fresh look at the material I had collected - and think again in a different direction. 

new thoughts: The relationship between puppet and puppeteer is already a violent one in itself. There are clear power structures as to who controls whom.

Where does the relationship between puppet and puppeteer tilt?     

Hanna Rode

How can we tell about human abysses without reproducing or trivializing violence? This question is the starting point for an experiment with a handmade puppet and its digital face double.

Researching Violence

2023

I am interested in immersion, the "reality" part of Virtual Reality. As CyberRäuber, Marcel and I have tried and created various avenues into immersion. A very short summary of our findings would be that a high degree of immersion is generally not necessary to tell a story or to convey what you want to. And that immersion generally suffers if you aim for too much realism: the more you rely on your target's collaboration, their imagination and established social and cultural norms, the more they will get "into it". Participation / interaction trumps detail, trumps realism.

But of course, the more you let people interact, the more you lose the the ability to steer a narrative. In this dilemma we usually opted for limited realism (high aestheticization), limited interaction, a limited scope of character, space and time, and worldbuilding as a narrative strategy.



 as an idea manifested itself over the course of years. It basically grew on the compost of CyberRäuber's work in theatre, in which we usually didn't have the time, scope and resources - and more importantly - not the framing for such a project.



 aims to be maximally immersive with a given, limited technological effort, following an approach of a minimally detailled realism, maximal imagination and direct, personal contact.

Hundred thousands of years of evolution have led us to develop certain quirks that can help storytellers create compelling narratives, something that can also be ab-used, of course. Think about what we imagine in clouds, in the shadows of a forest lighted by flickering flames of a campfire. What kind of faces we see in objects, landscapes, moving leaves, what voices and sounds we hear in the wind. The less there is to see or hear, the more we augment; little impulses, visual or aural clues are launch pads for our imagination, something every horror movie, every mystery story told on a nightly walk makes use of.



One way of direct, personal contact is the human voice. It can supply content and - with language, dialect, wording, timbre and emotion - context. We ascribe very much to a voice that could only be safely verified by meeting and conversing with the person with this voice.
In this project, the means to deliver the voice will be a recording, a voice message. Why? Because it is a controlled way of communication. For the ordinary person an ordinary conversation is something that can be quickly understood and deciphered, the situation, the conversation partner can be assessed. We would normally aim to reduce mystery, to understand what is happening and only continue to engage with the person and the topic if intrigued and feeling save. A voice message instead can be mysterious, incomplete, poetic, out of the norm. It can also contain additional information, environmental sounds. A voice message can be dilligently tailor-made for a situation and our goals. It doesn't need and anyway couldn't deal with an instant reply. The communication here isn't a realtime dialogue, but an elusive conversation over a longer timespan, similar to an infrequent exchange of letters, including lost and/or overlapping messages.




Nevertheless the recipient has to have the feeling of being heard, of being reacted to. And as in any communication, truth and trust plays an important role...

During the fellowship different areas have to be explored:

Voice - characteristics, difference to simple text

AI - what can be done with artificial neural networks? Story, text to speech, speech to text...

Soundscape - binaural sound, artificial and recorded; AMSR (Autonomous Sensory Meridian Response)

The role of time (length, pace, intervalls,...)

As was to be expected, my research plan didn't survive contact with reality. I thought the main focus of my research would be on "Soundscape" and "Story". Instead, most of my time was spent on voice and specifically the creation of text and voice with neural networks, with timing and the delivery system of the messages (AdaptorEx, Telegram). 

"Soundscape" didn't play the expected role due to the realization that interaction with smartphones, the chosen device, is mainly happening visually, and when people listen to voice messages, they oftentimes don't use higher quality headphones and will be surrounded by other environmental audio. After creating and afterwards checking out some binaural and ASMR material on the phone it didn't seem relevant any more in the scope of this research. It is one of those engineering issues that can take a lot of time and are fun to work on, but ultimately don't deliver for an audience.

"Story" got deprioritized when I realized that neural networks, or more precisely large language models (LLM), would take over a large part of creating the narrative. 
Here maybe some further explanation is in order. As CyberRäuber, we've been working on AI in theatre since 2018. In our so-called "neural theatre"-pieces (Prometheus unbound, Linz 2019; Der Mensch ist ein Anderer, Wiesbaden 2021; Mensch am Draht, Ingolstadt 2022; The Merge, Bangalore 2023) we aim to unleash the creative power of generative AI. The text the actors speak and play is created in real time by LLMs and transmitted to in-ear speakers via text-to-speech. You could also call this a text-to-body-mechanic, the actors repeat and interpret what they hear and thereby embody the AI. The process works amazingly well. While still maintaining parts of improv theatre, this isn't improv because the actors follow the logic and emotional content of the texts. Actors can never lose focus, have to stay "on" all the time and are oftentimes visibly struggling with their task, because the text will always be new and unknown, its twists and turns unforeseen. This is fresh material! The audience on the other hand usually themselves struggle with the uncanniness of the whole set-up, try to follow what's happening on stage, enjoy the actors' struggle and they take on much more responsibility, because the play develops on stage and in their heads.

So, AI is already playing a crucial part in CyberRäuber's artistic methods, and we've been following the latest developments in the field on a daily basis. And with the current generation of LLMs (2023/24) we now actually possess very powerful storytelling tools. The question was: is one person alone capable of creating, editing, curating large amounts of coherent, stylistically appealing, sensible text? Can a network convincingly take on the role of a narrator? To check and ultimately prove this thesis became one of the main isssues of the research at ATD.


This documentation is an ongoing process and will grow as the fellowship progresses. Last entry: 30. November 2023

Björn Lengers

You just received another one of those mysterious audio messages that strangely overlay your surroundings and ask you to perform a seemingly insignificant action: to walk a little slower, to press the pedestrian lights even though you don't actually want to cross the street, something else. A voice wants you to believe that you are decisively changing the future.
- - - Resonance is an interactive, immersive and theatrical narrative, which can be experienced by many people independent of time and place on their smartphones.

Resonance

The human being is not generally thought of as a staple food source.
Observing the anatomy and skeleton, one can see that the animal is neither built nor bred for its meat, and as such will not provide nearly as much flesh as a pig or cow.
The large central pelvis and broad shoulder blades also interfere with achieving perfect cuts.
There are advantages to this however, especially due to the fact that the typical specimen will weigh between 100-200 pounds, easily manipulated by one person with sufficient leverage. 

'Butchering the Human Carcass for Human Consumption'

In March 2001, 39yo German IT professional FRANKY and 43yo German engineer CATOR made an appointment on the internet. After meeting at Kassel main station for the first time in “reality” after their extensive online communication, FRANKY drove the two men to his house, where he cut off CATOR’s penis - which proved too chewy for the both of them to eat -, proceeded to kill him with a stab to the throat and then stored the pieces of meat from CATOR’s cut-up body in the freezer. Both had previously agreed to this, for while FRANKY longed to fully internalize another human being by eating it, CATOR wished to be fully disintegrated and disappear. The two men recorded parts of their meeting, wherein they were constantly forced to synchronize their quite different imaginations of the encounter with the reality of their shared physical and social situation.

15 years later, Taiwanese electronics producer HTC cooperated with US company Valve for launching a virtual reality headset they had christened “Vive”. Together with similar products, it marked the beginning of a slow mainstreaming of VR applications - a technology long envisioned with a moniker first coined by French playwright Antonin Artaud, the proponent of a “theatre of cruelty”. The “Vive” headset enabled its users to move in a small space within a computer-generated surrounding and interact with it. VR experiences proved to be highly immersive - and for many people at first physically quite disorienting, because they entailed being present in two realities at the same time. The headset was originally meant to be called “reVive” - but the name was changed before it hit the market; the notion of a new wondrous device transcending death or mortality thus obscured.

Subject Matter: Interviews, Autonarrations, Artistic Production

“Virtually incomprehensible - but he was happy, sitting there in his own blood.” In an interview, FRANKY thus comments on the sight of castrated CASTOR sitting in his bath tub. The desire for understanding, imaginations of fulfilment and redemption and their conflict with reality are recurring themes of the Rotenburg cannibal case. “Whatever you imagine, in reality it is basically always different,” says FRANKY later in the same interview, formulating a core experience of artistic work - only to dismiss it at once, chastened by his own experience: “Why should I do something, knowing it will never be like I imagined?”

Interviews and autonarrations by FRANKY form part of this project’s material as well as protocols of chat conversations by the two involved. Research also covers cases of other confirmed sexual cannibals (Jeffrey Dahmer, Issei Sagawa, Arthur Shawcross, Albert Fish…) as well as an extensive collection of images, videos and writings by people from the internet community of cannibalism fetishists - i.e. the artistic production of the scene itself. Here, not the shock value - the 

 - is of interest, but rather the genre’s recurrent figurations and formal rules. What are the conventions for depicting the not only unconventional, but outright taboo? An additional testament to the fundamental discrepancy between imagination and reality are documents of the thousands-of-years-long history of insinuations and mise-en-scènes of cannibalistic desire, which 

: From Saturn’s cruelty and St. Erasmus’ suffering, collective colonialist projections on the “other” and the monistic ecstasy of Kleist’s Penthesilea to Hannibal Lecter’s unsettling pop appeal. These may also hint at the function of said discrepancy.

The question remains: Why imagine something, knowing I will never fully understand it?

Working with “cases” that are in one way or the other “radical” or “extreme” is always in danger of giving in to sensationalism in representation and deprecation, belittling or denial in reception. Not surprisingly, the cannibalism case of 2001 has already undergone wholesale journalistic and artistic processing - not least, it inspired numerous films and theatre plays. Its spectacular value has been cannibalised for good. Unlike real flesh though, memories and fantasies are not destroyed through consumption. Today, rather will they be externalized digitally and reproduced forever.

The subject matter of sexual cannibalism is especially suited for performative VR application because it reformulates fundamental problems of the medium itself: the experience of being physically absent and present simultaneously, the relationship of a technologized world (both men came from technical professions) to the “flesh” we inhabit this world with, not least the desire for encounter and singular experience that is not only the basis of 

 performance and of the arts’ hunger for transgression, but for so many of today’s practices and services. VR as a medium - and XR/MR as its follow-up - performs such transgression by first and foremost creating the experience of being alive and present with one body in two distinct spaces at once - the “virtual” and the “real”. It is a lucid challenge for our consciousness, comparable to the first cinematic experiments - and a challenge for theatre, too.

The fellowship time will be used for research and technical try-outs regarding the setting, 3D visualization und interaction of the planned piece. Self-education on Unity 3D's shader graph, re-creating the physical performance space in VR & avatar creation and live control will also be part of the work. In the unlikely case of a global pandemic interfering with schedules, efforts will be made to boil down plans in a way so as to still achieve a presentable outline of the project involving VR elements.

Development will focus on cable connected higher end consumer HMDs, specifically on HTC Vive / Steam VR. Besides a Vive/Vive Pro headset with base stations and controllers, at least two Vive trackers will be part of the setup (more if used for performer body mocap). Wireless microphones for performer(s) and a computer being able to run the VR elements are also necessary.

The piece will reproduce a model of the actual physical performance space in VR, so knowledge of it and development time to produce the appropriate design beforehand as well as enough setup time to ensure correct alignment of virtual and real space are necessary.

As of now development aims for a ca. 30 minute 1-on-1 performance alternating between real and virtual space in the course of its narration.

Arne Vogelgesang

A guided, interactive VR performance which explores transgression, sensationalism and spectacle through the subject matter of sexual cannibalism. This topic is especially suited for performative VR application because it reformulates fundamental problems of the medium itself: the experience of being physically absent and present simultaneously, the relationship of a technologized world to the "flesh" we inhabit this world with, and the desire for encounter and singular experience that is the basis of one-to-one performance and of the arts' hunger for transgression. It is a lucid challenge for our consciousness, comparable to the first cinematic experiments - and a challenge for theatre, too. 

New Flesh

Performance

Avatars

Motion Capture/Tracking

2020

, is an artistic/methodological movement research. 

The aim of this research project is to take core principles of (analogue) puppetry animation and to translate them on the moving/dancing human body using motion capture techniques, in order to explore how this movement method can increase the experience of aliveness of 3D avatars, by making their movements more organic and establishing them as breathing/alive creatures.

I am inspired by, and draw upon my background as pluridisciplinary artist, with a focus on dance, movement, and puppetry/object theater. 

I'm interested in approaching 3D avatars as another form of body language with its specific dynamic and needs, which also implies other possibilities and limitations. For this language, the human body acts as the vector as well as a translator. The fundamental principles of puppetry animation are considerably different from the human way of moving regarding impulses, dynamics, rhythm, accentuation, tension, stop/echoes of the movements, etc. When these principles of puppetry are applied to the human body, and the resulting movements are used to animate an avatar, an act of translation occurs. It results in more precise movements and sequences, and the impression of ‘organic’ movement, because the avatars are established as ‘breathing’ and ‘alive’, which is the core of puppetry’s art.

You will find here the long interview about the Research and the Process itself:

{ More to come here, about the Process, the Whys, the Discovery }

As a showing from this research, ‘A_live, became a video installation on 3 canvases about those movements, impulses and breath, accompanied by live music.  

The setting is composed as follows: Movement sequences, recorded using motion capture techniques, are projected on the three canvases. The embodiment of this exploration is interpreted by three different avatars, each one with its own texture and characteristics. One avatar has a water texture, which has an ‘explosive’ capacity related to the strength and speed of the movement impulses. The second avatar is more ‘typical’: a humanoid shaped form, yet connected to a cloth which is reacting intricately and precisely to the avatar’s impulses, creating a ‘wind’ feeling as well as an extension of the breath’s embodiment. The third is made of particles which leave traces of light following each movement.

The emphasis of this visual experience is on the quality and precision of the movements, and the different impulses, explorations and nuances embodied by the avatars: How the different textures are reacting to those elaborated momentums, innovative way of moving and to the avatars breath are the leaders of this experience. Each texture with its respective features reacts in a different way to the impulses of the performer, creating various and specific visual echoes in harmony with one another. 

Thoughtful dramaturgical choices and precise editing creates an additional layer of choreography, from the timing of the appearance (and disappearance) of the sequences upon the canvases; the smoothness (and suddenness) of the transitions; the synchronicities and delays between the sequences which underline the specificity of the each movement or breath. We are invited to witness each avatar’s individual breath as well as experiencing a collective organic entity as it moves and breathes as one.

The animation’s cameras are connected to the pelvis of the avatars, which is to say, to the core of the movements. In fact, it is important to note that absolutely no extra effects were added, and there was no need at the end to clean the files. The cleanliness of the movements, and the possibility overall to use ‘raw’ files in this way, are the direct results of the method I’ve explored and developed. 

Although this showing is the first creative response to my research; the frame became not only an explicative presentation of the project so far, but an immersive installation in its own right. It was important to me to create a unified, elaborate setting, and to invite the audience to feel immersed in an atmospheric ‘island’: connecting with senses, finesse of movements, beauty, breath and ultimately a deep exploration of aliveness through external bodies.

                        If you wish to discover more about the installation itself, I invite you to consult the project description on my website. *

Concept, Movement Research, Embodiment: 


Baris Pekcagliyan / @0bapek
Beka Tcharbadze / @bekaarba
Alptuğ Çavuş / @talkingrocks

Warm thanks to the Academy for Theatre and Digitality, 
Mario Simon,
& KOPRODUKTIONSLABOR.

Laura Waltz

My fellowship project 'A_live' is an artistic/methodological movement research, and the use of MoCap technology to explore the possible 'organicness' and feeling of 'aliveness' of avatars through animation fundamentals from puppetry.


The aim of this research is to take core principles of (analogue) puppetry animation and to translate them on the moving/dancing human body using motion capture techniques, in order to explore how this movement method can increase the experience of aliveness of 3D avatars, by making their movements more organic and establishing them as breathing/alive creatures.

A_live

Sensors

How can we (as a field) make a dance in a virtual environment so that the audience “feels” the presence of the digital body?

What is needed for the audience to find new aesthetics when dance is mediated with digital environments like VR headsets?

, which explores the performative possibilities of VR/XR technologies through dance. It questions the concept of a 'stage' when dance occurs simultaneously upon hybrid stages (for instance, in both virtual and physical realities) and asks: how can this kind of content be shared with an audience?

 Virtual reality creates an experience where introspection can be achieved. This opens up possibilities for somatic and other movement practices. Working with the body and technology presents the opportunity to explore rigid computer systems through an ontological approach (praxis/doing), and to explore the medium's own possibilities. Often, dancers who work with technology forget their own embodied language (the 'why/how') and get confused and carried away by the pragmatic reality of technology, focusing instead on the content (the 'what'). They forget their own embodied power. The result is a superfluous object.

To tackle this I propose a praxis which privileges the process over the result. I work with avatars and sigils as a ritual practice with a non-anthropocentric approach, and the key aspect is to shift the focus in choreographic strategies. Instead of asking "what to 

This project continues my research into dance in VR that started in 2022 at 

Motion Bank - University of Applied sciences Mainz

 (Project originally founded by choreographer William Forsythe). 

Using a markerless motion capture system I created, I explored and documented simple visual representations of the body (avatars) in VR. 

. I achieved this using a non-anthropocentric approach. In the documentation it was important to use an inclusive, clear language that addressed both dancers and technologists. In the 1st cycle at Motion Bank, the focus was on the dancer.

In the 2nd cycle at the Academy I want to add a layer where I introduce the stage, thus exploring the relation between audience and dancer.

The new performance rather offers different entry points to the audience so that every audience member can have a different interpretation. Nevertheless, the core of the performance remains the same; we investigate a theatrical narrative while having virtual reality as an actor at the center of the stage. In the first performance, we used a linear narrative around VR to convey how we conduct a research. In the second performance, we used a less rigid structure, namely structured improvisation, to show how we develop a performance. It can be seen as a making-of where Jorge runs around the stage and prepares each technical element for a “real” performance although there is no such a “real” performance that comes out of the making-of.

Though our intention is to show behind-the-scenes of developing a VR-performance, we do not want to limit the scope to “struggle of working with technology in theater” because this theme may be too specific in the industry. We try to add embodied elements and add breaks in between to show humanness of a theater maker. The audience can relate this process to everyday technology such as smartphones, smart devices and potentially VR headsets, which are becoming a commodity.

For the overall structure, we divided the performance into small skits so that each skit features one element (e.g, video of eyes, VR meditation app). As a structure, we might think it follows an idea of Fluxus or it is minimalist; nonetheless, in the actual performance, we see that each element is not separable from each other and we can notice the interconnection between the selected element and other actors on the stage that are not mentioned (Jorge’s body, desk, floor, …). This is because working with technology is inherently a complex process.

Between Information and Noise :: a Practice on dance and VR

: To explore different perceptions of realities when dance happens within Hybrid stages - (VR and on-site)

Missing conceptual context to apply technical tools.

The natural empathic and embodied relation between dancer audience is partially lost when using VR headsets. This research aims to help the audience to find new aesthetics (

mediated experience of time created between the artwork, the artist, the interface and the audience

), to find engagement, new meanings and purpose when dance is mediated with digital environments like VR headsets In hope to 

create a visceral sense of unfolding or revelation, but also a feeling of immediacy and loss.

Exercises on subjective perception of time and space. The aim with this series of videos is in questioning, What is happening in real time.

. An action of a person drinking water (action 1) is recorded with 2 different camera perspectives. A setup in OBS is set to switch between 3 different perspectives of the camera; the 2 recordings and a 3rd live feed. A person comes into the space. There is a projection that switches between the 3 perspectives while a person is in the space doing the action of drinking water. This action is recorded. 


Sigil

Pathfinder

Choreographer, Media and Interaction Designer, Researcher

Jorge Guevara

The natural empathic-embodied relation between dancer - audience is partially lost when using VR headsets. This research aims to help the audience to find new aesthetics, new engagement, new meanings and purpose when dance is mediated with digital environments. Through two live performances, I created a visceral sense of unfolding or revelation, but also a feeling of immediacy and loss. 


Sigil :: bringing the inner world out

Choreography

Lighting Design

. We both studied puppetry at the Hochschule für Schauspielkunst “Ernst Busch” Berlin. Nico Parisius worked at the Puppentheater Halle for five years and discovered his interest for creative coding in his spare time. He worked on projects that merges his interest in computer systems with his proficiency on stage. Caspar Bankert is a freelance artist based in Berlin. He combines traditional and modern games and their possibilities with stage plays to create interesting new experiences.

 also studied puppetry at the Hochschule für Schauspielkunst “Ernst Busch” Berlin. She works as a freelance performer as well as in the field of theatre pedagogy for children and adults.

studied history of art in Göttingen and Museology in Amsterdam. She works as a provenance researcher at the Rijksmuseum Amsterdam.

 was to formulate a first draft of what Memory of Things could one day become, as well as to build a working prototype that outlines the requirements we would need to construct a final version. In the ordinary rehearsal schedule of any theatre performance there often isn’t enough time to try out new or just unfamiliar methods of play or interesting technology. Because of the tight deadlines there often is no room for error while testing things that are off the beaten path. We want to use the Fellowship to look at different hardware, software, objects, lights, sounds and narrative structures to create a room which tells their visitors an interactive narrative.

Smart Objects

How To: Setup and Narrative

Requirements: Hardware & Software

Process & Development Diary

Nico Parisius

Caspar Bankert

“Memory_of_things” is a walk-in installation where the audience explores objects in a room which are linked to historical fragments. These objects are equipped with sensors and can register interaction and trigger light, video and sound. The idea is to create a “performance of things”, telling stories about times when no people are left and only objects are the last witnesses. We want to use digital tools to make provenance research (the study of an object's history) tangible.

RESEARCH PLATFORM

Academy for Theatre and Digitality

Disco Planet: Letheia

Do Mayer Hen/i

2023 VR (Virtual Reality)Choreography Installation
completedHIDA

Similar Projects

Disco Planet: Letheia

Do MayerHen/i

2023VR (Virtual Reality)ChoreographyInstallationcompletedHIDA

Similar Projects

Do Mayer Hen/i

2023 VR (Virtual Reality)Choreography Installation
completedHIDA