is the knowledge-sharing platform for the 

. Explore the experiments and bold ideas behind the Academy's Fellowship programme, delve into the science behind the projects of our HIDA researchers, and explore the results of international collaborations.

is a platform for discovery and documentation. Each 

showcases a unique artistic-technological research project, with topics as diverse as the use of AI in theatrical design, choreography for and with robotics, and the possibilities for theatre in virtual spaces. Each 

 offers the chance to learn more about the members of our artistic research network. By pooling our knowledge in such a way, creative research into theatre, performance and digital technologies can become more accessible, sustainable, and artistically innovative.

Funded by the Federal Cultural Foundation, the Academy will award a total of 54 fellowships between 2019 and 2022, each lasting for five months. The 

 are designed for people interested in artistic-technical research which explores the possibilities for digital narratives in theatrical spaces, as well as their practical implementation. Fellows have researched a variety of digital methods (motion capture, coding, XR, VR, AR, sensor technologies, robotics, and more) across artistic fields including opera, dance, orchestra and drama. 

Helmholtz Information & Data Science Academy

(HIDA), the Academy for Theatre and Digitality has offered joint research fellowships at the interface of digital art and cutting edge scientific research since 2021. The HIDA fellowships are kindly supported by the 

Artificial Intelligence

VR (Virtual Reality)

Interactive

Sound Design

Projection/Mapping

3D Printing

XR (Extended Reality)

AV (Audiovisual Design)

Theater Dortmund
Theater Karree 1-3
44137 Dortmund
Germany
Fon: +49 (0) 231 226 14330

Address and location of the Academy for Theater and Digitality:
Speicherstrasse 17
44147 Dortmund
Germany

E-Mail: akademie@theater.digital
Web: theater.digital

Tax number: 314/5702/9367
Sales tax identification number: DE 124643894

Management Team:

Executive Director
Tobias Ehinger

Administrative Director
Martin Lizan

Director of the Opera
Heribert Germeshausen

Director of the Ballet
Xin Peng Wang

General Music Director
Gabriel Feltz

Artistic Director of the Schauspiel
Julia Wissert

Director of the Children's and Youth Theater
Andreas Gruhn

Director of the Academy for Theatre and Digitality
Marcus Lobbes

Please also note our privacy policy.

 as the sixth division of Theater Dortmund (hereinafter referred to as "Theater", information about Theater Dortmund can be found 

), is pleased that you are visiting our website. Data protection and data security when using our website are very important to us. We would therefore like to inform you at this point, in accordance with Article 13 of the EU General Data Protection Regulation (DSGVO) and Section 13 of the German Telemedia Act (TMG), which of your personal data we collect when you visit our website and for what purposes it is used. Since changes in the law or changes in our internal company processes may make it necessary to adapt this data protection declaration, we ask you to read through this data protection declaration regularly. The data protection declaration can be accessed, saved and printed out here at any time.


§ 1 Scope of application

This data protection declaration applies to the Internet offering of Theater Dortmund, which can be accessed under the domain www.theaterdo.de and the various subdomains (hereinafter referred to as "our website").


§ 2 What is personal data?

Personal data is individual information about personal or factual circumstances of a specific or identifiable natural person (data subject). This includes, for example, information such as your name, address, telephone number, date of birth or e-mail address. Information for which we cannot establish a reference to your person, e.g. by anonymizing the information, is not personal data.


§ 3 What personal data do we collect/process and use?

Access data without personal reference is stored, such as the name of your Internet service provider, the page from which you visit us, the names of the requested files and their retrieval date. These data are evaluated exclusively for the improvement of our offer and do not allow any conclusions to be drawn about your person. 

If you wish to make use of services offered by us on our website, such as purchasing tickets via the ticket store or ordering a newsletter, etc., it is necessary for you to provide further data. This is the data that is required for the respective processing, such as your payment information when purchasing tickets or your e-mail address when ordering a newsletter. 

We process the data we collect about you for the provision of the services we offer, to answer your questions and to operate and improve our website and applications. 

We process your personal data exclusively to provide you with a comprehensive service via our website or, if you use our contact form or, to process your ticket purchase or, to process your application or, to provide you with requested information about our offers.

Fellowship

HIDA

PlayOn!

discover cutting-edge digital research in theatre and performance 
meet our research fellows and collaborators in the 

Portal: Academy for Theatre and Digitality

17 September, 2021: Transcript of presentation at PlayOn! conference held at the Akademie für Theater und Digitalität, Dortmund

When we began our fellowship at the Academy 10 days ago, we rearticulated our project for this public wiki. Basically the next 10 minutes will be a sort of performative-documentary gesture of ultimately writing out our research questions in this wiki after we have taken you through our thought processes, starting by sharing a brief project synopsis.

“The collaborative research between dancer Kiraṇ Kumār and dramaturg Kai Tuchmann is towards the making of 'Dear Dead Doctor', a work of digital and documentary theatre. In the form of an open letter to Kiran’s late grandfather, who was the first doctor of western medicine in his family, the project explores performative strategies of bringing medical and somatic knowledges into dialogue on stage. This address to the doctor is conceived through Kiraṇs grandchild: a parthenogenetic, proto-progenic being (P3B). P3B will be digitally materialized by drawing on 'data points' from yogic, dance and martial embodied practices. 'Dear Dead Doctor' re-imagines what 'being human in the world' could imply in its continuity across pre-modernities and futurities.”

One of the central concerns of our research is how to understand P3B. This is a complex concern with dramaturgical, technological and philosophical dimensions - which each articulate the concern differently:

Let’s be clear that speaking with the dead is by no way a romantic return to a past. Indeed it is a rational and imaginary gesture to reach outside modernity and towards an altered futurity.

Since Kiran is a dancer, we will make a key assertion that the bearing of embodied practices like dance, yoga and martial arts for our future, does not lie in a continued virtuosic display of skilled individuals. So rather than building our futures on recurrent stories of individual geniuses, through our research we propose that embodiment is a shared intelligence that is sort of compressed into an interpersonal storage mechanism, for example of a dance form transmitted from teacher to student, from performer to audience etc. This transmission of embodied intelligence cannot happen in a solid state of storage, like a fixed national culture or a singular artistic genius. Transmission of embodied intelligence happens through a sublimation of the self, an exchange of vaporous particles between us.

This is a 2 dimensional visualization of our digital starting point for developing P3B, depicting how data can be deduced from embodied practice and materialized in a pixel-cloud:

The 3rd century Sanskrit text Tattva Bodham or Deep Materiality, offers a 3 dimensional conception of being-human. Moving away from the very binary conception of body and mind, this yogic conception instead describes a trinary materiality of gross, subtle, and causal dimensions of being-human. Our gross beings are bio-material substances, our subtle beings are psycho-energetic presences, and causal beings could well be our by now im/material ancestors.What we will do in the lab here at the Akademie is at first a Capture of images of Kiran`s gross body in yogasana/postures, dance and martial movements. Using photo, video and motion- capturing technologies, this will be a capture of the gross or bio-material dimension.After this digital capture of the gross body, rather than pictorial visualisation, we will move towards a pixelated visualisation in which the image is structured/choreographed using principles of the subtle body: sensory modulations such as touch of the skin and blink of the eye, using rhythms of breathing, using dream-images and visualisations.

Ultimately our creation of P3B will be a visualisation of a subtle body derived from data of a gross body. This data is primarily embedded in the living archives of embodied practices. But the troubling thing is: data, in contrast to evidence, is not something given. Rather it has to be extracted from evidence. In contrast to evidence, which can be binary: true or false, data is rhetorical; it is something constructed to create certain effects, therefore it has no truth value. Data is just the result of curating an archive. In this process of curation, data is transformed by abstraction, aggregation etc. Data eventually will be that what Kiran and I take into account. So we will struggle with questions that are deeply connected to this very notion of data and its visualization. Questions like: What kind of data counts as knowledge? Is it all about counting? What counts? Who counts? Can we blend counting and understanding in a meaningful way?

Yet even as we invoke such a dialogue of the philosophical with the technological, we are careful towards the Californian talks and promises around disruption through technology. Eventually this Californian perspective claims thatconsciousness could exists independent of embodiment. And that one, for example, could upload the consciousness into a cloud and that it would exist there independently from the body from which it emerged.

We want to intervene into this perspective of a loss of embodiment, which is also symptomatic for the western discourse of posthumanism; We want to ask instead, how the embodied practice of theatre and dance can be expanded into a practice that thinks with and through data? This also connects to Kai`s research interesest into the field of dramaturgy. Which Kai understands as a practice that is constantly expanding the possibilities of theatre.

The following is an excerpt of the film LOSERS AND WINNERS by Ulrike Franke and Michael Loeken, which documents the dismantling of the Kaiserstuhl coke factory here in Dortmund and its shipment to and reassembly in China in 2005:

We are introducing this excerpt here because it hints to a second biographical layer of our project. Because Kai was born and raised in this city and grew up in a workersfamily. Some of his relatives even were associated with the depicted coke factory- while he has moved to Beijing in 2016 where he developed a BA program in dramaturgy. Growing up here, he could witness the effects of the end of industrial culture on the local peoples minds: a loss of belief in any valuable future prospects, a rise of violence and xenophobia were very often the consequence of it. To some extent, we regard the city Dortmund as a symptom for a broader “western condition”, which we characterize through the three buzzwords: loss of identity, decline of western empire, transformation difficulties.

Seen from this perspective, Dortmund`s embracing of the digital (academy, port, university) points to questions around the sociological foundations of the digital: How can the digital be an opportunity for reinventing one´s identity as a city- or (in the case of nowadays Chinas) even as a nation? In our project, we are looking for ways to interact with the very concrete city that hosts us. We are planning interviews between P3B and local citizens of Dortmund, as well as with the global networks of media scholars and philosophers of science. The data collected through these interviews with P3B will then form the textual basis for the letter to the dead doctor.

Like we said in the beginning, we want to use this presentation as a performative-documentary gesture to write out our research questions in this wiki. And basically we have now oriented our research around the following 5 questions:

What kind of knowledge can data produce? Is it all about counting? What counts? Who counts? Can we blend counting and understanding in a meaningful way?

How can we possibly bring P3B from his 2 dimensional state into the 3rd dimension?

What Indic knowledges of the human body-mind did the doctor have to disinherit? And could these knowledgesbecome accessible through surviving practices of temple dance and yoga?

How can the embodied practice of theatre be expanded into a practice that thinks with and through data?

What does it mean when the digital becomes a medium for reinventing identity as an individual, as a city, or even as a nation?

[Rehearsal Notes: 19  September - 1 October, 2021]

In order to prepare the visit of our programmer Matthias Härtig, whose expertise will help us in creating P3B, we started discussing the question of how to select from the living archives of Yoga, Kalaripayatt and Odissi-dance. We want our methodology to be transparent and consequential, which is even more important when one considers, that in contrast to Yoga and Kalari, Indian dance was completely reconstructed during the turn of the 20th century. Eventually, we decided just to focus on the pedagogies of Yoga, Kalari and Odissi- as they were taught to and are practiced by Kiran. In case of dance, we will draw, among other things, on Kiran`s training units. The following video displays such a unit that Kiran actually recorded for his master Bijayna Sathbathy during the time of the Pandemic (when personal training was not possible).

Kiran posesses several documents from his late grandfather: 5 pages entitled “Dates to appreciate and to remember”, 16 pages entitled “List of Patients”, several unnumbered pages and several wordless pages, and eventually the draft for an autobiography (pages 3-115). On September 30th we started to decipher, transcribe and read this autobiography, that we eventually put up on the stage of our lab.

[Rehearsal Notes: 10 October - 17 October, 2021]

These videos show Matthias Härtig and Kiran Kumar during the Motion Capturing:

But it's only really digital when there are glitches:

In this interview our programmer Matthias Härtig talks about the historical and technical particularities of the software “Kalypso”, that he is using to process the data from Motion Capturing:

Photographs of the performance presented at Academy for Theatre and Digitality, January 21st, 2022

Essay: Dear Dead Doctor's Motion Capture and Eadweard Muybridge's studies on Motion Pictures

“Time was at his command as it had never been at anyone’s before. A new world had opened up for science, for art, for entertainment, for consciousness, and an old world had retreated farther…The modern world, the world we live in, began then, and Muybridge helped launch it.” (Solnit)

When we started the infrared recordings of Kiran, we were fascinated right at the beginning by how much the recordings resemble the optics of Eadweard Muybridge`s studies on motion.

Muybridge's work on the technology of moving images is at the beginning of artistic research as well as a new relationship to space and time. As an artistic research project that uses the digital to question modern understandings of space and time, it made us curious to think about Dear Dead Doctor in relation to Muybridge.

The 1870s revolutionized not only industry but also human perception. It was in this decade that the newly invented telephone and phonograph joined photography and railroads as instruments for erasing time and space. It is also this decade, that Muybridge begins his motion studies that will ultimately form the precursor to cinema. He is commissioned by the Californian ex-governor and railway entrepreneur Leland Stanford to find out whether a trotting horse has all four feet off the ground at a time (which it has!). This collaboration between the then already famous photographer Muybridge and the entrepreneur/politician Leland Stanford, takes place in Palo Alto. It is Leland who, in memory of his late son, will endow today's Stanford University, which has arguably become the academic incubator that has contributed significantly to the emergence of today´s Silicon Valley. The Muybridge/Stanford collaboration prefigures some of our current understanding of artistic research. In fact, photography in the 1870s was really a craft, because each photographer had to develop his own optical apparatuses and chemicals to be able to make images at all. Muybridge had become famous for his outstanding camera shutters. To solve Stanford's little scientific question about horse movement, Muybridge constructed a special track, with wires on the floor that the horses touched and thus triggered photographic recordings of their movements. At the end of the 1870s, Muybridge developed the technology of the zoopraxiscope that showed his still images in rapid movement. A precursor to today´s cinema.

Muybridge's study of Stanford's horse (which actually was called “Occident”) showed that the feet of the trotting horse do indeed leave the ground, but only when they are under its body, and not, like it has been always depicted throughout the history of painting, when they are stretched away from the body. Muybridge's Moving Image technology enabled the human eye to see what had never been seen before. Time stood still. Walter Benjamin will later speak in this context of the “optically unconscious”, which is made visible by the camera. And it is Eduardo Cadava who eventually showed how much Benjamin` s thinking of history is deeply endowed to the technological apparatus of the camera. This is especially evident in his theses on the philosophy of history, in which he speaks of the “standstill of history”, or of a past that “flashes” up. One could even try to extrapolate from here, that some significant parts of our modern discourse around history, would have not been possible without the technology of the photography and the conceptual metaphors it provided.

Muybridge has captured and recorded movements by photographing their sequence and stringing together 24 images of this sequence per second. We in turn capture motion by using 26,000 data points or pixels. We are interested in the changes that happen within a motion sequence. For this we understand the data point not as a representation of a positional state, but as the expression of a difference between positional states, which results from a movement. Thus, our data points are the memory of a motion. Again, this memory can only be expressed by specifying certain processing parameters, such as whether and how long a pixel should remain in existence after a movement has been performed. By setting these parameters, it is possible to visualize previously made movements, for example. This possibility represents, in analogy to Muybridge, a new form of making visible what was previously optically unconscious.

In our aesthetic approach, we short-circuit this form of visualization with the Vedic idea of a subtle body. For the possibility of making the absent visible refers to the immaterial reality of the subtle body. We use digitality here to counter the modern understanding of history generated by photography with a different experience of temporality.

Essay: Contaminatedly Yours: Genre, Information, Body by Nishant Shah - Reflections on Dear Dead Doctor

One of the keenest promise and biggest myth of digital technologies is cleanliness. There is a continued replaying of the digital systems as clean, reliable, and free from unwanted contamination.

• In e-governance it takes up the SMART (Simple Moral Accountable Responsible Transparent) rubrics, insisting that digital governance can be cleaned of the excesses and corruptions of analogue bureaucracies.

• In computational design, it favours storage over memory, presenting stored data as clean and sorted, able to be retrieved and remembered by querying algorithms, for an infinitely digital future.

• In probability driven computational networks where the human is often sought to be replaced by computers undertaking mechanical transactions and calculations that perform efficiency and scale, this is often illustrated by showing human circulation of information as constantly mutating and changing whereas computational circulation remains discrete and contained by the protocols within which it is established.

The affirmation of cleanliness is both an exercise of control and a black-boxing of technologies, despite the fact that we continually see how computation technologies are ontologically and manifestly produced through multiple layers of contamination. A cursory look at algorithmic governance practices opens up a field of intentionality, bias, encoded discrimination, and amplified filtering that leads to the production of harm and violence without accountability and restitution. The obsolescence of databases, the leap-frogging of technologies, and the continued breaches and leaks of data and information belie the idea of immortal data and indeed presents data and information infrastructure as fragile and prone to breakdown and manipulation. Especially in the world of self-learning algorithms and networks of correlation, we continually see our reliance on unexpected, undesigned, and unplanned for variable queering models, producing states of exception, and leading to designed deviance which can neither be planned nor controlled.

Cleanliness then is neither an attribute nor a condition of digital networks and their spaces. The foregrounding of cleanliness has to be seen as an attempt to clean bodies, informations, and approaches that threaten the power, destablise the status quo, and resist the benign narrative of computation that is being naturalised in our everyday practices of digitalization. Cleanliness has to be recognized as an active way by which resistant data and technology usage can be controlled, punished, and penalised in order for dominant narratives to be favoured.

This insistence on cleanliness and controllability of digital technologies also creates the dominant aesthetic of our times that reinforces this filtered, curated, cleaned digitality as the de facto mode of visualising and engaging with the digital. In Dear Dead Doctor, we eschew this promise of cleanliness, and even when working with these technologies of cleanliness, continually look at different conditions of contamination that the digital technologies have to offer. From parthogenetic births out of dirt to the diffuse and distributed data, we examine ways by which the aesthetics of digital contamination can be constructed by looking at the logic, logistics, mechanics and intentions of networked computation on three different sites.

There has been a recent autobiographical turn in the performing arts, where the attempt has been to centre the lived experiences and histories of the individual as data for new performances and interventions. The autobiographical turn seems to be human centric but perhaps mimics the ‘deep mining’ and ‘data harvesting’ possibilities that new digital technologies of ‘more than’ and ‘smaller than’ the human seem to offer. The autobiography becomes a way by which lived and affective experiences can be re-narrativised to become a spectacle which can then populate the ever expanding data sets of digital storage. Autobiography as a genre also lends itself to verisimilitude and historical reconstruction supported by archives and evidences that can help clean up the realm of the personal for better intelligibility and taxonomy.

However, the space of theatre and performance is not to deploy these digital aesthetics and queries in order to produce spectacles of machine logic. The challenge of theatre is to create new genres and modes of autobiography that can destabilise this individual turn by exploiting other potentials and possibilities of the digital.

In this performance, we are thus moving towards an biomythography, where through the tropes of autobiography, we produce new myths which are part fiction, part lived-reality, part fantasy, and part politics. The idea of biomythography is not a resistance to digitality but in fact a joyful appropriation of its cleanliness rhetoric and finding new modes by which the post-digital can be performed.

Instead of thus, relying on the archive as discrete, the memory as contained, the record as sacrosanct, and the evidence as the only way by which a story can be told, automythogaphy creates a new terrain of post-digital performance where we lean into our pasts as contaminated and queer, open to interpretation and intervention even as we use the elements of autobiography making. In this project, we are perhaps looking at how the digital affordances can help move from autobiography to biomythography:

Positioning the 'Origin Story' as a Collective rather than an Individual one

It is at the heart of digital computation that the nodes do not have a linear, comprehensive, origin story where it pre-exists the network and intention of information circulation. Nodes in a network are products of intention – and each node is ontologically replaceable by any other node in the network. This idea of nodes as universally interchangeable offers us a new entry point into the personal history: That the personal history is both an imagined one, and a collective one. Autobiographies, as Jeanette Winterson had once said, are stories about other people, while pretending that they are stories of the self. What would it mean to have this digital possibility of telling the story of multiple people – entire generations and communities, instead of focusing on the self.

If autobiography is about querying the self, biomythography has to be using the self as a query that tells the collective stories of the people, things, events, and memories that both precede and escape the formulation of the ‘I’.

One of the imperatives of the conventional autobiography has always been the act of reconciliation. Whether it be in epistolary or confessional forms, the autobiographical subject performs the stitching of fragments and actively filters things that cannot be synthesized into the discrete subjectivity.

Reconciliation is a performance towards an external authority that requires for the production of this discrete subjectivity so that it can be isolated, tracked, managed, and controlled in its unfolding. Within digital networks, there is, however, a possibility to exploit the mechanics of ‘hyperlinking’ which do not need to perform causality or synthesis between two disparate objects within a network. The hyperlink is a freedom from the tyranny of the narrative, and allows for new modes of co-existence where information can be stacked at the level of the index rather than at the semantic level of the lexicon.

Using this as a trigger point, we examine the potentials of mythobiography as straddling paradoxes without the burden of reconstitution:


The possibility of mythobiography is in letting go of the aesthetics of representation and leaning into the world of simulation, where there is no ‘external reference’ because it is a self-contained loop of small world phenomena. We are looking at mythobiography as an act of interpretation with tainted factuality – not a narrative of proof but a simulation of stake, and thus offer the space for wilful misreading.

If the autobiography is a space for narrative, then this digital mythobiography is a space for metaphors, where the ‘auto’ – the self, which is a representational and realist construction, is replaced by the ‘mytho’- a simulated and an imagined programmed entity, that allows us to explode the probability horizon through potentiality functions.

The autobiographical turn in digitalisation has often been celebrated because of the multiplicity of sources through which the personal narrative can be written. In an age of internet of things, where extreme surveillance is our de facto mode of existence, the presence of multiple recording devices from phones to drones, allows for a multi-faceted narrating of the subject through different sources. The montage of simultaneous streams that each offers a different vantage point and perspective has often been presented as the exploding point of the individual centred narrative.

However, in all of these processes of polyvocality, the autobiography stays compliant to the demands of fidelity. Even though we are already engaging in critical conversations around fakeness and reality, with fake news, misinformation, and deep fake becoming a part of our regular vocabulary, the question of fidelity – having that external referent which will make the distinction between real and fake, or clean and contaminated information continues to hold command.

Kiraṇ Kumār

Kai Tuchmann

The collaborative research between dancer Kiraṇ Kumār and dramaturg Kai Tuchmann is towards the making of 'Dear Dead Doctor', a work of digital and documentary theatre. In the form of an open letter to Kiran’s late grandfather, who was the first doctor of western medicine in his family, the project explores performative strategies of bringing medical and somatic knowledges into dialogue on stage. This address to the doctor is conceived through Kiraṇ's grandchild: a parthenogenetic, proto-progenic being (P3B). P3B will be digitally materialized by drawing on 'data points' from yogic, dance and martial embodied practices. 'Dear Dead Doctor' re-imagines what 'being human in the world' could imply in its continuity across pre-modernities and futurities.

Dear Dead Doctor

Motion Capture/Tracking

Dance

Data Visualisation

Documentary Theatre

2021

Christiane is working on a framework of how to prototype and evaluate social protocols and society building methods in play and theatre context with (inter)active audiences. Markus is curious about playing around/hacking different technologies. A shared interest that we figured out is smartwatches as social devices.

For interesting group interactions with smartwatches as devices (that do not require mobile phones), it would be interesting to have them interconnected, being able to sense (the distance and more) of each other, to measure the carrier's body-information (heart-rate, movement-pattern) and to be able to function as a communication device (with each other e.g. to push and receive game elements or to receive global push notifications).

Here are the specifications of the hardware we need:

how to organize subgroups (tribes) of an instant society in a way that their actions have meaningful impact and can be evaluated afterwards

how to prototype new kinds of social protocols with intuitive instructions/means

how to model a kind of “social sense” (to feel closeness and distance, to feel attention and aversion)

how to model abstract resources (as time) in an intuitive way

how to make the interface as intuitive as possible

build a network where the watches communicate with one another

build a server which saves game state and communicates with the smartwatches

2 x bangle.js *All the Espruinos!* edition

(Since some months we are watching the following watch  makers 

. Unfortunately through the outbreak of Corona the supply chain of the makers is interrupted, so we do not know if we can receive the devices (that were promised not before March 2020) in time. We started looking for alternatives now.)

a simple notebook which functions as a server

bluetooth hubs and maybe wiFi access points

Christiane Hütter

Markus Schubert

We like to see technology as possible space, or making possible space, for participatory processes. During this fellowship, we will use theatre as a living space for encounters, with the research questions: How can we construct society together? And how can we make this process visible for audiences? 
We want to develop a sustainable format for collaborative discourse, in which people can together experiment with different forms of organisation for society. This could start on a small scale, with a family, and can become quite big: How to organise a country?

******

Performance

Networked Objects

Sensors

2020

Besides the aesthetic implementation, we are interested in two main elements:

1. How can non-verbal communication between space (built from sound and scenography) and spectators imply a plot line or narrative?

2. How can the environment's own characteristics play a role, so that they can allow conclusions to be drawn about motivations and correlations? 

Jakob Lorenz

Stella Lennert

'PALIMPSEST' engages with virtual space as an interactive stage, and explores narration through interaction instead of representation. For this, the project takes theatrical literature as an anchor and combines it with elements of gaming. The dramatic texts act as a common ground, a guideline through unfamiliar virtual environments that follow strange rules. PALIMPSEST is experienced via VR goggles and headphones. 

PALIMPSEST

Drama

Stage Design

2022

Humane Methods (2019-present) is a project consisting of happenings and stage productions where dancetheatre, bio-art, interactive music, adaptive lighting, and AI technology integrate into living ecosystems. Each work in the series acts as a branch of a rhizome: each dives into different facets of the same theme and world. The works created so far – 

 (2020) and the forthcoming ΣXHALE (2021-) – combine in different ways performing arts with technology and the human with the non-human to reveal algorithmic violence through sensory, physical and ahuman strategies. It is an aesthetic research into how a synthesis of symbols, movement, music and AI can enable posthuman forms of empathy.

℧R is a solo performance for a body, AI algorithms, lights and the vibrational force of sound. 

Embodied in Donnarumma’s musical instrument XTH Sense, an interactive, machine learning algorithm transforms the inner sound of muscles, blood, and bones into a real-time choreography of movement, sounds and light. Drawing on Donnarumma’s more recent movement research, the piece combines multi-sensorial stimulation with human-computer interaction and somatic practices to test the seeming impossibilities of bodily articulation and repetition. 

℧R is a walk in the dark. A thin body walks while a dim light shines. A path covered in soil is between them. Gazeless, the figure walks and prays, walks and prays, walks and, entranced, pries into the darkness. Its red cloth sanctifies it. Its missing face alienates it. The walk is a loop, a prayer, an aggression, an annihilation and a transformation. An AI computer vision system watches over the body, attempting to distinguish the body from the ‘noise’ surrounding it. The AI wants to represents what it sees, but the system works against itself in a paradoxical loop; the more the algorithm learns to see, the brighter the light becomes. In the eyes of the algorithm, the body is eaten by the storm of light.

The latest chapter in the Humane Methods series is ΣXHALE (2021-). A poetic reflection on ethics in algorithmic societies, ΣXHALE is a procedural and participatory piece lasting several days.

It develops in loops, performed simultaneously by 6 performers* and an AI computer-vision system. As it watches the performers repeat a proto-prayer ritual, the AI learns their movements in the space and then begins searching for anomalies, something that deviates from what it expects to see. The learning process is accurately transformed in real time in music and lighting, influencing thus the events in the form of audiovisual stimuli. With each repetition, each variation and each anomaly, the loops mutate into stories of violence and empathy and the audience is asked to choose how they want to participate in these stories. The interaction between audience, performers and AI gives life to an endless sequence of micro-narratives.

AI in theatre and performance: Strategy & Questions

Technical Infrastructure

The Autoencoder

A Performing Machine

How To: Technological Infrastructure

Andrea Familari

Marco Donnarumma

Humane Methods (2019-present) is an ongoing project by Fronte Vacuo (Marco Donnarumma, Margherita Pevere, Andrea Familari) that reflects on the new forms of violence emerging in algorithmic societies. The technological research for ℧R and ΣXHALE have been conducted at the Academy for Theater and Digitality and this page describes our research questions, workflow and findings in relation to these projects. Please note that this page does not discuss the theatre pieces we created in their entirety, but focuses only on the technological research that underpins those pieces. For further information about the aesthetic, conceptual and dramaturgical research behind these works please visit our website.

Humane Methods

Lighting Design

The human being is not generally thought of as a staple food source.
Observing the anatomy and skeleton, one can see that the animal is neither built nor bred for its meat, and as such will not provide nearly as much flesh as a pig or cow.
The large central pelvis and broad shoulder blades also interfere with achieving perfect cuts.
There are advantages to this however, especially due to the fact that the typical specimen will weigh between 100-200 pounds, easily manipulated by one person with sufficient leverage. 

'Butchering the Human Carcass for Human Consumption'

In March 2001, 39yo German IT professional FRANKY and 43yo German engineer CATOR made an appointment on the internet. After meeting at Kassel main station for the first time in “reality” after their extensive online communication, FRANKY drove the two men to his house, where he cut off CATOR’s penis - which proved too chewy for the both of them to eat -, proceeded to kill him with a stab to the throat and then stored the pieces of meat from CATOR’s cut-up body in the freezer. Both had previously agreed to this, for while FRANKY longed to fully internalize another human being by eating it, CATOR wished to be fully disintegrated and disappear. The two men recorded parts of their meeting, wherein they were constantly forced to synchronize their quite different imaginations of the encounter with the reality of their shared physical and social situation.

15 years later, Taiwanese electronics producer HTC cooperated with US company Valve for launching a virtual reality headset they had christened “Vive”. Together with similar products, it marked the beginning of a slow mainstreaming of VR applications - a technology long envisioned with a moniker first coined by French playwright Antonin Artaud, the proponent of a “theatre of cruelty”. The “Vive” headset enabled its users to move in a small space within a computer-generated surrounding and interact with it. VR experiences proved to be highly immersive - and for many people at first physically quite disorienting, because they entailed being present in two realities at the same time. The headset was originally meant to be called “reVive” - but the name was changed before it hit the market; the notion of a new wondrous device transcending death or mortality thus obscured.

Subject Matter: Interviews, Autonarrations, Artistic Production

“Virtually incomprehensible - but he was happy, sitting there in his own blood.” In an interview, FRANKY thus comments on the sight of castrated CASTOR sitting in his bath tub. The desire for understanding, imaginations of fulfilment and redemption and their conflict with reality are recurring themes of the Rotenburg cannibal case. “Whatever you imagine, in reality it is basically always different,” says FRANKY later in the same interview, formulating a core experience of artistic work - only to dismiss it at once, chastened by his own experience: “Why should I do something, knowing it will never be like I imagined?”

Interviews and autonarrations by FRANKY form part of this project’s material as well as protocols of chat conversations by the two involved. Research also covers cases of other confirmed sexual cannibals (Jeffrey Dahmer, Issei Sagawa, Arthur Shawcross, Albert Fish…) as well as an extensive collection of images, videos and writings by people from the internet community of cannibalism fetishists - i.e. the artistic production of the scene itself. Here, not the shock value - the 

 - is of interest, but rather the genre’s recurrent figurations and formal rules. What are the conventions for depicting the not only unconventional, but outright taboo? An additional testament to the fundamental discrepancy between imagination and reality are documents of the thousands-of-years-long history of insinuations and mise-en-scènes of cannibalistic desire, which 

: From Saturn’s cruelty and St. Erasmus’ suffering, collective colonialist projections on the “other” and the monistic ecstasy of Kleist’s Penthesilea to Hannibal Lecter’s unsettling pop appeal. These may also hint at the function of said discrepancy.

The question remains: Why imagine something, knowing I will never fully understand it?

Working with “cases” that are in one way or the other “radical” or “extreme” is always in danger of giving in to sensationalism in representation and deprecation, belittling or denial in reception. Not surprisingly, the cannibalism case of 2001 has already undergone wholesale journalistic and artistic processing - not least, it inspired numerous films and theatre plays. Its spectacular value has been cannibalised for good. Unlike real flesh though, memories and fantasies are not destroyed through consumption. Today, rather will they be externalized digitally and reproduced forever.

The subject matter of sexual cannibalism is especially suited for performative VR application because it reformulates fundamental problems of the medium itself: the experience of being physically absent and present simultaneously, the relationship of a technologized world (both men came from technical professions) to the “flesh” we inhabit this world with, not least the desire for encounter and singular experience that is not only the basis of 

 performance and of the arts’ hunger for transgression, but for so many of today’s practices and services. VR as a medium - and XR/MR as its follow-up - performs such transgression by first and foremost creating the experience of being alive and present with one body in two distinct spaces at once - the “virtual” and the “real”. It is a lucid challenge for our consciousness, comparable to the first cinematic experiments - and a challenge for theatre, too.

The fellowship time will be used for research and technical try-outs regarding the setting, 3D visualization und interaction of the planned piece. Self-education on Unity 3D's shader graph, re-creating the physical performance space in VR & avatar creation and live control will also be part of the work. In the unlikely case of a global pandemic interfering with schedules, efforts will be made to boil down plans in a way so as to still achieve a presentable outline of the project involving VR elements.

Development will focus on cable connected higher end consumer HMDs, specifically on HTC Vive / Steam VR. Besides a Vive/Vive Pro headset with base stations and controllers, at least two Vive trackers will be part of the setup (more if used for performer body mocap). Wireless microphones for performer(s) and a computer being able to run the VR elements are also necessary.

The piece will reproduce a model of the actual physical performance space in VR, so knowledge of it and development time to produce the appropriate design beforehand as well as enough setup time to ensure correct alignment of virtual and real space are necessary.

As of now development aims for a ca. 30 minute 1-on-1 performance alternating between real and virtual space in the course of its narration.

Arne Vogelgesang

A guided, interactive VR performance which explores transgression, sensationalism and spectacle through the subject matter of sexual cannibalism. This topic is especially suited for performative VR application because it reformulates fundamental problems of the medium itself: the experience of being physically absent and present simultaneously, the relationship of a technologized world to the "flesh" we inhabit this world with, and the desire for encounter and singular experience that is the basis of one-to-one performance and of the arts' hunger for transgression. It is a lucid challenge for our consciousness, comparable to the first cinematic experiments - and a challenge for theatre, too. 

New Flesh

Avatars

, is an artistic/methodological movement research. 

The aim of this research project is to take core principles of (analogue) puppetry animation and to translate them on the moving/dancing human body using motion capture techniques, in order to explore how this movement method can increase the experience of aliveness of 3D avatars, by making their movements more organic and establishing them as breathing/alive creatures.

I am inspired by, and draw upon my background as pluridisciplinary artist, with a focus on dance, movement, and puppetry/object theater. 

I'm interested in approaching 3D avatars as another form of body language with its specific dynamic and needs, which also implies other possibilities and limitations. For this language, the human body acts as the vector as well as a translator. The fundamental principles of puppetry animation are considerably different from the human way of moving regarding impulses, dynamics, rhythm, accentuation, tension, stop/echoes of the movements, etc. When these principles of puppetry are applied to the human body, and the resulting movements are used to animate an avatar, an act of translation occurs. It results in more precise movements and sequences, and the impression of ‘organic’ movement, because the avatars are established as ‘breathing’ and ‘alive’, which is the core of puppetry’s art.

You will find here the long interview about the Research and the Process itself:

{ More to come here, about the Process, the Whys, the Discovery }

As a showing from this research, ‘A_live, became a video installation on 3 canvases about those movements, impulses and breath, accompanied by live music.  

The setting is composed as follows: Movement sequences, recorded using motion capture techniques, are projected on the three canvases. The embodiment of this exploration is interpreted by three different avatars, each one with its own texture and characteristics. One avatar has a water texture, which has an ‘explosive’ capacity related to the strength and speed of the movement impulses. The second avatar is more ‘typical’: a humanoid shaped form, yet connected to a cloth which is reacting intricately and precisely to the avatar’s impulses, creating a ‘wind’ feeling as well as an extension of the breath’s embodiment. The third is made of particles which leave traces of light following each movement.

The emphasis of this visual experience is on the quality and precision of the movements, and the different impulses, explorations and nuances embodied by the avatars: How the different textures are reacting to those elaborated momentums, innovative way of moving and to the avatars breath are the leaders of this experience. Each texture with its respective features reacts in a different way to the impulses of the performer, creating various and specific visual echoes in harmony with one another. 

Thoughtful dramaturgical choices and precise editing creates an additional layer of choreography, from the timing of the appearance (and disappearance) of the sequences upon the canvases; the smoothness (and suddenness) of the transitions; the synchronicities and delays between the sequences which underline the specificity of the each movement or breath. We are invited to witness each avatar’s individual breath as well as experiencing a collective organic entity as it moves and breathes as one.

The animation’s cameras are connected to the pelvis of the avatars, which is to say, to the core of the movements. In fact, it is important to note that absolutely no extra effects were added, and there was no need at the end to clean the files. The cleanliness of the movements, and the possibility overall to use ‘raw’ files in this way, are the direct results of the method I’ve explored and developed. 

Although this showing is the first creative response to my research; the frame became not only an explicative presentation of the project so far, but an immersive installation in its own right. It was important to me to create a unified, elaborate setting, and to invite the audience to feel immersed in an atmospheric ‘island’: connecting with senses, finesse of movements, beauty, breath and ultimately a deep exploration of aliveness through external bodies.

                        If you wish to discover more about the installation itself, I invite you to consult the project description on my website. *

Concept, Movement Research, Embodiment: 


Baris Pekcagliyan / @0bapek
Beka Tcharbadze / @bekaarba
Alptuğ Çavuş / @talkingrocks

Warm thanks to the Academy for Theatre and Digitality, 
Mario Simon,
& KOPRODUKTIONSLABOR.

Laura Waltz

My fellowship project 'A_live' is an artistic/methodological movement research, and the use of MoCap technology to explore the possible 'organicness' and feeling of 'aliveness' of avatars through animation fundamentals from puppetry.


The aim of this research is to take core principles of (analogue) puppetry animation and to translate them on the moving/dancing human body using motion capture techniques, in order to explore how this movement method can increase the experience of aliveness of 3D avatars, by making their movements more organic and establishing them as breathing/alive creatures.

A_live

2023

Puppetry

How can we (as a field) make a dance in a virtual environment so that the audience “feels” the presence of the digital body?

What is needed for the audience to find new aesthetics when dance is mediated with digital environments like VR headsets?

, which explores the performative possibilities of VR/XR technologies through dance. It questions the concept of a 'stage' when dance occurs simultaneously upon hybrid stages (for instance, in both virtual and physical realities) and asks: how can this kind of content be shared with an audience?

 Virtual reality creates an experience where introspection can be achieved. This opens up possibilities for somatic and other movement practices. Working with the body and technology presents the opportunity to explore rigid computer systems through an ontological approach (praxis/doing), and to explore the medium's own possibilities. Often, dancers who work with technology forget their own embodied language (the 'why/how') and get confused and carried away by the pragmatic reality of technology, focusing instead on the content (the 'what'). They forget their own embodied power. The result is a superfluous object.

To tackle this I propose a praxis which privileges the process over the result. I work with avatars and sigils as a ritual practice with a non-anthropocentric approach, and the key aspect is to shift the focus in choreographic strategies. Instead of asking "what to 

This project continues my research into dance in VR that started in 2022 at 

Motion Bank - University of Applied sciences Mainz

 (Project originally founded by choreographer William Forsythe). 

Using a markerless motion capture system I created, I explored and documented simple visual representations of the body (avatars) in VR. 

. I achieved this using a non-anthropocentric approach. In the documentation it was important to use an inclusive, clear language that addressed both dancers and technologists. In the 1st cycle at Motion Bank, the focus was on the dancer.

In the 2nd cycle at the Academy I want to add a layer where I introduce the stage, thus exploring the relation between audience and dancer.

The natural empathic-embodied relation between dancer - audience is partially lost when using VR headsets. This research aims to help the audience to find new aesthetics, new engagement, new meanings and purpose when dance is mediated with digital environments. Through two live performances, I created a visceral sense of unfolding or revelation, but also a feeling of immediacy and loss. 


RESEARCH PLATFORM

Academy for Theatre and Digitality

Hubs Theatre Tools

Nils Corte Roman Senkl

Motion Capture/Tracking Avatars VR (Virtual Reality)2020
completedFellowship

Summary
Hubs Theatre Tools are currently developed by Nils Corte (Coding) and Roman Senkl. Their purpose is to expand Mozilla Hubs with an easy to use motion capturing and playback tool for hubs avatars.

Considerations

Similar Projects

Hubs Theatre Tools

Nils CorteRoman Senkl

Motion Capture/TrackingAvatarsVR (Virtual Reality)2020completedFellowship

SummaryHubs Theatre Tools are currently developed by Nils Corte (Coding) and Roman Senkl. Their purpose is to expand Mozilla Hubs with an easy to use motion capturing and playback tool for hubs avatars.

Considerations

Similar Projects

Nils Corte Roman Senkl

Motion Capture/Tracking Avatars VR (Virtual Reality)2020
completedFellowship

Summary
Hubs Theatre Tools are currently developed by Nils Corte (Coding) and Roman Senkl. Their purpose is to expand Mozilla Hubs with an easy to use motion capturing and playback tool for hubs avatars.