is the knowledge-sharing platform for the 

. Explore the experiments and bold ideas behind the Academy's Fellowship programme, delve into the science behind the projects of our HIDA researchers, and explore the results of international collaborations.

is a platform for discovery and documentation. Each 

showcases a unique artistic-technological research project, with topics as diverse as the use of AI in theatrical design, choreography for and with robotics, and the possibilities for theatre in virtual spaces. Each 

 offers the chance to learn more about the members of our artistic research network. By pooling our knowledge in such a way, creative research into theatre, performance and digital technologies can become more accessible, sustainable, and artistically innovative.

Funded by the Federal Cultural Foundation, the Academy will award a total of 54 fellowships between 2019 and 2022, each lasting for five months. The 

 are designed for people interested in artistic-technical research which explores the possibilities for digital narratives in theatrical spaces, as well as their practical implementation. Fellows have researched a variety of digital methods (motion capture, coding, XR, VR, AR, sensor technologies, robotics, and more) across artistic fields including opera, dance, orchestra and drama. 

Helmholtz Information & Data Science Academy

(HIDA), the Academy for Theatre and Digitality has offered joint research fellowships at the interface of digital art and cutting edge scientific research since 2021. The HIDA fellowships are kindly supported by the 

PlayOn! New Storytelling With Immersive Technologies

 is a larger scale cooperation project, funded by Creative Europe, designed to understand, learn, test and apply immersive technology to theatre, in order to shift classical storytelling to interactive narrative formats.

Artificial Intelligence

VR (Virtual Reality)

Interactive

Sound Design

Projection/Mapping

3D Printing

XR (Extended Reality)

AV (Audiovisual Design)

Theater Dortmund
Theater Karree 1-3
44137 Dortmund
Germany
Fon: +49 (0) 231 226 14330

Address and location of the Academy for Theater and Digitality:
Speicherstrasse 17
44147 Dortmund
Germany

E-Mail: akademie@theater.digital
Web: theater.digital

Tax number: 314/5702/9367
Sales tax identification number: DE 124643894

Management Team:

Executive Director
Tobias Ehinger

Administrative Director
Martin Lizan

Director of the Opera
Heribert Germeshausen

Director of the Ballet
Xin Peng Wang

General Music Director
Gabriel Feltz

Artistic Director of the Schauspiel
Julia Wissert

Director of the Children's and Youth Theater
Andreas Gruhn

Director of the Academy for Theatre and Digitality
Marcus Lobbes

Please also note our privacy policy.

 as the sixth division of Theater Dortmund (hereinafter referred to as "Theater", information about Theater Dortmund can be found 

), is pleased that you are visiting our website. Data protection and data security when using our website are very important to us. We would therefore like to inform you at this point, in accordance with Article 13 of the EU General Data Protection Regulation (DSGVO) and Section 13 of the German Telemedia Act (TMG), which of your personal data we collect when you visit our website and for what purposes it is used. Since changes in the law or changes in our internal company processes may make it necessary to adapt this data protection declaration, we ask you to read through this data protection declaration regularly. The data protection declaration can be accessed, saved and printed out here at any time.


§ 1 Scope of application

This data protection declaration applies to the Internet offering of Theater Dortmund, which can be accessed under the domain www.theaterdo.de and the various subdomains (hereinafter referred to as "our website").


§ 2 What is personal data?

Personal data is individual information about personal or factual circumstances of a specific or identifiable natural person (data subject). This includes, for example, information such as your name, address, telephone number, date of birth or e-mail address. Information for which we cannot establish a reference to your person, e.g. by anonymizing the information, is not personal data.


§ 3 What personal data do we collect/process and use?

Access data without personal reference is stored, such as the name of your Internet service provider, the page from which you visit us, the names of the requested files and their retrieval date. These data are evaluated exclusively for the improvement of our offer and do not allow any conclusions to be drawn about your person. 

If you wish to make use of services offered by us on our website, such as purchasing tickets via the ticket store or ordering a newsletter, etc., it is necessary for you to provide further data. This is the data that is required for the respective processing, such as your payment information when purchasing tickets or your e-mail address when ordering a newsletter. 

We process the data we collect about you for the provision of the services we offer, to answer your questions and to operate and improve our website and applications. 

We process your personal data exclusively to provide you with a comprehensive service via our website or, if you use our contact form or, to process your ticket purchase or, to process your application or, to provide you with requested information about our offers.

Fellowship

HIDA

PlayOn!

discover cutting-edge digital research in theatre and performance 
meet our research fellows and collaborators in the 

Portal: Academy for Theatre and Digitality

17 September, 2021: Transcript of presentation at PlayOn! conference held at the Akademie für Theater und Digitalität, Dortmund

When we began our fellowship at the Academy 10 days ago, we rearticulated our project for this public wiki. Basically the next 10 minutes will be a sort of performative-documentary gesture of ultimately writing out our research questions in this wiki after we have taken you through our thought processes, starting by sharing a brief project synopsis.

“The collaborative research between dancer Kiraṇ Kumār and dramaturg Kai Tuchmann is towards the making of 'Dear Dead Doctor', a work of digital and documentary theatre. In the form of an open letter to Kiran’s late grandfather, who was the first doctor of western medicine in his family, the project explores performative strategies of bringing medical and somatic knowledges into dialogue on stage. This address to the doctor is conceived through Kiraṇs grandchild: a parthenogenetic, proto-progenic being (P3B). P3B will be digitally materialized by drawing on 'data points' from yogic, dance and martial embodied practices. 'Dear Dead Doctor' re-imagines what 'being human in the world' could imply in its continuity across pre-modernities and futurities.”

One of the central concerns of our research is how to understand P3B. This is a complex concern with dramaturgical, technological and philosophical dimensions - which each articulate the concern differently:

Let’s be clear that speaking with the dead is by no way a romantic return to a past. Indeed it is a rational and imaginary gesture to reach outside modernity and towards an altered futurity.

Since Kiran is a dancer, we will make a key assertion that the bearing of embodied practices like dance, yoga and martial arts for our future, does not lie in a continued virtuosic display of skilled individuals. So rather than building our futures on recurrent stories of individual geniuses, through our research we propose that embodiment is a shared intelligence that is sort of compressed into an interpersonal storage mechanism, for example of a dance form transmitted from teacher to student, from performer to audience etc. This transmission of embodied intelligence cannot happen in a solid state of storage, like a fixed national culture or a singular artistic genius. Transmission of embodied intelligence happens through a sublimation of the self, an exchange of vaporous particles between us.

This is a 2 dimensional visualization of our digital starting point for developing P3B, depicting how data can be deduced from embodied practice and materialized in a pixel-cloud:

The 3rd century Sanskrit text Tattva Bodham or Deep Materiality, offers a 3 dimensional conception of being-human. Moving away from the very binary conception of body and mind, this yogic conception instead describes a trinary materiality of gross, subtle, and causal dimensions of being-human. Our gross beings are bio-material substances, our subtle beings are psycho-energetic presences, and causal beings could well be our by now im/material ancestors.What we will do in the lab here at the Akademie is at first a Capture of images of Kiran`s gross body in yogasana/postures, dance and martial movements. Using photo, video and motion- capturing technologies, this will be a capture of the gross or bio-material dimension.After this digital capture of the gross body, rather than pictorial visualisation, we will move towards a pixelated visualisation in which the image is structured/choreographed using principles of the subtle body: sensory modulations such as touch of the skin and blink of the eye, using rhythms of breathing, using dream-images and visualisations.

Ultimately our creation of P3B will be a visualisation of a subtle body derived from data of a gross body. This data is primarily embedded in the living archives of embodied practices. But the troubling thing is: data, in contrast to evidence, is not something given. Rather it has to be extracted from evidence. In contrast to evidence, which can be binary: true or false, data is rhetorical; it is something constructed to create certain effects, therefore it has no truth value. Data is just the result of curating an archive. In this process of curation, data is transformed by abstraction, aggregation etc. Data eventually will be that what Kiran and I take into account. So we will struggle with questions that are deeply connected to this very notion of data and its visualization. Questions like: What kind of data counts as knowledge? Is it all about counting? What counts? Who counts? Can we blend counting and understanding in a meaningful way?

Yet even as we invoke such a dialogue of the philosophical with the technological, we are careful towards the Californian talks and promises around disruption through technology. Eventually this Californian perspective claims thatconsciousness could exists independent of embodiment. And that one, for example, could upload the consciousness into a cloud and that it would exist there independently from the body from which it emerged.

We want to intervene into this perspective of a loss of embodiment, which is also symptomatic for the western discourse of posthumanism; We want to ask instead, how the embodied practice of theatre and dance can be expanded into a practice that thinks with and through data? This also connects to Kai`s research interesest into the field of dramaturgy. Which Kai understands as a practice that is constantly expanding the possibilities of theatre.

The following is an excerpt of the film LOSERS AND WINNERS by Ulrike Franke and Michael Loeken, which documents the dismantling of the Kaiserstuhl coke factory here in Dortmund and its shipment to and reassembly in China in 2005:

We are introducing this excerpt here because it hints to a second biographical layer of our project. Because Kai was born and raised in this city and grew up in a workersfamily. Some of his relatives even were associated with the depicted coke factory- while he has moved to Beijing in 2016 where he developed a BA program in dramaturgy. Growing up here, he could witness the effects of the end of industrial culture on the local peoples minds: a loss of belief in any valuable future prospects, a rise of violence and xenophobia were very often the consequence of it. To some extent, we regard the city Dortmund as a symptom for a broader “western condition”, which we characterize through the three buzzwords: loss of identity, decline of western empire, transformation difficulties.

Seen from this perspective, Dortmund`s embracing of the digital (academy, port, university) points to questions around the sociological foundations of the digital: How can the digital be an opportunity for reinventing one´s identity as a city- or (in the case of nowadays Chinas) even as a nation? In our project, we are looking for ways to interact with the very concrete city that hosts us. We are planning interviews between P3B and local citizens of Dortmund, as well as with the global networks of media scholars and philosophers of science. The data collected through these interviews with P3B will then form the textual basis for the letter to the dead doctor.

Like we said in the beginning, we want to use this presentation as a performative-documentary gesture to write out our research questions in this wiki. And basically we have now oriented our research around the following 5 questions:

What kind of knowledge can data produce? Is it all about counting? What counts? Who counts? Can we blend counting and understanding in a meaningful way?

How can we possibly bring P3B from his 2 dimensional state into the 3rd dimension?

What Indic knowledges of the human body-mind did the doctor have to disinherit? And could these knowledgesbecome accessible through surviving practices of temple dance and yoga?

How can the embodied practice of theatre be expanded into a practice that thinks with and through data?

What does it mean when the digital becomes a medium for reinventing identity as an individual, as a city, or even as a nation?

[Rehearsal Notes: 19  September - 1 October, 2021]

In order to prepare the visit of our programmer Matthias Härtig, whose expertise will help us in creating P3B, we started discussing the question of how to select from the living archives of Yoga, Kalaripayatt and Odissi-dance. We want our methodology to be transparent and consequential, which is even more important when one considers, that in contrast to Yoga and Kalari, Indian dance was completely reconstructed during the turn of the 20th century. Eventually, we decided just to focus on the pedagogies of Yoga, Kalari and Odissi- as they were taught to and are practiced by Kiran. In case of dance, we will draw, among other things, on Kiran`s training units. The following video displays such a unit that Kiran actually recorded for his master Bijayna Sathbathy during the time of the Pandemic (when personal training was not possible).

Kiran posesses several documents from his late grandfather: 5 pages entitled “Dates to appreciate and to remember”, 16 pages entitled “List of Patients”, several unnumbered pages and several wordless pages, and eventually the draft for an autobiography (pages 3-115). On September 30th we started to decipher, transcribe and read this autobiography, that we eventually put up on the stage of our lab.

[Rehearsal Notes: 10 October - 17 October, 2021]

These videos show Matthias Härtig and Kiran Kumar during the Motion Capturing:

But it's only really digital when there are glitches:

In this interview our programmer Matthias Härtig talks about the historical and technical particularities of the software “Kalypso”, that he is using to process the data from Motion Capturing:

Photographs of the performance presented at Academy for Theatre and Digitality, January 21st, 2022

Essay: Dear Dead Doctor's Motion Capture and Eadweard Muybridge's studies on Motion Pictures

“Time was at his command as it had never been at anyone’s before. A new world had opened up for science, for art, for entertainment, for consciousness, and an old world had retreated farther…The modern world, the world we live in, began then, and Muybridge helped launch it.” (Solnit)

When we started the infrared recordings of Kiran, we were fascinated right at the beginning by how much the recordings resemble the optics of Eadweard Muybridge`s studies on motion.

Muybridge's work on the technology of moving images is at the beginning of artistic research as well as a new relationship to space and time. As an artistic research project that uses the digital to question modern understandings of space and time, it made us curious to think about Dear Dead Doctor in relation to Muybridge.

The 1870s revolutionized not only industry but also human perception. It was in this decade that the newly invented telephone and phonograph joined photography and railroads as instruments for erasing time and space. It is also this decade, that Muybridge begins his motion studies that will ultimately form the precursor to cinema. He is commissioned by the Californian ex-governor and railway entrepreneur Leland Stanford to find out whether a trotting horse has all four feet off the ground at a time (which it has!). This collaboration between the then already famous photographer Muybridge and the entrepreneur/politician Leland Stanford, takes place in Palo Alto. It is Leland who, in memory of his late son, will endow today's Stanford University, which has arguably become the academic incubator that has contributed significantly to the emergence of today´s Silicon Valley. The Muybridge/Stanford collaboration prefigures some of our current understanding of artistic research. In fact, photography in the 1870s was really a craft, because each photographer had to develop his own optical apparatuses and chemicals to be able to make images at all. Muybridge had become famous for his outstanding camera shutters. To solve Stanford's little scientific question about horse movement, Muybridge constructed a special track, with wires on the floor that the horses touched and thus triggered photographic recordings of their movements. At the end of the 1870s, Muybridge developed the technology of the zoopraxiscope that showed his still images in rapid movement. A precursor to today´s cinema.

Muybridge's study of Stanford's horse (which actually was called “Occident”) showed that the feet of the trotting horse do indeed leave the ground, but only when they are under its body, and not, like it has been always depicted throughout the history of painting, when they are stretched away from the body. Muybridge's Moving Image technology enabled the human eye to see what had never been seen before. Time stood still. Walter Benjamin will later speak in this context of the “optically unconscious”, which is made visible by the camera. And it is Eduardo Cadava who eventually showed how much Benjamin` s thinking of history is deeply endowed to the technological apparatus of the camera. This is especially evident in his theses on the philosophy of history, in which he speaks of the “standstill of history”, or of a past that “flashes” up. One could even try to extrapolate from here, that some significant parts of our modern discourse around history, would have not been possible without the technology of the photography and the conceptual metaphors it provided.

Muybridge has captured and recorded movements by photographing their sequence and stringing together 24 images of this sequence per second. We in turn capture motion by using 26,000 data points or pixels. We are interested in the changes that happen within a motion sequence. For this we understand the data point not as a representation of a positional state, but as the expression of a difference between positional states, which results from a movement. Thus, our data points are the memory of a motion. Again, this memory can only be expressed by specifying certain processing parameters, such as whether and how long a pixel should remain in existence after a movement has been performed. By setting these parameters, it is possible to visualize previously made movements, for example. This possibility represents, in analogy to Muybridge, a new form of making visible what was previously optically unconscious.

In our aesthetic approach, we short-circuit this form of visualization with the Vedic idea of a subtle body. For the possibility of making the absent visible refers to the immaterial reality of the subtle body. We use digitality here to counter the modern understanding of history generated by photography with a different experience of temporality.

Essay: Contaminatedly Yours: Genre, Information, Body by Nishant Shah - Reflections on Dear Dead Doctor

One of the keenest promise and biggest myth of digital technologies is cleanliness. There is a continued replaying of the digital systems as clean, reliable, and free from unwanted contamination.

• In e-governance it takes up the SMART (Simple Moral Accountable Responsible Transparent) rubrics, insisting that digital governance can be cleaned of the excesses and corruptions of analogue bureaucracies.

• In computational design, it favours storage over memory, presenting stored data as clean and sorted, able to be retrieved and remembered by querying algorithms, for an infinitely digital future.

• In probability driven computational networks where the human is often sought to be replaced by computers undertaking mechanical transactions and calculations that perform efficiency and scale, this is often illustrated by showing human circulation of information as constantly mutating and changing whereas computational circulation remains discrete and contained by the protocols within which it is established.

The affirmation of cleanliness is both an exercise of control and a black-boxing of technologies, despite the fact that we continually see how computation technologies are ontologically and manifestly produced through multiple layers of contamination. A cursory look at algorithmic governance practices opens up a field of intentionality, bias, encoded discrimination, and amplified filtering that leads to the production of harm and violence without accountability and restitution. The obsolescence of databases, the leap-frogging of technologies, and the continued breaches and leaks of data and information belie the idea of immortal data and indeed presents data and information infrastructure as fragile and prone to breakdown and manipulation. Especially in the world of self-learning algorithms and networks of correlation, we continually see our reliance on unexpected, undesigned, and unplanned for variable queering models, producing states of exception, and leading to designed deviance which can neither be planned nor controlled.

Cleanliness then is neither an attribute nor a condition of digital networks and their spaces. The foregrounding of cleanliness has to be seen as an attempt to clean bodies, informations, and approaches that threaten the power, destablise the status quo, and resist the benign narrative of computation that is being naturalised in our everyday practices of digitalization. Cleanliness has to be recognized as an active way by which resistant data and technology usage can be controlled, punished, and penalised in order for dominant narratives to be favoured.

This insistence on cleanliness and controllability of digital technologies also creates the dominant aesthetic of our times that reinforces this filtered, curated, cleaned digitality as the de facto mode of visualising and engaging with the digital. In Dear Dead Doctor, we eschew this promise of cleanliness, and even when working with these technologies of cleanliness, continually look at different conditions of contamination that the digital technologies have to offer. From parthogenetic births out of dirt to the diffuse and distributed data, we examine ways by which the aesthetics of digital contamination can be constructed by looking at the logic, logistics, mechanics and intentions of networked computation on three different sites.

There has been a recent autobiographical turn in the performing arts, where the attempt has been to centre the lived experiences and histories of the individual as data for new performances and interventions. The autobiographical turn seems to be human centric but perhaps mimics the ‘deep mining’ and ‘data harvesting’ possibilities that new digital technologies of ‘more than’ and ‘smaller than’ the human seem to offer. The autobiography becomes a way by which lived and affective experiences can be re-narrativised to become a spectacle which can then populate the ever expanding data sets of digital storage. Autobiography as a genre also lends itself to verisimilitude and historical reconstruction supported by archives and evidences that can help clean up the realm of the personal for better intelligibility and taxonomy.

However, the space of theatre and performance is not to deploy these digital aesthetics and queries in order to produce spectacles of machine logic. The challenge of theatre is to create new genres and modes of autobiography that can destabilise this individual turn by exploiting other potentials and possibilities of the digital.

In this performance, we are thus moving towards an biomythography, where through the tropes of autobiography, we produce new myths which are part fiction, part lived-reality, part fantasy, and part politics. The idea of biomythography is not a resistance to digitality but in fact a joyful appropriation of its cleanliness rhetoric and finding new modes by which the post-digital can be performed.

Instead of thus, relying on the archive as discrete, the memory as contained, the record as sacrosanct, and the evidence as the only way by which a story can be told, automythogaphy creates a new terrain of post-digital performance where we lean into our pasts as contaminated and queer, open to interpretation and intervention even as we use the elements of autobiography making. In this project, we are perhaps looking at how the digital affordances can help move from autobiography to biomythography:

Positioning the 'Origin Story' as a Collective rather than an Individual one

It is at the heart of digital computation that the nodes do not have a linear, comprehensive, origin story where it pre-exists the network and intention of information circulation. Nodes in a network are products of intention – and each node is ontologically replaceable by any other node in the network. This idea of nodes as universally interchangeable offers us a new entry point into the personal history: That the personal history is both an imagined one, and a collective one. Autobiographies, as Jeanette Winterson had once said, are stories about other people, while pretending that they are stories of the self. What would it mean to have this digital possibility of telling the story of multiple people – entire generations and communities, instead of focusing on the self.

If autobiography is about querying the self, biomythography has to be using the self as a query that tells the collective stories of the people, things, events, and memories that both precede and escape the formulation of the ‘I’.

One of the imperatives of the conventional autobiography has always been the act of reconciliation. Whether it be in epistolary or confessional forms, the autobiographical subject performs the stitching of fragments and actively filters things that cannot be synthesized into the discrete subjectivity.

Reconciliation is a performance towards an external authority that requires for the production of this discrete subjectivity so that it can be isolated, tracked, managed, and controlled in its unfolding. Within digital networks, there is, however, a possibility to exploit the mechanics of ‘hyperlinking’ which do not need to perform causality or synthesis between two disparate objects within a network. The hyperlink is a freedom from the tyranny of the narrative, and allows for new modes of co-existence where information can be stacked at the level of the index rather than at the semantic level of the lexicon.

Using this as a trigger point, we examine the potentials of mythobiography as straddling paradoxes without the burden of reconstitution:


The possibility of mythobiography is in letting go of the aesthetics of representation and leaning into the world of simulation, where there is no ‘external reference’ because it is a self-contained loop of small world phenomena. We are looking at mythobiography as an act of interpretation with tainted factuality – not a narrative of proof but a simulation of stake, and thus offer the space for wilful misreading.

If the autobiography is a space for narrative, then this digital mythobiography is a space for metaphors, where the ‘auto’ – the self, which is a representational and realist construction, is replaced by the ‘mytho’- a simulated and an imagined programmed entity, that allows us to explode the probability horizon through potentiality functions.

The autobiographical turn in digitalisation has often been celebrated because of the multiplicity of sources through which the personal narrative can be written. In an age of internet of things, where extreme surveillance is our de facto mode of existence, the presence of multiple recording devices from phones to drones, allows for a multi-faceted narrating of the subject through different sources. The montage of simultaneous streams that each offers a different vantage point and perspective has often been presented as the exploding point of the individual centred narrative.

However, in all of these processes of polyvocality, the autobiography stays compliant to the demands of fidelity. Even though we are already engaging in critical conversations around fakeness and reality, with fake news, misinformation, and deep fake becoming a part of our regular vocabulary, the question of fidelity – having that external referent which will make the distinction between real and fake, or clean and contaminated information continues to hold command.

Kiraṇ Kumār

Kai Tuchmann

The collaborative research between dancer Kiraṇ Kumār and dramaturg Kai Tuchmann is towards the making of 'Dear Dead Doctor', a work of digital and documentary theatre. In the form of an open letter to Kiran’s late grandfather, who was the first doctor of western medicine in his family, the project explores performative strategies of bringing medical and somatic knowledges into dialogue on stage. This address to the doctor is conceived through Kiraṇ's grandchild: a parthenogenetic, proto-progenic being (P3B). P3B will be digitally materialized by drawing on 'data points' from yogic, dance and martial embodied practices. 'Dear Dead Doctor' re-imagines what 'being human in the world' could imply in its continuity across pre-modernities and futurities.

Dear Dead Doctor

Motion Capture/Tracking

Dance

Data Visualisation

Documentary Theatre

2021

Christiane is working on a framework of how to prototype and evaluate social protocols and society building methods in play and theatre context with (inter)active audiences. Markus is curious about playing around/hacking different technologies. A shared interest that we figured out is smartwatches as social devices.

For interesting group interactions with smartwatches as devices (that do not require mobile phones), it would be interesting to have them interconnected, being able to sense (the distance and more) of each other, to measure the carrier's body-information (heart-rate, movement-pattern) and to be able to function as a communication device (with each other e.g. to push and receive game elements or to receive global push notifications).

Here are the specifications of the hardware we need:

how to organize subgroups (tribes) of an instant society in a way that their actions have meaningful impact and can be evaluated afterwards

how to prototype new kinds of social protocols with intuitive instructions/means

how to model a kind of “social sense” (to feel closeness and distance, to feel attention and aversion)

how to model abstract resources (as time) in an intuitive way

how to make the interface as intuitive as possible

build a network where the watches communicate with one another

build a server which saves game state and communicates with the smartwatches

2 x bangle.js *All the Espruinos!* edition

(Since some months we are watching the following watch  makers 

. Unfortunately through the outbreak of Corona the supply chain of the makers is interrupted, so we do not know if we can receive the devices (that were promised not before March 2020) in time. We started looking for alternatives now.)

a simple notebook which functions as a server

bluetooth hubs and maybe wiFi access points

Christiane Hütter

Markus Schubert

We like to see technology as possible space, or making possible space, for participatory processes. During this fellowship, we will use theatre as a living space for encounters, with the research questions: How can we construct society together? And how can we make this process visible for audiences? 
We want to develop a sustainable format for collaborative discourse, in which people can together experiment with different forms of organisation for society. This could start on a small scale, with a family, and can become quite big: How to organise a country?

******

Performance

Networked Objects

Sensors

2020

Besides the aesthetic implementation, we are interested in two main elements:

1. How can non-verbal communication between space (built from sound and scenography) and spectators imply a plot line or narrative?

2. How can the environment's own characteristics play a role, so that they can allow conclusions to be drawn about motivations and correlations? 

Jakob Lorenz

Stella Lennert

'PALIMPSEST' engages with virtual space as an interactive stage, and explores narration through interaction instead of representation. For this, the project takes theatrical literature as an anchor and combines it with elements of gaming. The dramatic texts act as a common ground, a guideline through unfamiliar virtual environments that follow strange rules. PALIMPSEST is experienced via VR goggles and headphones. 

PALIMPSEST

Drama

Stage Design

2022

Humane Methods (2019-present) is a project consisting of happenings and stage productions where dancetheatre, bio-art, interactive music, adaptive lighting, and AI technology integrate into living ecosystems. Each work in the series acts as a branch of a rhizome: each dives into different facets of the same theme and world. The works created so far – 

 (2020) and the forthcoming ΣXHALE (2021-) – combine in different ways performing arts with technology and the human with the non-human to reveal algorithmic violence through sensory, physical and ahuman strategies. It is an aesthetic research into how a synthesis of symbols, movement, music and AI can enable posthuman forms of empathy.

℧R is a solo performance for a body, AI algorithms, lights and the vibrational force of sound. 

Embodied in Donnarumma’s musical instrument XTH Sense, an interactive, machine learning algorithm transforms the inner sound of muscles, blood, and bones into a real-time choreography of movement, sounds and light. Drawing on Donnarumma’s more recent movement research, the piece combines multi-sensorial stimulation with human-computer interaction and somatic practices to test the seeming impossibilities of bodily articulation and repetition. 

℧R is a walk in the dark. A thin body walks while a dim light shines. A path covered in soil is between them. Gazeless, the figure walks and prays, walks and prays, walks and, entranced, pries into the darkness. Its red cloth sanctifies it. Its missing face alienates it. The walk is a loop, a prayer, an aggression, an annihilation and a transformation. An AI computer vision system watches over the body, attempting to distinguish the body from the ‘noise’ surrounding it. The AI wants to represents what it sees, but the system works against itself in a paradoxical loop; the more the algorithm learns to see, the brighter the light becomes. In the eyes of the algorithm, the body is eaten by the storm of light.

The latest chapter in the Humane Methods series is ΣXHALE (2021-). A poetic reflection on ethics in algorithmic societies, ΣXHALE is a procedural and participatory piece lasting several days.

It develops in loops, performed simultaneously by 6 performers* and an AI computer-vision system. As it watches the performers repeat a proto-prayer ritual, the AI learns their movements in the space and then begins searching for anomalies, something that deviates from what it expects to see. The learning process is accurately transformed in real time in music and lighting, influencing thus the events in the form of audiovisual stimuli. With each repetition, each variation and each anomaly, the loops mutate into stories of violence and empathy and the audience is asked to choose how they want to participate in these stories. The interaction between audience, performers and AI gives life to an endless sequence of micro-narratives.

AI in theatre and performance: Strategy & Questions

Technical Infrastructure

The Autoencoder

A Performing Machine

How To: Technological Infrastructure

Andrea Familari

Marco Donnarumma

Humane Methods (2019-present) is an ongoing project by Fronte Vacuo (Marco Donnarumma, Margherita Pevere, Andrea Familari) that reflects on the new forms of violence emerging in algorithmic societies. The technological research for ℧R and ΣXHALE have been conducted at the Academy for Theater and Digitality and this page describes our research questions, workflow and findings in relation to these projects. Please note that this page does not discuss the theatre pieces we created in their entirety, but focuses only on the technological research that underpins those pieces. For further information about the aesthetic, conceptual and dramaturgical research behind these works please visit our website.

Humane Methods

Lighting Design

How to depict violence in theatre?           

Without reproducing violence?
Without aestheticizing it?
Without trivializing it?
Without inflicting violence on others while trying to find answers?

>>>>>>> Thoughts and insights from this search <<<<<<<


In the beginning of 2023 there is this vague vision for a setup which gives me the freedom to experiment with different aspects of the depiction of violence:

a human-sized puppet | its digital face-double | a green screen studio

___________________________________________________________________________________________________


I knew right from the start that I want to work with a puppet.              Firstly, because I think that this is a logical approach to the question of how to depict violence in theatre without inflicting violence on others while trying to find answers. The puppet itself feels nothing. And since I am going to animate the puppet myself, I am autonomous - and bypass putting others in situations of violence.
But above all, I choose the puppet because I am obsessed with the effect it has on the viewer.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

RELATIONSHIP BETWEEN SPECTATOR AND PUPPET

THE SPECTATOR`S (LIFE-BRINGING) GAZE
Although the puppet is obviously artificial (manufactured & dead), the subconscious draws a line from the viewer's body to the puppet's body and brings it to life. 

The uncanniness inherent in the puppet is partly due to its resemblance to humans.

THE PUPPET AS 'DESTROYER' OF THE BODY ORDER
This process creates a kind of feedback: we are thrown back on ourselves when looking at the puppet - the sight of it destroys the illusion of the perfect, complete body.

>>> The act itself is already a violent one.

>>> The phenomenon of the puppet lies in the bodily consternation it triggers in us.

What is the puppet's digital face-double?

I am driven by the idea to combine two components - analog theater work and digital innovation - to create a character that serves as a test subject for me in an attempt to research violence.
I asked myself how I could enhance the strange liveliness of the puppet.
Then I had the idea: what if the puppet can speak for itself?
What if I 3D-scanned the puppet's face and then animated it, let it talk?

The original goal at the time was to merge these two components - and create a hybrid character.
An analogue puppet with a digital face - united in one body within a video work.


By working in the green screen studio, I can put the puppet into different local contexts which gives me a endless freedom.
I´m wearing a green morphsuit so I can erase myself from the material.
The puppet moves as if by itself.

In the end, I want to have a character which is half analog and half digital puppet. It moves and speaks by itself, acts in different "worlds" - which consist of videos, images, different sounds and noises that I collect.
_____________________________________________________________________________________________________

What should the puppet I will be researching with look like?
Do I want to set a clear frame with its appearance? Prescribe interpretations?
What type of material do I want to use?
What must the puppet be able to do - how should it move?

//collaboration with puppeteer and puppet maker Emilien Truche


Since I've never built a human-sized puppet that can move like a human and be animated by just one person, I worked with a professional. Emilien was there to advise me (I had the ambition to really build the puppet myself). He was recommended to me by a friend who works at the 

.
His first questions were mainly about what postures the puppet should adopt and what expressions it should master. In order to be able to communicate more clearly with each other, I looked for sculptures, paintings, and photographs from the news and social media that tell a story of violence in their posture. 

//studying Sicilian puppet theatre - opera dei pupi


Before I applied for the fellowship, I had planned a short trip to Sicily in October. To study and research Sicilian puppet theater there.                   It was important to me to first study a traditional form of puppet theater intensively in order to incorporate what I had learned into my work. What do I take away from what I have seen? What do I explicitly want to do differently?                                                                                               The Sicilian tradition is one that I can experience and explore in life - although of course it is by no means the oldest form of puppetry (as it developed in the first half of the 19th century). A major component of the Opera dei Pupi is the violence: swordfights, tournaments and battles with monsters, sorceresses and enemies.
This research trip was really insightful and inspiring for my work on „Researching Violence“ - and perfect for this point in the work process. I had the chance to dive deep into the portrayal of violence through (traditional) puppet animation and to get involved with puppet making and puppetry.

I had the opportunity to see several performances, paying particular attention to the study of the gestures and postures of violence.

In addition, I was lucky to visit various studios. I was able to talk to the puppet makers themselves, ask questions and even try out the puppets myself and study them up close.
For example, when a character strikes with his sword, the opponent can be split in half, fall apart or lose his head. In the puppet maker's (Italian: oprante) workshop I was able to study the mechanisms and construction of the puppets at close quarters.

// building up my 'archive' (by collecting sounds, texts, interviews, images and video material of places)


From the very first day, I collected material for my archive for the experimental setup.
I was interested in building up a broad spectrum and I mainly used my iPhone for the documentation. Regardless of whether it was video recordings of
- places and spaces that are commonly associated with violence
or as a counterpart to it
- places and spaces that are commonly associated with peace
- I also tried to find 'neutral' places
- as well as places that I personally associate with violence and peace

examples of sound recordings I recorded:  

- Munich Central Station during the Oktoberfest
- bee colony in Düsseldorf
- shopping mall in Dortmund city center
- eerie sounds at night on the way home
- cicadas by the wayside in Palermo                                                             - sounds of the sword fight of the dolls in Sicily


- I was lucky to be able to attend a Blender workshop (by Elisabeth  Drache, 3D artist and a member of the duo The Ruhrgebieterinnen) at the Academy, which proved useful for the next steps in animating the 3D scan.

NOVEMBER+DECEMBER 2023// ANIMATION + COMPOSITION                         

At the beginning of November I took part in a motion and face capture workshop as part of the DIVE FESTIVAL OF IMMERSIVE ARTS (In cooperation with the Figurentheater-Kolleg Bochum). Irina Shutova and Warja Rybakova (both working for DOCKdigital BERLIN) drew my attention to the plug-in Faceit from Blender after I told them what I was planning to do with my puppet. The workshop also gave me a better idea of what I need to pay attention to when animating the handmade puppet in the green screen studio.

//animating the puppet in the green screen studio

3 days of intensive exchange, reviewing the video material, discussing.

WORKING THESES 
a) the puppet itself feels nothing
b) the puppet is not able to play in comparison to the actor or actress
>>> Which means e.g. that when an actor dies on stage, they are only pretending. This can still touch us, but we know that they don't really die. The puppet dies the moment it dies. Right before our eyes.<<<
c) the puppet is familiar to every human being and therefore accessible (in the sense of not elitist)

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++



With this first material of the human-animated puppet in the green screen in front of me, I began to go through my archive (with sounds, videos etc.) and experiment with putting the individual components together.


In order to be able to act as quickly and flexibly as possible, I worked with Final Cut. On the one hand, to be able to cut out the puppet with one click and, on the other, to be able to put my video and sound material together as simply as possible.                                                    At the same time I am working with Blender and the plug-in Faceit to animate the puppet´s face. Since I had hardly any Blender experience, I always used apps (with AI-based animation tools for faceplay and deep fake, like Tokking-Heads or Reface) in between to be able to check ideas quickly.


The composition of the individual parts allows contrasts to be created between the puppet's expression, physical movements, spoken text, sound and setting.
How do expressions of horror that really reach us come about?
When does empathy arise, when does alienation?

We have access to so many images of atrocities that the individual images lose their impact. My working hypothesis is that both a puppet and a completely artificial person (avatar) have the power to sensitize us as viewers to the experience of visual and linguistic violence. The puppet and the avatar feel nothing; we create their suffering, pain and exhaustion solely through our imagination. This creates a direct connection between the viewer and the representation.

// REALIZATION OF THE IMPOSSIBLE
The days of trial and error made me rethink my approach completely.   

                                                                                                                      I had the feeling that - with everything I tried - I was aestheticizing. And thus trivializingviolence.


>> You can't depict violence without reproducing it in some way. <<

I also couldn't get out of the vicious circle of thinking that everything is just my personal view. How do I get to the point where something becomes more universal? To become a little more detached from myself and my feelings?

And who am I to talk about this?
What position do I have within the "narrative of violence"?

These doubts made me take a fresh look at the material I had collected - and think again in a different direction. 

new thoughts: The relationship between puppet and puppeteer is already a violent one in itself. There are clear power structures as to who controls whom.

Where does the relationship between puppet and puppeteer tilt?     

Hanna Rode

How can we tell about human abysses without reproducing or trivializing violence? This question is the starting point for an experiment with a handmade puppet and its digital face double.

RESEARCH PLATFORM

Academy for Theatre and Digitality

QUEUE

Christoph Wirth Sarah Buser Markus Wagner Fehime Seven

active

Similar Projects

QUEUE

Christoph WirthSarah BuserMarkus WagnerFehime Seven

active

Similar Projects

Christoph Wirth Sarah Buser Markus Wagner Fehime Seven