is the knowledge-sharing platform for the 

. Explore the experiments and bold ideas behind the Academy's Fellowship programme, delve into the science behind the projects of our HIDA researchers, and explore the results of international collaborations.

is a platform for discovery and documentation. Each 

showcases a unique artistic-technological research project, with topics as diverse as the use of AI in theatrical design, choreography for and with robotics, and the possibilities for theatre in virtual spaces. Each 

 offers the chance to learn more about the members of our artistic research network. By pooling our knowledge in such a way, creative research into theatre, performance and digital technologies can become more accessible, sustainable, and artistically innovative.

Funded by the Federal Cultural Foundation, the Academy will award a total of 54 fellowships between 2019 and 2022, each lasting for five months. The 

 are designed for people interested in artistic-technical research which explores the possibilities for digital narratives in theatrical spaces, as well as their practical implementation. Fellows have researched a variety of digital methods (motion capture, coding, XR, VR, AR, sensor technologies, robotics, and more) across artistic fields including opera, dance, orchestra and drama. 

Helmholtz Information & Data Science Academy

(HIDA), the Academy for Theatre and Digitality has offered joint research fellowships at the interface of digital art and cutting edge scientific research since 2021. The HIDA fellowships are kindly supported by the 

PlayOn! New Storytelling With Immersive Technologies

 is a larger scale cooperation project, funded by Creative Europe, designed to understand, learn, test and apply immersive technology to theatre, in order to shift classical storytelling to interactive narrative formats.

Artificial Intelligence

VR (Virtual Reality)

Interactive

Sound Design

Projection/Mapping

3D Printing

XR (Extended Reality)

AV (Audiovisual Design)

Theater Dortmund
Theater Karree 1-3
44137 Dortmund
Germany
Fon: +49 (0) 231 226 14330

Address and location of the Academy for Theater and Digitality:
Speicherstrasse 17
44147 Dortmund
Germany

E-Mail: akademie@theater.digital
Web: theater.digital

Tax number: 314/5702/9367
Sales tax identification number: DE 124643894

Management Team:

Executive Director
Tobias Ehinger

Administrative Director
Martin Lizan

Director of the Opera
Heribert Germeshausen

Director of the Ballet
Xin Peng Wang

General Music Director
Gabriel Feltz

Artistic Director of the Schauspiel
Julia Wissert

Director of the Children's and Youth Theater
Andreas Gruhn

Director of the Academy for Theatre and Digitality
Marcus Lobbes

Please also note our privacy policy.

 as the sixth division of Theater Dortmund (hereinafter referred to as "Theater", information about Theater Dortmund can be found 

), is pleased that you are visiting our website. Data protection and data security when using our website are very important to us. We would therefore like to inform you at this point, in accordance with Article 13 of the EU General Data Protection Regulation (DSGVO) and Section 13 of the German Telemedia Act (TMG), which of your personal data we collect when you visit our website and for what purposes it is used. Since changes in the law or changes in our internal company processes may make it necessary to adapt this data protection declaration, we ask you to read through this data protection declaration regularly. The data protection declaration can be accessed, saved and printed out here at any time.


§ 1 Scope of application

This data protection declaration applies to the Internet offering of Theater Dortmund, which can be accessed under the domain www.theaterdo.de and the various subdomains (hereinafter referred to as "our website").


§ 2 What is personal data?

Personal data is individual information about personal or factual circumstances of a specific or identifiable natural person (data subject). This includes, for example, information such as your name, address, telephone number, date of birth or e-mail address. Information for which we cannot establish a reference to your person, e.g. by anonymizing the information, is not personal data.


§ 3 What personal data do we collect/process and use?

Access data without personal reference is stored, such as the name of your Internet service provider, the page from which you visit us, the names of the requested files and their retrieval date. These data are evaluated exclusively for the improvement of our offer and do not allow any conclusions to be drawn about your person. 

If you wish to make use of services offered by us on our website, such as purchasing tickets via the ticket store or ordering a newsletter, etc., it is necessary for you to provide further data. This is the data that is required for the respective processing, such as your payment information when purchasing tickets or your e-mail address when ordering a newsletter. 

We process the data we collect about you for the provision of the services we offer, to answer your questions and to operate and improve our website and applications. 

We process your personal data exclusively to provide you with a comprehensive service via our website or, if you use our contact form or, to process your ticket purchase or, to process your application or, to provide you with requested information about our offers.

Fellowship

HIDA

PlayOn!

discover cutting-edge digital research in theatre and performance 
meet our research fellows and collaborators in the 

Portal: Academy for Theatre and Digitality

At the beginning of the project, the questions we asked ourselves were essentially two: how can one "create from chaos"? Dr. Frankenstein generates his creature not from absence but from hyperpresence. That was an exciting point for me to analyse through the medium of theatre. Second, I was fascinated by the real mystery of the novel: why is Frankenstein not proud of his creation? And, in parallel, why did Mary Shelley always tend to devalue herself and her work? I wanted to investigate this denial. The Creature is the Book. The Book is a vision of the future: it redeems female creativity by claiming its diversity but putting it on an equal footing with male creativity. The gender gap suffered by the world of culture and science is a great mistake we have been carrying on our shoulders until now. We start by redeeming Mary Shelley by marrying her life to her work, creation.

The performance uses two types of technologies: audio material is created with the 

, a microphone in the shape of a human head that reproduces sound in 360 degrees with very realistic effects, picking up both live and recorded sounds. The viewer, equipped with headphones, enjoys the work with total immersion. This recorder creates incredible suggestions and immersion. It is a bit like having Dolby surround sound in headphones. For example, at a certain moment during the performance, in the headphones you listen to the rain and it feels like it's raining in the theatre!

The second technology we use is projection. The show is almost all projection. There is only one actress on stage, Maria Laura Palmeri, and two actresses in holographic form, Lara Di Bello and Giuditta Mingucci. We conducted an in-depth study of a range of perspectives that would make the video relate to the actress on stage in an innovative way. The work on the footage made use of very particular perspectives, even on a human level. For example, during the performance, there is a monologue by Mary Shelley's mother, feminist intellectual Mary Wollstonecraft, who is filmed on a black background and in perspective: the virtual actress's measurements are the same as the real actress's. This makes for a very strange play: one does not realise whether the actress is physically on stage or not.

We tried to embrace an approach that reflected the idea of creation tout cour. In a sense, the idea of "creation out of chaos," which I mentioned earlier, was not only the content but also the inspiration for the method of making the performance, thus in its form and use of technology. Creation thinking of the feminine is something that needs to be detached from just the reproductive sense of materiality, but should be placed on the same level as male creation. In this sense, we feel compelled to pursue a method that brings all these components to the theatre, including human and technological, in parallel.

We would define our research theatre as "technologically oriented" - which is then the motto of our festival, LUCY - in that, we move in the direction of experimentation, of using technology to research new languages and create different degrees of experientiality in theatre, not only predominantly visual, playing on sensory intensity. Our technologically oriented footprint, however, does not neglect an equally immersive focus on the dramaturgy of front-viewing spoken word theatre, which we have been pursuing since 2016."

How did the collaborators work together on this project?

We surrounded ourselves with a group of professional figures with deep expertise to see how far we could go with technological, sensory, and, certainly, human experimentation. We worked, in particular, with a very good sound designer, Simone Arganini, and a visual artist, Cristina Spelti, who also does the mapping of buildings. She helped us calculate all the perspectives, both in terms of the human figure and the spaces. Again, the skills of each person collaborating in the creation are many and sometimes even unforeseen, but that in the end are crucial to the final result.

As just mentioned, the approach we use to do theatrical research is complex, involving different knowledge, people, and sensibilities. In this context of theatrical research, the show is the outcome. Still, all the study and attention behind it requires continuous experimentation, figuring out what works and what doesn't-here the trial-and-error-how to translate an idea into the theatre.

In this case, the design work on video and audio is a year's work. Before assembling the show, we spent a year studying perspective and sound. Where, how to record, and where to place the head on stage (it is a very delicate tool). There was a year of preliminary research on and with new technologies, which takes a lot of time.

Ivonne Capece, Simone Arganini, Cristina Spelti

Did you use existing software and applications


Yes, Simone Arganini, our sound designer and technician, has used some existing software: Ableton Live, Qlab, and binaural software from “Dear Reality”, especially the dearVR micro and dearVR pro plugins, and dearVR Ambi micro for the decoding of ambisonics audio recording. For internal audio routing on MacOS during the show he has also used the BlackHole driver from “Existential Audio”.

Did any of your questions or goals change?

Certainly having worked with these kinds of technologies has created a fundamental question: what kind of contribution can they make to live performance? Having enormous potential, it would be nice to explore them fully. So how far can you go with technologies without losing quality in artistic output? I say this because projects that rely heavily on technology often lose out on the artistic level. This happens partly because they get phagocytized and partly because the use of technology, especially at this moment in history, is at a stage that is still primordial. In our art project, however, it seems to me that the artistic plane has not lost value. However, it would be interesting to understand how far we can go even when the projects are completed.

What were the key milestones in the development of the production?


A key step was finding the team, and so professionals suitable for this work, for sure. Especially for the work done by Cristina, because we didn't need just a video maker, but someone with much broader skills, both in terms of filming and studying perspectives.

Another important part was the choice of materials: figuring out which would perform better technologically and artistically.

The work significantly exceeded original expectations. On the artistic level, it started with a well-defined idea. In terms of staging, I did not expect the use of technology to make such a fundamental contribution to the work, but it did. They supported directing without "swallowing it". Both plans work very well together, so we're also pleased with the initial expectations.

In what ways was the production a success?


From a technological point of view, we can certainly say that we have brought to Italy an almost entirely new type of theatre experience. I have never seen shows in Italy that use the dummy head, so surely ours is experimentation. This new entry, however, is not solely justified by a taste for experimentation, because we offer the viewer an immersion from the point of view of audio and video that is unprecedented. This study on perspectives, on putting two virtual actors is new. For these reasons, we are satisfied with our work.

If other people were to take a similar approach, what advice would you give?


The advice I would give to those who want to enter this experimental approach is to surround themselves with professionals familiar with such technologies because there can be a lot of trial and error before you reach the goal. Working with professionals decreases the (still long) research and creation time somewhat. Then depending on the work you are preparing to do there are pros and cons. But forming a team through which one can face all kinds of challenges (inherent in experimentation), and try to solve them as one goes along the work, is indispensable. My advice, therefore, is to have an even heterogeneous but capable and cohesive group.

Team

Giuditta Mingucci

Ivonne Capece

Simone Arganini

A natural science student decides to create a human being. The experiment is an unexpected success. Its creator, however, despairs. We discover that the author, Mary Godwin Shelley, is not entirely comfortable with it either. Throughout her life she denies and downplays the literary scope of her work. Between wireless headsets, virtual settings, real actors and holographic avatars, the show is a dreamlike journey into the novel and, at the same time, into the lived experience of its author who confuses herself with Dr.Frankenstein, the Creature with the Work and the characters with memories. 
___________________________________________________________
An Elsinor production as part of the Creative Europe project PlayOn!

Frankenstein

Spatial Design

Elsinor Centro Di Produzione Teatrale

In this research it is important to not use visuals as a mere translational system. The main interest — which more and more becomes Max Schweders expertise — lies in working on music and their visuals simultaneously, where one artform reciprocally informs and inspires the other. Decisions and actions echo back and forth during the creative process.

Reactively visualizing all dimensions of a sound, is a complex task. Not only common or just bipolar parameters, such as dynamics, pitch or articulation, but also the more complex relations and connections between sounds can be visualized. Additionally, a sound-reactive virtual embodiment of a performer, provided by e.g. 3D Camera Input - can furthermore amplify the connection between performer and sound.

Many core elements of the code, such as the 

 Object for Pd, necessary for sending analyzation-data from Pd to Unity, have been developed by Collaborator and Mentor 

Recommended minimum Sensor SDK configuration for Windows is: Seventh Gen Intel® CoreTM i3 Processor (Dual Core 2.4 GHz with HD620 GPU or faster) 4 GB Memory. Dedicated USB3 port. Graphics driver support for OpenGL 4.4 or DirectX 11.0.

Azure Kinect SDK (refer to repo for version)

https://github.com/strangerattractor/Soundvision.git

For installation and setup, follow the instructions in the Repository

How many channels can be analysed simultaneously?

Currently 16 audio channels can be constantly and simultaneously analysed

What type of data do you get from the channels?

Loudness of manually chosen parts of the frequency spectrum (float)

Index of a closest match of a list of trained sounds using 

What type of data do you get from the AzureKinect?

Frame Difference, between two frames (2d texture)

SOUNDVISION can be cued manually by clicking the Next/Previous/Reset button in the Pd-Patch, or it can be cued by external Midi-Inputs. Refer to Tutorial-Videos for more info. A cuelist is achieved in the form of a .csv file that is being read by a C#-script in Unity. Different states of visualization are achieved by utilizing Unity's 

 feature. The “playhead” of the timeline is moved to different markers in the timeline, following the cue-list.

SOUNDVISION also has a Script following to the method CYLVESTER uses to cueing its shows, utilizing Midi-Program-Changes coming from the CYLVESTER-Instrument-Setup.


As a musician, performer and multimedia-artist Max Schweder works for the international, inclusive and interdisciplinary performing arts company 

. Un-Label pushes towards a more diverse, mixed-abled cast on stage and challenge themselves to make their performances as accessible to anybody as possible. They use artistic creativity and technology to achieve these goals. With his electro-duo 

, Max Schweder plays concerts and parties all over Germany. CYLVESTER developed their very own visualization 

music-video FRANK by CYLVESTER (2018), showcasing CYLvision:

Max Schweder

As a researcher at the Academy for Theater and Digitality in Dortmund, Max Schweder is developing SOUNDVISION, the open source successor of CYLvision.

SOUNDVISION is being developed as an opensource Unity and PureData based artistic toolset for reactive real-time-visualizations of sound, music and movement with the goal of making performances more visually perceivable, also for example for a deaf audience.

Visualization of sound is always interpretation, translation, transformation from one sense to another. Can we generalize this translation? Can we shorten the necessary steps that have to be taken? Or is the absolute subjective, artistic interpretation the key factor to make visuals tangible?

SOUNDVISION

Data Visualisation

Performance

2019

I am interested in immersion, the "reality" part of Virtual Reality. As CyberRäuber, Marcel and I have tried and created various avenues into immersion. A very short summary of our findings would be that a high degree of immersion is generally not necessary to tell a story or to convey what you want to. And that immersion generally suffers if you aim for too much realism: the more you rely on your target's collaboration, their imagination and established social and cultural norms, the more they will get "into it". Participation / interaction trumps detail, trumps realism.

But of course, the more you let people interact, the more you lose the the ability to steer a narrative. In this dilemma we usually opted for limited realism (high aestheticization), limited interaction, a limited scope of character, space and time, and worldbuilding as a narrative strategy.



 as an idea manifested itself over the course of years. It basically grew on the compost of CyberRäuber's work in theatre, in which we usually didn't have the time, scope and resources - and more importantly - not the framing for such a project.



 aims to be maximally immersive with a given, limited technological effort, following an approach of a minimally detailled realism, maximal imagination and direct, personal contact.

Hundred thousands of years of evolution have led us to develop certain quirks that can help storytellers create compelling narratives, something that can also be ab-used, of course. Think about what we imagine in clouds, in the shadows of a forest lighted by flickering flames of a campfire. What kind of faces we see in objects, landscapes, moving leaves, what voices and sounds we hear in the wind. The less there is to see or hear, the more we augment; little impulses, visual or aural clues are launch pads for our imagination, something every horror movie, every mystery story told on a nightly walk makes use of.



One way of direct, personal contact is the human voice. It can supply content and - with language, dialect, wording, timbre and emotion - context. We ascribe very much to a voice that could only be safely verified by meeting and conversing with the person with this voice.
In this project, the means to deliver the voice will be a recording, a voice message. Why? Because it is a controlled way of communication. For the ordinary person an ordinary conversation is something that can be quickly understood and deciphered, the situation, the conversation partner can be assessed. We would normally aim to reduce mystery, to understand what is happening and only continue to engage with the person and the topic if intrigued and feeling save. A voice message instead can be mysterious, incomplete, poetic, out of the norm. It can also contain additional information, environmental sounds. A voice message can be dilligently tailor-made for a situation and our goals. It doesn't need and anyway couldn't deal with an instant reply. The communication here isn't a realtime dialogue, but an elusive conversation over a longer timespan, similar to an infrequent exchange of letters, including lost and/or overlapping messages.




Nevertheless the recipient has to have the feeling of being heard, of being reacted to. And as in any communication, truth and trust plays an important role...

During the fellowship different areas have to be explored:

Voice - characteristics, difference to simple text

AI - what can be done with artificial neural networks? Story, text to speech, speech to text...

Soundscape - binaural sound, artificial and recorded; AMSR (Autonomous Sensory Meridian Response)

The role of time (length, pace, intervalls,...)

As was to be expected, my research plan didn't survive contact with reality. I thought the main focus of my research would be on "Soundscape" and "Story". Instead, most of my time was spent on voice and specifically the creation of text and voice with neural networks, with timing and the delivery system of the messages (AdaptorEx, Telegram). 

"Soundscape" didn't play the expected role due to the realization that interaction with smartphones, the chosen device, is mainly happening visually, and when people listen to voice messages, they oftentimes don't use higher quality headphones and will be surrounded by other environmental audio. After creating and afterwards checking out some binaural and ASMR material on the phone it didn't seem relevant any more in the scope of this research. It is one of those engineering issues that can take a lot of time and are fun to work on, but ultimately don't deliver for an audience.

"Story" got deprioritized when I realized that neural networks, or more precisely large language models (LLM), would take over a large part of creating the narrative. 
Here maybe some further explanation is in order. As CyberRäuber, we've been working on AI in theatre since 2018. In our so-called "neural theatre"-pieces (Prometheus unbound, Linz 2019; Der Mensch ist ein Anderer, Wiesbaden 2021; Mensch am Draht, Ingolstadt 2022; The Merge, Bangalore 2023) we aim to unleash the creative power of generative AI. The text the actors speak and play is created in real time by LLMs and transmitted to in-ear speakers via text-to-speech. You could also call this a text-to-body-mechanic, the actors repeat and interpret what they hear and thereby embody the AI. The process works amazingly well. While still maintaining parts of improv theatre, this isn't improv because the actors follow the logic and emotional content of the texts. Actors can never lose focus, have to stay "on" all the time and are oftentimes visibly struggling with their task, because the text will always be new and unknown, its twists and turns unforeseen. This is fresh material! The audience on the other hand usually themselves struggle with the uncanniness of the whole set-up, try to follow what's happening on stage, enjoy the actors' struggle and they take on much more responsibility, because the play develops on stage and in their heads.

So, AI is already playing a crucial part in CyberRäuber's artistic methods, and we've been following the latest developments in the field on a daily basis. And with the current generation of LLMs (2023/24) we now actually possess very powerful storytelling tools. The question was: is one person alone capable of creating, editing, curating large amounts of coherent, stylistically appealing, sensible text? Can a network convincingly take on the role of a narrator? To check and ultimately prove this thesis became one of the main isssues of the research at ATD.


This documentation is an ongoing process and will grow as the fellowship progresses. Last entry: 30. November 2023

Björn Lengers

You just received another one of those mysterious audio messages that strangely overlay your surroundings and ask you to perform a seemingly insignificant action: to walk a little slower, to press the pedestrian lights even though you don't actually want to cross the street, something else. A voice wants you to believe that you are decisively changing the future.
- - - Resonance is an interactive, immersive and theatrical narrative, which can be experienced by many people independent of time and place on their smartphones.

Resonance

2023

Story- or theme-wise we wanted to see, if we could push people with just a short period of time into more radical thinking. It seems, we were too successful in that (as a final test we gave the audience the chance to vote off their extremist captain – in all of 18 shows it never happened, that all four capsules voted off their leaders (a couple of times we got rid of three).

What technology was the focus of the work, and why?

Algorithms and how they can impact the world, including how they can lead to extremes (similar to, for example, YouTube algorithms offering more and more extreme political content, see 

) and how when we are very reliant on decisions from AI (even though our project wasn’t AI by definition), extreme groups can form. 

For audio we were experimenting with different audio equipment to have parallel storylines happening with discrete audio, but also having the opportunity for the groups to interact with each other when necessary.

Testing out when is technology useful in theatrical story telling and when it becomes a distraction, is always an interesting subject. In this project we tried to make technology as „invisible“ as possible.

In audio we were hoping to get as far as possible in creating 4 distinct worlds that ran in parallel. Switching from one group to another would leave the listener in the same spot in the story, but convey a different auditory environment.

How did the collaborators work together on this project?


We had a tight core team, consisting of director, dramaturg, composer and video-artist and quite soon the developer also joined the group. When we discussed the narrative concept, we immediately talked about all the technical aspects. We always tried to look at the concept as a whole picture. This probably helped us to avoid quite many errors, but, yeah, still many-many new problems popped up, when the real rehearsals started.

We explored different technologies connected to motion tracking and consulted experts on the subject. It was also invaluable to present our project to another group and have them give feedback/questions/concerns as an outside perspective.

The audience tests, especially the very early ones with students, were super useful – live coding and fixing bugs in the local network, so everyone just received the fix immediately. We should have organised more of these. We did have a few tries where the web app completely crashed and the actors continued analogue – it sucked, but it was good to have the experience. Of course, we also got to fix a bug thanks to that.

Firstly the idea, how to synchronise the activities in four capsules “jumped” in our heads quite late in the rehearsal process (probably already while the audience test were going on) – suddenly we realised, that the only possibility is, that we set exact times for every phase and put the timer in the pads of the captains, so they always see, how much time is left.

Secondly we planned to have feedback round immediately after the show in every audience test, but not in the official shows. On the day of the premiere we suddenly realised that the feedback had turned into an organic and logical part of the show and we couldn´t skip it. So we didn´t call it a feedback round any more, but the finale of the show.

Did you use existing software and/or applications?


Video: Zoom on 4 separate laptops and large TV screens for video calls between the capsules.

Audio: we used Ableton Live, since it offers a lot of flexibility in terms of controlling different audio streams. We also used silent disco headphones and a separate audio guide system to achieve 4 separate channels for the audience.

Did any of your questions or goals change?


At first audience participation was a bigger goal throughout the play, but we soon decided to leave more control in the hands of the actor in the group or the whole show would lose control (especially with the timers running etc.).

Finishing certain stages of the specific web app, e.g. when the Humanity Shop was finished or when the first version of the Quiz games were finished;

Audience tests – each one was a milestone to find the proper balance and also shortcomings in the technology used.

First milestone was 9 months before the premiere, when we realised that we can´t continue with the motion-tracking technology and suddenly many other possibilities were on the table.

Second milestone was 6 months before the premiere, when we met the sociologist Andero Uusberg, who gave positive feedback to our division-into-four-groups-system and also suggested that we should create our own test group, who´s results could be useful basis for the personality test. In the end almost 100 volunteers answered our questionnaire and helped us to perfect the sorting system.

Third milestone was three months before the premiere, when we finally decided to use four green houses as the capsules. After that everything else started getting clearer on a daily basis.

We got some data and human experiment results. How fun! We are in the process of analysing the data, which is going to offer a base for writing more academically about the production–as the storyline changed for each evening, we can analyse patterns and more common answers, etc.

We also now have a web framework for creating more of these types of projects.

We hoped to create a show in which the audience feels individually tied to the outcome of the show. We had a goal of giving the members a series of decisions and their movement during the play would dictate the outcome. In the end we had to change the process of audience participation, but we still managed to give several discrete experiences for the audience members and made them consider the choices they are making.

In addition to the web app, the fact that we incorporated so many complex technological solutions in one single production is really impressive (video calls, synced headsets).

The audience experienced a novel kind of theater and became invested in the success of their assigned group.

What elements of the final production would you change?


The play definitely has a lot of potential to grow larger and have more branching outcomes.

With this one we tried out an extremely compact version of this “game” (1 hour in capsule, 2 hours altogether), but it would be quite a different thing, if this experiment would last 4 hours (probably the audience will get much more freedom etc.).

What advice would you give to artists taking a similar approach?


Get outside help as soon as you get stuck. We (I) really solved the scalability issue at the last minute.

Creating a web app is very annoying when you have to take into account older phones – we did have spare ones, but it didn’t help that much, and we still encountered issues. If financially feasible, having the same device for everyone in the audience is a good plan and will relieve a lot of stress. This goes without saying for native (non-web browser) apps.

Workshop different paths the story could take. The production will definitely take a lot more time than a normal play would, so it would be best to map out the story paths as early as possible. Map out important story beats – times where the different parallel storylines converge and change the goals/situation of all groups.

Is the coding for this project open source? What language was used, what is the function of this code, and where can we find it?


I haven’t included a licence, but I can – the architecture was created in a “plug-and-play” way*. It was my Bachelor’s project, so it actually has architecture drawings etc as well, which I can include if we ever get to writing wiki/documentation.

*Well, it does require coding if you would wish to change the logic. But scanning QR codes and making quizzes with phases that can be done in groups – 100% reusable.

Building new games on top of the existing audience logic is also feasible.

Helen Rekkor

Mihkel Seeder

Vivaarium is a shot at survival. It’s hope for a lengthy, successful symbiosis on this beautiful blue planet. Vivaarium is a high-tech capsule, where people can lock themselves up for a period of time, so that our planet could heal itself from all the man-made harm. 23 years in a capsule for survival of our planet – not bad deal or what? 

This production combines elements of technological and traveling theater. The journey gives the audience a chance to make decisions and re-evaluate their points of view. The central theme then becomes: how do we notice the information bubble around us and how do we make sure that it doesn’t become too restrictive for us? 

How does it work: audience members make different test and are divided into four groups based of the results. Every group is gathered into a capsule, where they should spend the next 23 years, but in reality it takes a lot less time for their good faithed captain to turn into a complete tyrant. How bad does it have to get, that you will want to abandon your own capsule and a group of people most similar to you? 
___________________________________________________________
A VAT Teater production as part of the Creative Europe project PlayOn! in collaboration with Teatro O Bando

Vivaarium

VAT Teater

The most important question was to figure out what kind of story requires to be performed not inside the theatre venue, but on the street.

This is an audiowalk, so the audience listens to the performance with headphones on. The special thing about this performance is that they hear not only pre-recorded sound but also the actors' voices live. Another challenging technical solution was the use of AR. For this, we made a motion capture recording, and then with this avatar, the main character has a scene in the performance.

I was curious about how the audience’s focus can be led in a busy street or square, where anything can happen, and how the street turmoil can be incorporated into the performance. Also, whether many seemingly loosely connected episodes come together to form a complex experience.

The most important thing was the planning: since the performance consisted of many individual parts, we first saw how these separate elements were connected only during the run-throughs. Therefore, our collaborators often worked completely separately from each other.

One of the biggest challenges was adapting the live voice to the sound system, creating the avatar, and finally creating an application. They had many flaws.

Did you use existing software and/or applications?

We used existing motion capture recording software (Rokoko), but the coders developed AR software for the avatar. The app is called 

 and can be downloaded from known distributors (AppStore and Google Play)

Now, I see how many hidden challenges there are to make a performance in a busy urban space.

What were the key milestones in the development of the production?

Finding the topic, then choosing the location (planning the route), technical preparation (motion capture recording - avatar; headphones, live sound), developing the text and story, rehearsals and testing.

The performance is a success that exceeds all expectations. We have created an audio walk that is more complex than any similar production that I or my experienced colleagues have seen before.

In what ways was the production a success?

On one hand, it can be considered successful due to its uniqueness, and on the other hand, due to the unusual genre, the spectator receives a special, incomparable experience.



What elements of the final production would you change?

There were parts of the show that I changed or left out before the premiere. But I would change the scene with the avatar if I could.



What advice would you give to artists taking a similar approach?

When you choose the route, observe not only the locations but also what usually happens there, as this information should be integrated into the performance.

György Vidovszky

Kolibri Theatre’s latest PlayOn! coproduction invites the audience on a special journey through time and space using immersive technologies and interactive elements. Realised within the framework of the ’Budapest Grand Novel’ project "Urbán legendák - A tartozás / Urban Legends - The Debt" invites the spectators to step out of the theatre space and into the city, where they can take part in an extraordinary "theatre audio walk" with headphones on their ears.  
 
The play is based on Renátó Fehér's "Kaleidoscope", originally published as the 6th District/Terézváros chapter of the Budapest Grand Novel, which was adapted for Kolibri Theatre by István Tasnádi. 

In the story, a father and son set out to collect a debt. They wander the streets of Terézváros from Jókai Square to Nyugati Railway Station but find only fragments of the past: objects from a lost exhibition from the change of regime, a homeless man from a better time, moments from the ever-changing, much-seen Nyugati Square. The surreal walk, full of surprises, ends with the evocative opening of the shopping mall next to the station. "We got home. We are at home finally." 
___________________________________________________________
A Kolibri Theatre production as part of the Creative Europe project PlayOn! in collaboration with the Academy for Theatre and Digitality

Urban Legends: The Debt 

Motion Capture/Tracking

AR (Augmented Reality)

Kolibri Theatre

The main idea of THE FUTURE is to develop a play that tries to refer to the current situation of mankind. Young people are facing a future that seems to be chaotic, dark, overwhelming and disturbing. Due to the corona virus we expand even more into digital worlds where we find no structures and ways to connect, but where we can also get lost and alienated. But what will the upcoming society be like? What are we connected to? How does our reality feel like? Are we lost to the world? Is the world lost to us? Have we ever been modern? Who can we trust? What can we hope for? Is the bubble we live in going to explode?

The Internet has great impact on our perception – we are influenced on many layers. We are deluged by DATA and information and we have to cope with it. The time is literary out of joint. Past, presence and future seem to take place at the same time. It’s hard to get a clear idea while there are so many pictures and sounds penetrating us. So many news and stories that have no context. We need context. And maybe we have just no words to express what is really going on.

The aim is to develop a collage of voice, body, sound and surrounding in order to depict the simultaneity of digital environment but also to explore the current situation and to ask for strategies to face the future with courage and joy?

Website

Nils Corte

Roman Senkl

Annette Müller

In the shadow of current global crises (pandemic and climate change), the world premiere by KJT Dortmund asks how young people can share their future and decide which structures and systems are worth preserving. How can young people avoid losing touch with their world despite the overwhelming power of digital communication in the pandemic? With poetic devotion, this research-based play deals with spaces of possibility and visions of a positively shapeable future.
___________________________________________________________
A Kinder- und Jugendtheater Dortmund production as part of the Creative Europe project PlayOn! in collaboration with the Academy for Theatre and Digitality

RESEARCH PLATFORM

Academy for Theatre and Digitality

The Future

Nils Corte Roman Senkl Annette Müller

Sound Design Spatial Design Theater Dortmund
completedPlayOn!

Similar Projects

The Future

Nils CorteRoman SenklAnnette Müller

Sound DesignSpatial DesignTheater DortmundcompletedPlayOn!

Similar Projects

Nils Corte Roman Senkl Annette Müller

Sound Design Spatial Design Theater Dortmund
completedPlayOn!