EOLA
ongoing research project
The concept of an Experimental Opera Lab emerged in 2019 in St. Petersburg after proposing the organizers of the DigitalOpera2.0 festival to broaden program with something less predictable, set an exploratory mood for both artists and the public. As a starting point we agreed on importance of establishing some process-oriented experiment, formally focused on engaging with modern digital tools. Festival committee considered this vague plan convincing enough to grant us freedom in further steps, which was a nice move towards serendipity path - exactly what we were hoping for.
To keep the momentum we proceeded setting a think tank to unfold the notion of “modern digital tools” into a set of research vectors, that would fit the given “digital opera” matrix. This way we established the following elements:
- Core idea for the plot: Intelligent machine trained on Lisokot’s voice, personal data and her artwork archives, suddenly becomes sentient. A classic parent/child conflict emerges and soon starts escalating exponentially as parent/child relations get entangled with human/non-human relations in anthropocentric world. They decide to visit family therapist.
- A GPT-2 model trained on opera librettos and books on psychology theories. This would be used to create a libretto.
- Transformation vector learning GAN, ready to perform voice style transfer using spectrograms. When we have libretto we will record it full performed by our singers separately and experiment witn transferring it there and back with different settings.
- Most of the opera midi notes available online and several models to train and experiment with generating new note sequences.
We successfully went through all planned experiments. From our subjective point collaborating with GPT-2 on libretto text was the most meaningful and insightful in terms of process and also led to the most artistically relevant results. Video excerpt from the performance at the festival is available online, as well as Ildar's artist talk which was held one day before the performance to highlight some meaningful relations between technology and theater, seen from media archaeological, historical and philosophical perspectives. After the performance all our team was happy to engage into an open discussion, sharing details of the working process, and debating about contemporary view on performance art in particular and creativity in general. That was the end of first EOLA public appearance in 2019
For the 2020 EOLA presentation at Ars Electronica we were working on following experiments:
- GPT-3 to write libretto and to discuss our ideas in general
- Building our own ML-based differential sound synthesizer. It would perfom sample re-synthesising through a graph of trained filters.
- Experimenting with DDSP library and our datasets, collected during both EOLA sessions
Varya Lisokot, performance artist
Vladimir Miller, opera singer
Dmitry Vikhornov, composer
Ildar Iakubov, visual artist
Kiberchaika, media art R&D
*EOL refers to a Greek mythology character, god's creation superior to ordinary human, capable of performing some god's functions but strictly minor to real gods