Artificial intelligence helps Europe’s musical treasure trove to open up to researchers and cultural enthusiasts
The aim of REPERTORIUM, which is an EU Horizon project, is to protect Europe’s valuable ancient musical roots. The project uses artificial intelligence and deep learning to identify and search music from different sheet music archives. When digitised, notations and manuscripts can be connected to various global databases and thus made available to professionals and enthusiasts.
Researchers and companies in eight European countries are working hard to make the musical treasure trove public and shared. In France, for example, approximately 2.200 hours of Gregorian chants will be recorded at the Abbey of Sainte Madeleine. Up to 3.000 arias from opera scores are digitised from a Spanish corpus.
At Tampere University, the Audio Research Group, led by Professor Tuomas Virtanen, is developing real-time sound source separation tools based on artificial intelligence. With the help of the tools, the sounds of individual instruments, for example, from a performance by a symphony orchestra, can be extracted and separated from each other in real time. Tools developed in the project, including tools for optical music recognition and sound field reconstruction, will become publicly available.
Classical music karaoke and 3D concert experiences
When each individual instrument of an orchestra can be isolated from the whole, a vast number of new opportunities for technological innovation will open up.
“We can conduct live streaming virtual symphony concerts and authentic acoustic spaces where the audience can experience an immersive 3D music experience. A sense of authenticity arises from the fact that the sound of an individual instrument is modified according to how the user navigates the virtual-reality performance space,” Virtanen says.
The technology also makes it possible to mute individual instruments or create a solo in real time. This is how the so-called karaoke versions of songs are created. “A music student can take advantage of this in their practice by playing or singing the muted part themselves,” says Postdoctoral Research Fellow David Diaz Guerra, who works in the project.
Before the REPERTORIUM project, the Audio Research Group has been pioneering for years on signal processing methods to separate different sound sources from each other. In addition to music, the methods have been applied e.g. to automatic speech recognition, to improve the sound quality of hearing aids, and to achieve privacy in acoustic monitoring.
Read more on the REPERTORIUM project website
Professor Tuomas Virtanen nominated as the Distinguished Lecturer of IEEE Signal Processing Society
Tuomas Virtanen has been nominated as a 2024 Distinguished Lecturer of the IEEE Signal Processing Society. The Distinguished Lecturer Program provides financial support for well-known educators and authors in the fields of signal processing to lecture at events of the society.
IEEE has over 400,000 members in 160 countries and is a leading authority on a wide variety of areas ranging from aerospace systems, computers and telecommunications to biomedical engineering, electric power and consumer electronics. The association publishes 30 percent of the world’s literature in the electrical and electronics engineering and computer science fields and has developed more than 1.300 active industry standards. The IEEE’s first society, the Signal Processing Society is the world’s premier professional society for signal processing scientists and professionals since 1948.
Read more about IEEE Read more about Signal Processing Society
Tuomas Virtanen
- Professor of Signal Processing at Tampere University
- Expertise: audio signal processing, computational acoustic scene analysis, and machine learning for audio.
- Has actively conducted research and teaching related to audio signal processing and machine learning for audio.
- Has developed sound event detection and source separation methods, which have numerous applications, for example, in context-aware devices, acoustic monitoring and assistive technologies.
- Has been able to open new avenues, for example, for improving the speech intelligibility in hearing aids, and noise-robust automatic speech recognition.
- Has been giving several keynote and plenary talks and tutorials at major international conferences, and many lectures at international universities and companies.
Photos: Mika Kanerva, Virpi Andersin
