6. Musimap’s Emotional AI: – Decoding the DNA of Music by combining human emotions and AI to gain insight into music consumption. In conversation with Thomas Lidy, Chief Innovation Officer Musimap.

6. Musimap’s Emotional AI: – Decoding the DNA of Music by combining human emotions and AI to gain insight into music consumption. In conversation with Thomas Lidy, Chief Innovation Officer Musimap.

Musimap | Emotional AI | DNA Music | Music Consumption | FuturePulse

Episode 6: In conversation with Thomas Lidy, Chief Innovation Officer Musimap.

Musimap's Emotional AI:
- Decoding the DNA of music by combining human emotions and AI to gain insight into music consumption.

Support by listening, subscribing and sponsoring!

My guest today is my friend Thomas Lidy. We met in Athens 2018, when I was involved in the European Horizon 2020 research project FuturePulse through Soundtrack Your Brand. Thomas is from Vienna Austria and has been active in semantic audio analysis and AI-based music recognition since 2004. He has a Master of Science and has pursued a PhD in computer science (which he has not finished, though) from Vienna University of Technology, where he also started his research career, focusing on Music Information Retrieval.

Previously, Thomas was the founder and CEO of the music discovery startup Spectralmind. Today, he is the Chief Innovation Officer at Musimap, a company combining human knowledge and AI to gain insight into music and music consumption. Musimap owns one of the largest annotated music databases in the world.

Together with his team of musicologists and data scientists, he has created an emotion-aware music AI that is able to analysis both music and people based on their emotional signature in order to enhance music recommendation and enable completely new branding opportunities with a deep link to people's music preferences. Thomas is the creator and co-host of the successful Vienna Deep Learning Meetup, He has also co-organized AI Summit Vienna 2017 and has published over 40 publications in the field of music analysis & AI. In this episode, Thomas and I discuss the ins and outs of Emotional AI and music analysis and recommendation.


“Our goal is to becoming the “Ultimate Music Assistant” by decoding the DNA of music.”.
– Thomas Lidy

Musimap | Emotional AI | DNA Music | Music Consumption | FuturePulse
Play Video

Full version vlog episode

Musimap | Emotional AI | DNA Music | Music Consumption | FuturePulse

Soundbite #6.1
How Thomas Lidy’s combined passion for computers and music brought him to become one of the world's top experts in music AI, revolutionizing the music-tech industry. (03:42 ) 

Soundbite #6.2 
How FuturePulse is empowering the music industry with predictive analytics on artists, tracks, playlists and genres with just a few clicks. (05:30) 

Soundbite #6.3  
Semantic audio analysis and music recognition explained. (13:25) 

Soundbite #6.4 
Bridging the gap between academia and industry: Vienna University of Technology -> Music Information Retrieval Research -> Spectralmind -> #Music Bricks -> Musimap. (18:22) 

Soundbite #6.5 
The successful foundation of the Belgian B2B music AI company Musimap is backed by 20 years of human research combined with audio-processing and AI.(23:40) 

Soundbite #6.6  
Becoming the “Ultimate Music Assistant” by scientifically decoding the DNA of music. (26:26) 

Soundbite #6. 7 
A“humanized” algorithm = a human fused AI + feedback loops. (34:19) 

Soundbite #6. 8 
Is a universal music taxonomy possible? (38:57) 

Soundbite #6. 9 
MusiMe: You are what you listen to. Your Spotify playlists reveal your personality. Try it out. https://yawylt.musimap.io/(45:13) 

Soundbite #6. 10 
How Emotional Artificial Intelligence Music is revolutionizing audio branding, e-commerce and the dating industry. (54:11)

Soundbite #6.1

Soundbite #6.2

Soundbite #6.3

Soundbite #6.4

Soundbite #6.5

Soundbite #6.6

Soundbite #6.7

Soundbite #6.8

Soundbite #6.9

Soundbite #6.10

MusiMe: You are what you listen to​

Quincy Jones and Musimap to refashion music consumption with the psychographic profiling engine of MusiMe …

Quincy Jones’ songs appear as conveying good vibrations, with an intent to share, with energy and enthusiasm, enclosing positivity, freedom, comfort and care (nourishment). They are imprinted with love, with important seductive & sensuality dimensions, romantic with a touch of femininity, with a hint of spirituality and harmony. They contain a bit of extroversion, self-confidence and a somewhat flashy attitude. They have a strong anchorage (roots), are organic and slightly tribal.

MusiMe classifies Quincy Jones as an « Entertainer, Performer » following the MBTI model. He is of the ESFP type, with a focus on extroversion; sensing, feeling and perceiving the world intensely. His way of facing life is « Demonstrative ». His « temperament Family » is the one of the « Tactical Artisan » kind, with a strength on « Sensation and Perception ». According to the results, Quincy Jones permanently seeks the joy of living, is people-oriented, appreciates a lot materialistic comfort, has a constant need for stimulation and pursues out of the common experiences. He is a seducer, wants to be admired by others and can enjoy spending and showing off from time to time. He is a generous person, likely to offer gifts to his partners. He has absolutely no desire to control others but needs to feel free as well. He is a very good listener and wants to be surrounded with people who resemble him. On the weakness side, Musimap’s engine points out that he might be procrastinating at times, with a lack of focus and that he is hyperactive and reckless. 

This is what you listen to (related to MusiMotion): https://tiwylt.musimap.com

You are what you listen to (related to MusiMe): https://yawylt.musimap.io 

Musimap Mood taxonomy: https://moods.musimap.net 

FuturePulse

Research by Thomas Lidy

Evaluation of feature extractors and psycho-acoustic transformations for music genre classification
T Lidy, A Rauber
ISMIR, 34-41
291 2005
Structured visual markers for indoor pathfinding
M Kalkusch, T Lidy, N Knapp, G Reitmayr, H Kaufmann, D Schmalstieg
The First IEEE International Workshop Agumented Reality Toolkit,, 8 pp.
156 2002
Experimenting with musically motivated convolutional neural networks
J Pons, T Lidy, X Serra
2016 14th international workshop on content-based multimedia indexing (CBMI …
114 2016
Improving Genre Classification by Combination of Audio and Symbolic Descriptors Using a Transcription Systems.
T Lidy, A Rauber, A Pertusa, JMI Quereda
ISMIR, 61-66
108 2007
CQT-based convolutional neural networks for audio scene classification
T Lidy, A Schindler
Proceedings of the detection and classification of acoustic scenes and …
93 2016
On the suitability of state-of-the-art music information retrieval methods for analyzing, categorizing and accessing non-western and ethnic music collections
T Lidy, CN Silla Jr, O Cornelis, F Gouyon, A Rauber, CAA Kaestner, ...
Signal Processing 90 (4), 1032-1048
71 2010
Automatic audio segmentation: Segment boundary and structure detection in popular music
E Peiszer, T Lidy, A Rauber
Proceedings of the International Workshop on Learning the Semantics of Audio …
42 2008
DelosDLMS-the integrated DELOS digital library management system
M Agosti, S Berretti, G Brettlecker, A Del Bimbo, N Ferro, N Fuhr, D Keim, ...
International DELOS Conference, 36-45
34 2007
Parallel convolutional neural networks for music genre and mood classification
T Lidy, A Schindler
MIREX2016
32 2016
Combining audio and symbolic descriptors for music classification from audio
T Lidy, A Rauber, A Pertusa, J Inesta
Music Information Retrieval Information Exchange (MIREX)
30 2007
The map of mozart
R Mayer, T Lidy, A Rauber
na
24 2006
A cartesian ensemble of feature subspace classifiers for music categorization
T Lidy, R Mayer, A Rauber, PJ Ponce de León Amador, A Pertusa, ...
International Society for Music Information Retrieval
23 2010
Comparing Shallow versus Deep Neural Network Architectures for Automatic Music Genre Classification.
A Schindler, T Lidy, A Rauber
FMT, 17-21
22 2016
Automatic audio segmentation; segment boundary and structure detection in popular music
E Peiszer
 
22 2007
A multi-modal deep neural network approach to bird-song identification
B Fazeka, A Schindler, T Lidy, A Rauber
arXiv preprint arXiv:1811.04448
21 2018
Content-based organization of digital audio collections
R Neumayer, T Lidy, A Rauber
na
21 2005
Method and system to organize and visualize media
T Lidy, W Jochum, E Peiszer
US Patent App. 13/808,484
18 2013
Evaluation of new audio features and their utilization in novel music retrieval applications
T Lidy
na
16 2006
Decision manifolds—a supervised learning algorithm based on self-organization
G Polzlbauer, T Lidy, A Rauber
IEEE Transactions on Neural Networks 19 (9), 1518-1530
15 2008
Spectral convolutional neural network for music classification
T Lidy
Music information retrieval evaluation eX-change (MIREX), Malaga, Spain
14 2015
Music information technology and professional stakeholder audiences: Mind the adoption gap
C Liem, A Rauber, T Lidy, R Lewis, C Raphael, JD Reiss, T Crawford, ...
Dagstuhl follow-ups 3
13 2012
Sound Re-synthesis from rhythm Pattern Features-audible insight into a Music Feature Extraction Process.
T Lidy, G Pölzlbauer, A Rauber
Ann Arbor, MI: Michigan Publishing, University of Michigan Library
13 2005
Combined fluctuation features for music genre classification
T Lidy, A Rauber
Music Information Retrieval Evaluation eXchange
13 2005
Ambient music experience in real and virtual worlds using audio similarity
J Frank, T Lidy, E Peiszer, R Genswaider, A Rauber
Proceedings of the 1st ACM international workshop on Semantic ambient media …
12 2008
Multi-temporal resolution convolutional neural networks for acoustic scene classification
A Schindler, T Lidy, A Rauber
arXiv preprint arXiv:1811.04419
11 2018
Diego Milano, Moira Norrie, Paola Ranaldi, Andreas Rauber, Hans-Jörg Schek, Tobias Schreck, Heiko Schuldt, Beat Signer, Michael Springmann, DelosDLMS-the integrated DELOS …
M Agosti, S Berretti, G Brettlecker, A del Bimbo, N Ferro, N Fuhr, D Keim, ...
Proceedings of the 1st international conference on Digital libraries …
11 2007
Mirex audio genre classification
K West
Music Information Retrival Evaluation eXchange (MIREX)
11 2005
Map-based music interfaces for mobile devices
J Frank, T Lidy, P Hlavac, A Rauber
Proceedings of the 16th ACM international conference on Multimedia, 981-982
10 2008
Classification and clustering of music for novel music access applications
T Lidy, A Rauber
Machine Learning Techniques for Multimedia, 249-285
9 2008
Visually profiling radio stations
T Lidy, A Rauber
ISMIR, 186-191
9 2006
Testing supervised classifiers based on non-negative matrix factorization to musical instrument classification
E Benetos, C Kotropoulos, T Lidy, A Rauber
2006 14th European Signal Processing Conference, 1-5
8 2006
Genre-oriented organization of music collections using the somejb system: An analysis of rhythm patterns and other features
T Lidy, A Rauber
na
8 2003
Fashion and apparel classification using convolutional neural networks
A Schindler, T Lidy, S Karner, M Hecker
arXiv preprint arXiv:1811.04374
6 2018
Audio music classification using a combination of spectral, timbral, rhythmic, temporal and symbolic features
T Lidy, A Rauber, A Pertusa, PJ Ponce de León Amador, JM Iñesta
 
6 2008
Bringing Mobile Map-Based Access to Digital Audio to the End User
R Neumayer, J Frank, P Hlavac, T Lidy, A Rauber
14th International Conference of Image Analysis and Processing-Workshops …
6 2007
Mirex 2007 combining audio and symbolic descriptors for music classification from audio
T Lidy, A Rauber, A Pertusa, JM Inesta
MIREX 2007-Music Information Retrieval Evaluation eXchange
6 2007
Mirex 2005: Combined fluctuation features for music genre classification
T Lidy, A Rauber
Proceedings of the 6th Annual International Symposium on Music Information …
4 2005
Creating ambient music spaces in real and virtual worlds
J Frank, T Lidy, E Peiszer, R Genswaider, A Rauber
Multimedia Tools and Applications 44 (3), 449-468
3 2009
Music Information Retrieval
T Lidy, A Rauber
Handbook of Research on Digital Libraries: Design, Development, and Impact …
3 2009
Computing statistical spectrum descriptors for audio music similarity and retrieval
T Lidy
Third Music Information Retrieval Evaluation eXchange (MIREX), 2006
3 2006
Marsyas and rhythm patterns: Evaluation of two music genre classification systems
T Lidy
Proc. Workshop Data Anal.(June 2003)
3 2003
Multi-temporal resolution convolutional neural networks for the DCASE acoustic scene classification task
A Schindler, T Lidy, A Rauber
Detection and Classification of Acoustic Scenes and Events
2 2017
A synthetic 3d multimedia environment
R Genswaider, H Berger, M Dittenbach, A Pesenhofer, D Merkl, A Rauber, ...
Computational Intelligence in Multimedia Processing: Recent Advances, 79-98
2 2008
Mirex 2006: Computing statistical spectrum descriptors for audio music similarity and retrieval
T Lidy, A Rauber
na
2 2006
Deep Learning for MIR Tutorial
A Schindler, T Lidy, S Böck
arXiv preprint arXiv:2001.05266
1 2020
MusicBricks: Connecting digital Creators to the Internet of Music Things
T Lidy, A Schindler, M Magas
ERCIM NEWS 101, 39-40
1 2015
MIREX 2009 enhancing audio classification with template features and postprocessing existing audio descriptors
A Grecu, T Lidy, A Rauber
 
1 2009
MIREX 2009 spectral and rhythm audio features for music similarity retrieval
T Lidy, A Rauber
Music Information Retrieval Evaluation Exchange (MIREX'09)
1 2009
Natural/Novel User Interfaces for Mobile Devices
S Siltanen, C Woodward, S Valli, P Honkamaa, A Rauber, J Frank, T Lidy, ...
Multimodal Processing and Interaction-Audio, Video, Text
1 2008
Decision manifolds: Classification inspired by self-organization
G Pölzlbauer, T Lidy, A Rauber
International Workshop on Self-Organizing Maps: Proceedings (2007)
1 2007
Experimenting with musically motivated convolutional neural networks
J Pons Puig, T Lidy, X Serra
14th International Workshop on Content-Based Multimedia Indexing (CBMI …
  2016
Klingende Bausteine für die Industrie; Ein Projekt ebnet der Musiktechnologie den Weg in den Markt
T Lidy, A Schindler
OCG Journal
  2015
" Offizielle" elektronische Version der Publikation (entsprechend ihrem Digital Object Identifier-DOI)
M Zeppelzauer, D Mitrovic, C Breiteneder
EURASIP Journal on Image and Video Processing 46 (2013)
  2013
SONARFLOW-VISUAL MUSIC EXPLORATION & DISCOVERY
T Lidy
 
  2010
" Offizielle" elektronische Version der Publikation (entsprechend ihrem Digital Object Identifier-DOI)
I Krikidis, J Thompson, S McLaughlin, N Görtz
IEEE Transactions on Wireless Communications 8 (6), 3016-3027
  2009
Shaping 3D multimedia environments: the MediaSquare
M Dittenbach, H Berger, R Genswaider, A Pesenhofer, A Rauber, T Lidy, ...
Proceedings of the 6th ACM international conference on Image and video …
  2007
Un metodo basato su DSP per l'enhancement dei transienti nella musica registrata
M Magrini, T Giunti, G Bertini, A Rauber, T Lidy, ICNR Pisa
na
  2006
Maps of music
A Rauber, T Lidy, R Neumayer
na
  2005
Phases 2 and 3 of Project Spamabwehr: SMTP Based Concepts and Cost-Profit Models.
R Neumayer, A Rauber, W Gansterer, A Janecek, R Neumayer, ...
Proceedings of the 29th European Conference on Information Retrieval ({ECIR …
  2005
" Decision Manifolds: Classification Inspired by Self-Organization"; Vortrag: International Workshop on Self-Organizing Maps (WSOM'07), Bielefeld, Germany; 03.09. 2007-06.09 …
G Pölzlbauer, T Lidy, A Rauber
 
   
" Map of Mozart"; OCG Journal, 2/2006 (2006), 2; S. 12-13.
R Mayer, T Lidy, A Rauber
 
   
Buchbeiträge
M Magrini, T Giunti, G Bertini, A Rauber, T Lidy
 
   
" The Map of Mozart"; Poster: International Conference on Music Information Retrieval (ISMIR), Victoria, Kanada; 08.10. 2006-12.10. 2006; in:" Proceedings of the 7th …
R Mayer, T Lidy, A Rauber
 
   
TESTING SUPERVISED CLASSIFIERS BASED ON NON-NEGATIVE MATRIX FACTORIZATION TO MUSICAL INSTRUMENT CLASSIFICATION
EBC Kotropoulos, T Lidy, A Rauber
 
   
DelosDLMS&# 8722; the Integrated DELOS Digital Library Management System
M Agosti, S Berretti, G Brettlecker, A Del Bimbo, N Ferro, N Fuhr, N Keim, ...
 
   
Buchbeiträge
T Lidy, A Rauber
 
   
" Klingende Bausteine für die Industrie"; OCG Journal (eingeladen), 40 (2015), S. 17-18.
T Lidy, A Schindler
 
   
Buchbeiträge
M Kalkusch, T Lidy, M Knapp, G Reitmayr, H Kaufmann, D Schmalstieg
 
   
" Content-based Organization of Digital Audio Collections"; Vortrag: Open Workshop of MUSICNETWORK, Vienna, Austria; 04.07. 2005-05.07. 2005; in:" Proceedings of the 5th Open …
R Neumayer, T Lidy, A Rauber
 
   
EXTENDED ABSTRACT FOR MIREX 2005
T Lidy, A Rauber
 
   
Buchbeiträge
M Agosti, S Berretti, G Brettlecker, A Del Bimbo, N Ferro, N Fuhr, N Keim, ...
 
   
TU Vienna-IFS Participation in MIREX 2006
T Lidy, A Rauber
Report on Benchmark-Based Evaluations, 13
   
" Un metodo basato su DSP per l'enhancement dei transienti nella musica registrata (A DSP-based method for transient restoration of recorded music)"; Vortrag: DSP Application …
M Magrini, T Giunti, G Bertini, A Rauber, T Lidy
 
   
" MusicBricks: Connecting Digital Creators to the Internet of Music Things"; ERCIM NEWS (eingeladen), 101 (2015), S. 39-40.
T Lidy, A Schindler, M Magas
 
   
SUBMITTED TO VMDL07 AND ICIAP07
R Neumayer, J Frank, P Hlavac, T Lidy, A Rauber
 
   
Buchbeiträge
R Genswaider, H Berger, M Dittenbach, A Pesenhofer, W Merkl, A Rauber, ...
 
   
MIREX 2006 Audio Music Similarity and Retrieval Submission (1st version)
T Lidy, A Rauber
MIREX 2006, 14
   
Wissenschaftliche Berichte
T Lidy, P van der Linden
 
   
" Comparing shallow versus deep neural network architectures for automatic music genre classification"; Vortrag: 9th Forum Media Technology (FMT2016), St. Pölten, Austria; 23 …
A Schindler, T Lidy, A Rauber
 
   
VMDL 2007 Workshop Papers
E Chang, R Neumayer, J Frank, P Hlavac, T Lidy, A Rauber, N Orio, ...
 
   
Buchbeiträge
S Siltanen, C Woodward, S Valli, P Honkamaa, A Rauber, J Frank, T Lidy, ...
 
   
MIREX 2007 COMBINING AUDIO AND SYMBOLIC DESCRIPTORS FOR AUDIO MUSIC SIMILARITY AND RETRIEVAL
T Lidy, A Rauber, A Pertusa, JM Inesta

Read more

Music is one of the fastest evolving media industries, currently undergoing a transformation at the nexus of music streaming, social media and convergence technologies. As a result, the music industry has become a mixed economy of diverse consumer channels and revenue streams, as well as disruptive innovations based on new services and content distribution models. In this setting, music companies encounter daunting challenges in dealing successfully with the transition to the new field that is shaped by streaming music, social media and media convergence. The availability of huge music catalogues and choices has rendered the problems of recommendation and discovery as key in the competition for audience, while the continuous access to multiple sources of music consumption have resulted in a dynamic audience, characterized by a highly diverse set of tastes and volatility in preferences which also depend on the context of music consumption.

To serve the increasingly complex needs of the music ecosystem, FuturePulse will develop and pilot test a novel, close to market music platform in three high-impact use cases:

  • Record Labels
  • Live Music
  • Online Music Platforms

The project will help music companies leverage a variety of music data and content, ranging from broadcasters (TV, radio) and music streaming data, to sales statistics and streams of music-focused social media discussions, interactions and content, through sophisticated analytics and predictive modelling services to make highly informed business decisionsto better understand their audience and the music trends of the future, and ultimately to make music distribution more effective and profitableFuturePulse will offer these capabilities over a user-friendly, highly intuitive and visual web solution that will enable the immersion of music professionals in the realm of music data, and will support them to make highly informed and effective business decisions.

Get in touch

#audiobranding #sonicbranding #thepowerofaudioscienceai #musictagging #audio #music #sound #science #musictech #ai #jasminemoradi #musimap #sounddesign #AI #b2bmusic #thomaslidy #musicresearch #audioroi