IIIT-H develops model linking music-induced movements to traits

Hyderabad, July 17

Just like physical gestures are quick giveaways of your personality and your current emotional state, the way you groove to music also says a lot about you.

In a new study that has significant implications for music cognition research, scientists at the International Institute of Information Technology, Hyderabad (IIITH) have developed a machine learning model that can look at listeners' natural movement to music and predict their personalities and cognitive styles.

"Natural swaying of the body and movement is a common response to music. And based on individuals' movements to music, we can enhance their listening experience and make recommendations on what kind of music they might like in the future," say Prof Petri Toiviainen, University of Jyvaskyla, Finland and Dr Vinoo Alluri, who leads music research at IIITH.

Previous music research has found an association between dance movements and personality traits especially in the extraversion and neuroticism dimensions.

For instance, those who scored high on the extraversion dimension tended to be more energetic and expressive, dominating the dance floor. Those who scored high on neuroticism exhibited jerky movements while remaining confined to a small area.

In the current study which has been accepted at the International Society for Music Information Retrieval Conference, the IIITH team automated a machine learning model to investigate the same.

The idea was to study music-induced movement patterns that could predict individual traits, which could then be linked to music preferences and recommendations.

For this, in a collaborative effort with the Department of Music, Arts and Culture, University of Jyvaskyla, Finland, the participants' personality and cognitive styles were assessed. The big five model was used where personality is ranked in terms of one's openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism.

Next, for cognitive styles, they were measured on the EQ-short and SQ-short questionnaires -- both instruments for measuring cognitive styles.

The EQ or the Empathising Quotient is the measure of an individual's drive to identify another person's emotions and thoughts and to respond to these with an appropriate emotion. SQ or the Systemising Quotient is a measure of an individual's drive to analyse or construct systems in a bid to understand the rules that govern them.

The participants were asked to move naturally to music selected across 8 different genres, ranging from the Blues to Hip-hop to Jazz, and Pop. With the help of markers placed at various joints of the body, these movements were then recorded via motion capture cameras.

"The novel part of the study is the end-to-end architecture, which uses natural movement to music to predict individual traits accurately, and further explore particular joints which are relatively important in characterising those traits," said Yudhik Agrawal, the first author of the study.

Apart from building a more personalised music recommendation system with these traits mapped to movement patterns, this study serves as an initial step in the direction of autism research.

"All participants in this study were healthy individuals. However, if we know that certain movements might be more typical to extreme systemizers, there are broader implications of this research in the domain of Autism Spectrum Disorder (ASD)," said Dr Alluri. —IANS