Cool or creepy? A Computer Algorithm That Knows Your Interpersonal Stance From The Way You Speak
The role of emotions within Human-Computer Interaction (HCI) is gaining importance. Introducing emotions into computer applications typically makes these systems more efficient, effective and enjoyable. During the Fourth International Workshop on Multimodal Analyses enabling Artificial Agents in Human Interaction, Daniel Formolo and Tibor Bosse presented an application that is able to automatically extract someone’s ‘interpersonal stance’ (their interpersonal communication style, such as ‘aggressive’ or ‘cooperative’) from their vocal signals.
Take aways
- With current technology, it is possible to build computer algorithms that are able to automatically extract someone’s ‘interpersonal stance’ (for example, ‘aggressive’ or ‘cooperative’) from the way he or she speaks.
- This offers interesting possibilities for the development of voice-based computer applications that automatically adapt to the user’s interpersonal stance.
Study information
Who?
To build the algorithm, the researchers created a dataset of 3840 spoken sentences, spoken by 20 semi-professional actors (10 males and 10 females).
Where?
United States
How?
Participants were asked to record 24 sentences 8 times, where every time they had to use a different intonation, corresponding to the 8 categories of the Rose of Leary. The Rose of Leary is a psychological theory about styles of interpersonal communication.
The researchers developed the model using Support Vector Machines, a specific machine learning approach. They used two third of the data to train the algorithm and one third to test it. The researchers presented spoken sentences to the algorithm and then checked how well the algorithm categorized the interpersonal stance of the people who spoke the sentences.
Facts and findings
- The algorithm performed well when tested with sentences that were spoken by the same person that also trained the algorithm. However, the algorithm performed poor when the sentences were spoken by new people who were unknown to the algorithm.
- The algorithm recognized the interpersonal stance of individuals that were also part of the training in more than 80% of the time, compared to less than 20% of the time for new people.