Founder XMplify Consulting Ltd
Share this content

Listening to the mood music with emotion analytics

2nd Jul 2018
Founder XMplify Consulting Ltd
Share this content

Like many geeky teenagers in the 80's, I was fascinated by electronic music and especially bands like Kraftwerk that were developing the technology as well as the genre, and I ended up building a synthesizer in the vain hope of releasing my inner electro pop star; suffice to say that knowledge without talent (and practice) is not enough to top the charts (my teens predated the punk era).

One of the things I did learn was about sound 'envelopes'; specifically: Attack, Decay, Sustain and Release (Also known as ADSR). 

In a synthesizer, ADSR controls and modifies the amplitude (volume) of a sound during 4 distinct phases; after the initial stimulus (e.g. key press - attack), whilst the key remains pressed (decay and sustain) and what happens when the key is let go (release). It partly explains why a harp does not sound like a piano, even though both use strings. However, I squirrelled this knowledge away with no expectation of ever finding a use for it again.

ADSR Sound Envelope

So why think about it now? because I am working a lot with emotions and how to analyse and represent them in predictive computer models of behaviour.

Early in my research, I realised that I was going to have to find a way to take into account that emotions are not simply 'on or off' or maintain a continuous amplitude once the stimulus has passed. In some ways, the intensity of emotions is very like an ADSR envelope - with different envelopes for different emotions and in response to different stimuli (e.g. like the difference between striking a string and plucking it, a piano and a harp).

  • Attack phase - how quickly the intensity of an emotion changes in response to the initial stimulus;
  • Decay phase - how quickly it starts to subside, even though the stimulus remains in place;
  • Sustain phase - the lingering effect of a continuous stimulus; and finally
  • Release phase - the rate of return to an underlying normal state once the stimulus is removed.

Taking two core emotions as examples - the intensity envelope for surprise looks very different to the one for joy.

Emotions are not simply 'on or off' - they are far more complex and subtle than that.

To continue the analogy; the complex interactions of emotions is a little like playing chords - some pleasing, some discordant and jarring. Repeated stimuli can add to the overall sounds or distract. I could go on but you probably get the picture (or hear the tune) by now.

In my opinion, it is not enough to say that a customer feels 'happy' or 'sad', or even 'very happy' or a 'little sad' - if we are to get even close to analysing and modelling emotions over time (and I have), we are going to have to start listening to the music - especially if we ever hope to be able to play.

Replies (0)

Please login or register to join the discussion.

There are currently no replies, be the first to post a reply.