Emotion & Theme Analysis of Call Recordings
60,000+ support calls transcribed and analysed into dashboards on emotion, theme and outcome
An AI framework for a health support call centre that transcribes call audio, then detects and summarises caller emotions, emotion transitions and themes, correlated against metadata to show where services could improve.
- Azure Cognitive Services
- Topic modelling
- Emotion detection
- Text classification
- Keyword extraction
- Python
- Pandas
- Power BI
Problem
A health support call centre held over 60,000 call recordings, 60,081 of them, from 2018 to 2021, for Cancer Council Victoria’s 13 11 20 information and support line. Everything useful in them (what callers were worried about, whether a call left someone better off than it found them) was locked in audio nobody had time to listen to. Service improvement decisions were being made on anecdote and the memory of whoever took the call.
Approach
Transcription was the enabling step, not the interesting one. The design question was what to measure once the text existed.
Sentiment at the call level turned out to be close to useless: nearly every call about a serious diagnosis starts negative. What mattered was the transition: whether a caller moved from distress toward reassurance over the course of the conversation. That reframing, from a static score to a trajectory, is what made the output actionable, and it became the basis of a best-paper-awarded publication.
On top of that:
- Thematic analysis through topic modelling and keyword extraction, to surface what callers actually raised rather than what the call categories assumed, cross-tabulated against cancer stage so a spike in financial or mental-health themes at, say, terminal stage was visible rather than averaged away.
- Per-emotion keyword extraction, so a service team could see not just that trust or disgust was elevated in a cohort but the actual phrases (“really supportive”, “shocked about diagnosis”) driving that score.
- Correlation against metadata: diagnosis type, stage, demographics and call length, so patterns could be traced to specific cohorts.
- Power BI dashboards as the delivery surface, because the audience was a service team, not analysts. A model nobody can interrogate changes nothing.
Working with health data meant the pipeline had to keep identifiable content inside approved infrastructure end to end, which shaped the tooling choices more than model performance did.
Result
A deployed AI framework plus a series of interactive dashboards detecting, analysing and summarising emotions, emotion transitions and themes across the full recording archive. Quarter-over-quarter trend modelling showed a consistent pattern: positive emotions (joy, trust, anticipation) trending up and negative ones (sadness, disgust) trending down over the service’s engagement with a caller, evidence that conversations with support staff were measurably improving how callers felt. The underlying method was published as An Artificial Intelligence Framework for the Detection of Emotion Transitions in Telehealth Services, which won best paper at IEEE HSI 2022, with a companion abstract at COSA’s 49th Annual Scientific Meeting.
What I’d do differently
Transcription quality varied more than expected across accents and call conditions, and the downstream emotion analysis inherited every one of those errors. I would measure word error rate by cohort before trusting any comparison between cohorts: an apparent difference in emotion between two groups can just be a difference in how well they were transcribed.