Sentiment and Quality Automation

Agent Introduction
Charleston 2/11/20

2
Quality Behaviors are the Foundation of a Great Experience
Sentiment is built on the S4X Behaviors that we know greatly impact customer experience when demonstrated effectively

What is Sentiment and Why is the benefit of evolving How does quality
Quality Automation? the way we assess quality? automation work?

Sentiment and Quality Automation
are machine learning models that
assess numerous factors, including
demonstration of behaviors, to
provide a score that is an indicator
of overall customer experience.
S4X Quality Interactions is the
foundation for Sentiment and
behaviors still matter. Sentiment
and Quality Automation give us the
ability to assess our customer
experience on a larger scale.
All customer interactions are
assessed using machine learning
models to produce indicators,
trends, and insights for coaching.
This helps us target areas of
opportunity for improvement.
Automation Provides Behavior Insights for Coaching
Sentiment and Quality Automation provide indicators across all interactions to help guide coaching and improve behavior demonstration

Current State – Manual Evaluations Future State - Quality Automation

Behavior and BPA scores on less than 1% Overall quality and behavior performance
of total interaction volume indicators across 90%+ of all interactions

Limited manual evaluations prevent Performance indicators have a strong

leaders from connecting results to correlation to tNPS surveys, JD Power Index
customer experience feedback results, S4X behaviors and Rep Sat

Variability of scoring in manual Consistent machine learning scoring and

evaluations between Ops and Quality behavior query details on all interactions

Performance trends, directional indicators

Feedback limited to 2-3 behaviors makes (i.e. above/ below the average) and real time
it challenging to identify trends examples provided in one location
Sentiment Monitoring
A sentiment score is produced for each voice interaction based on the monitoring of several factors related to customer experience

Nexidia Sentiment is machine learning built on Comcast tNPS surveys identifying themes and
characteristics of high & low performing NPS/Rep Sat interactions. It looks for several elements to produce a
score at the interaction level, that is a quality indicator aligned to customer experience.

SPOKEN WORDS LAUGHTER DETECTION PITCH & TONE

• Language models identify positive and negative • Indicates positive emotional state • Intensity
words and phrases No problem • Pervasive element • Pitch
• Positive phrases can this is ridiculous • Can prevent false positives • Jitter
off-set negative ones awesome and unnecessary alerts • Shimmer
I’m so annoyed
• Speaking rate
 Sentiment Measurement and Transition
What is Sentiment Transition and how do we measure it?

 All calls begin at 0 with sentiment measured at equal segments throughout the call.
 As the interaction progresses, the machine learning model will move sentiment up and down.
 The final score is an average of each segment.
 A Sentiment Transition score can show customer disposition at the end compared to how they came into the call.
Note: This information is only on 35% of calls.**

Sentiment score Customer & Agent

moves up and down Spoken Words & Agent & Customer Agent & Customer
based on: Phrases Pitch & Tone Laughter

0 Sentiment Score

Interaction End
Interaction Start

Segment 1 Segment 2 Segment 3 Segment 4

Score calculated Score calculated Score calculated Score calculated

*If there is not a large enough change the metric is not captured. Example bad call ends as a bad call it is not captured .
Sentiment Distribution and Difference
Sentiment scores are grouped into positive, neutral, and negative based on established scoring ranges

 Sentiment Distribution is the % of all interactions grouped into three categories; positive, negative, and neutral
 Individual interactions are categorized based on established thresholds tied to likelihood to recommend
 Sentiment Difference is the total % of positive interactions – the total % of negative interactions

Sentiment Distribution
positive neutral negative
26% 68% 6%

Sentiment Difference
Percent of Positive Calls Percent of Negative Calls Sentiment Difference
26% 6% 20%
Quality Automation – Warm & Friendly
Behavior-focused queries look for spoken words and phrases during the conversation
Partially Demonstrated

Unique words and phrases stand out when comparing Be Warm & Friendly partially demonstrated to exceptionally
demonstrated. The exceptional results show more personalized language and an attempt to make a genuine connection.
Exceptionally Demonstrated
Quality Automation – Behavioral Indicators
Machine Learning analysis evaluates each behavior individually and provides an indicator of agent level performance

 Results show percentages for the frequency when the behavior performance is demonstrated
effectively.
 Percentages are not an indication of performance above or below established behavior goals
Sentiment and Quality Automation Dashboard
Several key metrics are displayed in the current dashboards - below is an overview of each and why they matter

Non-talk Percentage represents the total non-talk Agent Non-Talk % scores can be Be Warm Friendly ratings decrease
against all sentiment calls in Nexidia compared against one another to as Non-Talk Time % increases
Time (~90% of recorded interactions) determine opportunities for efficiency

Sentiment Sentiment provides insight across ~90% of Higher Sentiment = Higher Rep Sat & Strong correlation between an
agent recorded interactions Likelihood to Recommend agent’s Rep Sat & tNPS performance
Distribution and their overall sentiment score

Count can be compared to overall call

Represents the number of calls that went Agents with a large call percentage
Sentiment
Sentiment Score from negative to positive or vice versa
volume to see if the agent is in general
in the neutral range you can identify
moving the conversations higher or
Transition (~35% of recorded interactions)
lower on the sentiment scale
the trend to help guide coaching

Behavior Behavior queries and machine learning Behavior strengths/ opportunities are an The higher the query % score for
provide insights on demonstrated aggregate of all calls and should be Positive queries the better an agent
Strength/ strengths and opportunities applied as a general coaching theme has historically performed on S4X &
Opportunity (~90% of recorded interactions) versus focusing on one call example aligned behaviors
Agent Overview Dashboard Review
Wrap Up and Action Planning
How do we get started using Sentiment to have deeper conversations about behavior demonstration?

 The Quality team will continue to evaluate individual interactions – 4 per agent per fiscal month
 Leaders will begin utilizing the Sentiment and Quality Automation information in coaching in September
 Leaders will review interactions associated with the Sentiment data and bring key observations to coaching
 Employee S4X performance will be reviewed at the end of the month
 Agents receiving an “achieves” or better will retain their performance for scorecard
 Agents below an “achieves” rating will be reviewed but scorecard will not be impacted

What can you do to get started?

Think about the behaviors Come to your next Ask lots of questions to
you could demonstrate coaching session ready to better understand current
more consistently or more learn more about your results and build a plan to
effectively performance indicators improve performance
QUESTIONS?