You are on page 1of 9

VOOP.

GLOBAL 21 JUNE 2018

AN ARTIFICIAL INTELLIGENCE APPROACH TO
MEASURING CULTURE: VOOP GLOBAL
CULTURAL INDEX

By: Kristin O’Brien, Founder, Voop.Global

Introduction
Organisational Culture has recently moved into the spotlight in Australia. In
the most part this has been due to the Royal Commission into misconduct in
Banking, Superannuation and Financial Services that is currently underway.
The APRA Inquiry into the Commonwealth Bank of Australia (CBA) (Broadbent
J et al, 2018) discusses Culture as a key contributor to the incidents that have
damaged the reputation and public standing of the bank. There has been
evidence of equally alarming behaviour at many other financial institutions,
and no doubt more will follow as the inquiry continues.

Culture is described by APRA as a system of shared values and norms that
shape behaviours and mindsets within an institution (Broadbent J et al, 2018).
An organisation’s Purpose and Values should be the scaffolding of an
organisations Culture, yet what we have seen in the banks is a significant
misalignment between espoused Values and the workforce’s actions and
behaviours.

1
VOOP.GLOBAL 21 JUNE 2018

An internet search of any of these organisations shows that they all espouse to
value Integrity, Service, Collaboration, Excellence or similar. We are now
aware that these Values were not the standards of the Banks mentioned. The
APRA report has demonstrated to us that for Culture, a lag measure is too late
with a tremendous loss of reputational value.

Australian Treasurer Scott Morrison has declared that the APRA report into
CBA is ‘a wake-up call’ for every Board Member and ‘goes to the heart of what
every Board Member does’ (McPherson C, 2018). Yet Boards are not positioned
or equipped with a lead indicator providing not only a clear line of sight to a
Culture at risk but predictive actions that can be taken well in advance.

How Culture is currently assessed
In many organisations Culture is measured using an Employee Engagement
Survey (sometimes called a Culture survey). These are excellent vehicles for
employees to provide feedback. Employees can share their view on systems and
processes, their leaders and whether they have what the need to do their job.
There is a direct link between employee engagement and productivity. In 2002
Harter, Schmidt, and Hayes demonstrated through their extensive study that
leaders adjusting their behaviour to create a more engaged workforce will result
in increased profitability.

Diagram 1. Lead vs Lag Indicators

2
VOOP.GLOBAL 21 JUNE 2018

There are some tools in the market that
take a Values based approach to assessing
Culture, which also provide insightful and
detailed reports, however, with Culture and
Engagement tools currently in the market,
they are lag, not lead measures. There is
an urgent need for a lead measure of
Culture that is available in real time as a
predictive measure rather than as a rear-
view indicator.

Cultural Index – creating
a global benchmark for
Culture
Artificial Intelligence(AI), or more Voop is an
specifically sentiment scanning and
natural language processing, can read and acronym for
summarise large volumes of written
‘Voice of Our
communication. To date, the main
application of this has been to understand People'
potential customers via their social media
profiles and then to adjust ones selling
style in response. Voop Global’s
hypothesis is that this technology can also
be used to determine organisational
culture.

3
VOOP.GLOBAL 21 JUNE 2018

Voop Global has conducted an experiment to test this hypothesis. Voop Global
has conducted the experiment in four organisations using a prototype. The
prototype uses AI to scan a series of publicly available documents either written
for, or about the test company, and measures the alignment with the company’s
espoused Values to create a cultural index. The results have been validated with
a Senior Leader from each of the organisations to test the accuracy of the tools
findings.

The prototype is called Voop Global, which is an acronym for ‘Voice of Our
People’. The word Global is important as this is a cloud-based AI tool that is
not constrained by geography, thus offering a global benchmark for Culture to
which Companies can hold themselves to account.

The Results of the Voop Global Experiment
All four companies that received a report using the Voop prototype validated its
accuracy and usefulness. In each of these companies the Senior Executive
consulted confirmed that the Voop findings matched the organisations internal
assessment of the current state of company Culture.

Diagram 2. Findings from Voop experiment in 4 test organisations

4
VOOP.GLOBAL 21 JUNE 2018

It was interesting that both of the Financial Services organisations in the
experiment presented the same findings (although the espoused Values of these
organisations are different). As the tool is further developed and the quantity
of data used in the analysis increases, it can be anticipated that the findings
will differentiate more.

Not static, real time feedback
It was clear in initially testing that Company D, a bank, had been impacted by
recent negative reports regarding Banking, in the Media. As the tool develops a
time line of data, Voop will show how key events such as ‘all company’
communications, ‘town hall’ meetings, company re-organisations and re-
structures impact organisational Culture (see example in Diagram 3). Tracking
the Cultural index over time will enable machine learning to inform Voop what
positively impacts the desired Culture and make recommendations relating to
this. It is anticipated that their will be unlikely correlations between
organisational activity and culture that have not been recognised before. Does
one campus lift being out of operation for a day have more impact on company
Culture than we realise?

Diagram 3. Voop Global is a real time longitudinal study of Culture

5
VOOP.GLOBAL 21 JUNE 2018

Transparency throughout
the organisation
The Executive from Company B, from the
Telecommunications industry, shared a
story of the CEOs confusion about results
of the recent engagement survey that
didn’t seem to align to their experience
having regular morning teas with different
groups of frontline staff. ‘Why is
engagement low when the energy on the
floor seems so positive’ the CEO was
quoted as saying. This could be attributed
to several things; the team behaving the
way the CEO desired when in their
company; the delay between when the
engagement survey was run and when the
CEO received the report; or employees not
completing the engagement survey
honestly.

Using AI to measure organisational culture
will create unprecedented transparency
where the mood or sentiment of the "Using AI to
organisation can be assessed daily. Using
AI and big data also mitigates the impact
measure
of the Hawthorne effect which is the organisational
alteration of behaviour by the subjects of a
study due to their awareness of being culture will
observed (Roethlisberger, F.J, 1939). create
unprecedented
transparency"

6
VOOP.GLOBAL 21 JUNE 2018

Not relying on a survey creates lead not lag
indicators and can’t be ‘gamed’
Every Executive interviewed to validate their company’s Cultural Index Result
talked about their experience completing Employee Engagement Culture surveys
and how using AI would mitigate the downsides of these tools. There was more
than one instance where surveys were ‘gamed’ to give a higher result than what
was accurate. When company bonuses are dependent on an engagement score
gateway, then the organisation ensures the results are high. Removing the need
to use Engagement scores as a Leadership measure also removes the need to
‘game’ this data source.

Some Executives also shared stories of changes in response to employee
feedback not filtering back to employees from six weeks to six months after the
survey was completed. The longer the delay in the results, the higher the
anecdotal survey fatigue. One of the companies in this experiment described
their HR department as ‘creating an industry to lift completion rates when their
time could be better spent on strategic matters’.

During this experiment, in summary, Voop Global reached the conclusion that
creating a Culture Index that operates separately to an employee feedback tool
will:
• provide a lead measure of Culture;
• mean that current employee feedback tools would not be ‘gamed’
• alert the organisation to a Culture at risk;
• provide transparency to Boards and CEOs; and
• create a feedback loop to Leaders.

Is reading employees work emails a breach of
privacy?
All the Executives interviewed as part of the experiment anticipated resistance
from employees about using AI to read employee’s emails that they send and
receive from their work email address. Each company that participated in the
experiment already had a policy in place to ensure employees knew and agreed

7
VOOP.GLOBAL 21 JUNE 2018

to their company accessing, monitoring or examining any data held in the
company’s possession. This would include email, messenger and any other form
of written communication.

The consensus from the sample of companies in this experiment was that some
employees would feel discomfort about a tool that used AI to read their written
communication and implementation of such a tool would require careful Change
Management highlighting the benefits not only to the organisation, but also to
employees.

Other applications
The Executive from Company A, within the Insurance Industry, described
Values as Important to their organisation. They are embedded in HR processes
such as recruitment, quarterly scorecards, performance reviews and ratings
which drive remuneration. When the Values are used as a benchmark for
assessment, it is up to an employee’s line manager to determine what the correct
rating is. The Executive from company B, a Bank, described the use of the
Values in their organisation the same way. This Leader has a national team
who they speak to on a weekly or fortnightly basis. They described assessing
Values as very difficult when leading a distributed team. To make the
assessment, they sometimes gather feedback from their stakeholders however
they queried the validity of this. Both company representatives identified the
prototype as a solution to the problem of not having accurate Values
assessments for performance reviews (which were the foundation of
remuneration reviews in their organisation).

Interest in Voop Global has also been generated from Health and Safety
professionals. A Health and Safety Manager from a global FMCG brand has
highlighted the requirements of the Safe Work Australia guide on Work-related
Psychological Health and Safety: ‘Organisations are required to acquire and
update knowledge of work-related psychological health and safety matters’.
There is potential for Voop to be designed to provide a lead measure of
psychological health and safety risks, thus reducing this risk which has both
human and commercial benefits.
8
VOOP.GLOBAL 21 JUNE 2018

The next iteration of Voop
The initial experiment has shown value in further testing of Voop Global. Next
steps will be to automate the parts of the first experiment that were done
manually. Rather than a person manually seeking publicly available
information for Voop to use, this feature will be built into Voop so it can
process thousands of articles at a time, thus increasing the granularity of the
data and the potential for insights.

Alongside this, Voop Global plans to test the tool on internal written
communication that is the every-day written ‘voice’ of a company’s workforce.
Voop is seeking partner organisations who would like to work with us to
conduct our next experiment.

Conclusion
This experiment has shown that there is a need for Boards and CEOs to have a
real time lead indicator of Culture to alert them to a Culture at risk, to create
transparency, and to create a feedback loop of impacts on Culture. Initial
testing of the prototype has shown that generating a Cultural Index is possible
without surveying employees. Further testing is required to obtain more
granular data and to validate the usefulness of this tool using internal data.

Voop is seeking 3-5 partner organisations to work with as we build and test the
next iteration of this prototype. We are seeking organisations of 100-1000
employees, or a business unit of this size within a larger organisation.

For more information or to contact us, please visit voop.global or email
Kristin@voop.global

References
Broadbent J, Laker J, Samual G, Prudential Inquiry into the Commonwealth Bank of
Australia (CBA) Final Report, 30 April 2018
McPherson C, Seven News, 30 April 2018
Harter, James K.,Schmidt, Frank L.,Hayes, Theodore L., Journal of Applied Psychology,
Vol 87(2), , 268-279, Business-unit-level relationship between employee satisfaction,
employee engagement, and business outcomes: A meta-analysis, April 2002
Roethlisberger, F.J. and Dickson, W.J., Harvard University Press, Management and the
Worker: An Account of a Research Program Conducted by the Western Electric Company,
Hawthorne Works, Chicago, 1939
9