Pervasive Analytics

As data storage progressively has become more affordable so have data processing and complex
analytics tools. A retailer who used to consider forecasting demand at store level to be a pipe dream
does that as a matter of routine now. A casino chain whose idea of state-of-the-art analytics was
compensating front-line employees based on customer satisfaction rather than fixed hourly wages, is
today running fraud detection platforms that would make a banker proud.

Smart Organizations:

In the near future, analytics must be a part of all business decisions. An intelligent organization or smart
organization will be defined by the extent to which it uses its available data to make decisions and
therefore how analytically mature they are.

Today, static data structure constraining analytics and decision making is already a thing of the past. In
the future analytics will show certain key features:

1. There will be pervasive analytics and chief analytics officers. Almost a certainty in any company
that moves into the smart organization definition. The organization below the CAO will be the
analytics COE (center of excellence, driving and facilitating analytics in the organization)
2. Analytics will co-exist in descriptive, diagnostic, predictive and prescriptive forms. Based on
the business operation level and decision to be supported, and the data availability, analytics
complexity will vary. But all forms will exist in the enterprise.
3. Self service will be a key feature. Decision makers will not depend on assistants and analysts to
do all analysis. Dashboards will be interactive and will allow drill down, and could even be
embedded into workflows. In fact even statistical analysis will have business friendly user
interfaces so that, a marketer, for example, can run a segmentation without statistical help.
4. Just like Google makes the internet navigable by allowing us to quickly narrow in on what we
need from the vast resources of the internet, there will be intelligent exception management
systems that control the interaction with business decision makers so that the large extent of
data availability is precisely distilled to provide decision support without swamping the decision
maker with “insights”.
5. Streaming data analysis and real time intelligent decision making is going to be hygiene in
most organizations. Reaction time after the fact cannot be so long as to lose relevance with the
insight presented. Predictive analytics may provide insights to decisions needed to be taken
“before time” such as order placement, but real time analytics will enable corrections to those
decisions as necessary e.g. when an order is not fulfilled, due to a supply chain event.
6. Machine learning and Knowledge Discovery will see increasing adoption to allow analytics to
leap beyond human limitations of the user. The user’s imagination will not constrain the
patterns that can be identified and used for prediction and thence for decision making.
“Unsupervised” is the technical term; increasingly firms are looking to use unsupervised
techniques to unearth nuggets of information that help in estimates for a future event.

These features of analytics in the future were visible from the advent of big data and unstructured data
analytics itself. But the balancing forces of investments needed and the relative comfort of the existing
way of doing things, will mean organizations get to this future state gradually over time.
Current State of analytics in industry:

Today advanced analytics is visible in three different forms in business organizations.

1. Advanced analytics embedded into systems and processes e.g. organizations using Oracle’s
Demantra for forecasting are effectively using multiple forecasting models and auto-selecting
the most effective one for various demand categories. Demantra often is integrated with
Oracle’s EbS or R12 for forecasts to be utilized in business processes.
2. Advanced analytics used to answer critical business questions e.g. when a leading mid west
department store wanted to know whether moving into the furniture business has affected
other category sales unfavorably and to what extent, they turned to analytics to arrive at the
answer.
3. Advanced analytics used to un-earth “discoveries” from data e.g. pattern identification allowed
a retailer’s data to be analysed to predict forecast errors, thus enabling better exception
management in forecasts being used for ordering, etc. Certain repeat patterns in the demand
signals were indicative of oncoming variance between forecasted and actual values!

There are organizations at different levels of maturity in the context of readiness to adopt various levels
of analytics. Most organizations are at a level that they use advanced analytics for certain critical areas
such as cross-sell, pricing and forecasting but not for other areas. But that is quickly changing. In some
industries more than others, companies are embedding advanced analytics into their decision making
processes especially in areas such as fault identification, fraud analytics, marketing and e-business.
Some organizations are experimenting with using advanced analytics through lower cost trials typically
leveraging cloud and analytics-as-a-service arrangements. Fewer still are trusting pattern identification
and knowledge discovery from data.

So, the movement in the analytics journey will differ from organization to organization based on their
current maturity. However, in general, one can see the paradigm shifts in analytics complexity over time
as shown below:

Analytics paradigm Example Data Analysis

Price elasticity using
demand and price points/ indexing/ Crystalline, structured,
Current attributes stable Descriptive
Structured and
Intelligent pricing unstructured,
using elasticity after factoring amorphous, high velocity,
competitor data and reaction, high complexity and
Forward looking channel data, supplier media activity. volume Predictive
 Investigate what affects elasticity
itself, using DSR (demand signal
repositories). Multiple sources of
potential causal factors, inventory,
weather, square feet, adjacency,
store associates, parking availability,
freeway access, queue levels, etc. Predictive,
Impact price amongst other factors, non-linear -
after estimating competitor reaction, heavy use of
to favorably move demand and Exponential change in machine
Futuristic margins. volume and variety learning

Analytics Maturity Journeys:

There are two primary maturity journeys that organizations undertake in the context of analytics. One is
in terms of the level of analysis they leverage their data for. The other is the model of engagement they
adopt with analytics providers.

In adopting more advanced levels of analytics, there are still organizations grappling with, not whether
the business case exists, but whether their organization is ready to adopt the same. In simple terms, will
my teams tie themselves into knots with the rope of analytics rather than build a ladder. Therefore, we
see business leaders acknowledging the logic of analytics leading to intelligent decision making, but still
hesitating before trying advanced analytics.

The fear is well founded as well, because without the correct approach, analytics can be extremely mis-
leading. At the very least, it fails to prove business benefits and is relegated to something that is being
leveraged on the surface while decision makers still primarily use their instinct like they always have.
To derive sustainable advantage, the firm needs to make strategic investments in analytics. The journey
towards that realization and readiness is depicted above. The journey is not about internal process or
systems changes as much as it is about the success of the team that is internally “selling” the value of
analytics.

For example, for a retail leader, setting up an analytics COE is an investment option that has to prove its
ROI against that from opening a few more stores for market penetration or against a packaging plant for
private label products. It can therefore be expected that most business leaders will seek a journey
towards strategic investment in enterprise scale analytics rather than doing the same as a big bang
approach. Hence, a roadmap where low hanging analytics fruits can be harvested earlier to, in essence,
pay for itself, is an essential aspect of any analytics strategy.

Therefore, analytics will have to prove itself and the danger is real that analytics projects will stall or
become dormant, as many have in industry. The devil is in the details as we well know. It is a powerful
tool but one must know how best to leverage it, else it can cause more harm than benefit.

Challenges faced:

While the most significant hurdle faced today is in data readiness, we shall not focus on the same in this
section. There are also significant hurdles in organization readiness to leverage insights from analytics,
but that as well is not central to our discussion on analytics. The core problem of failed analytics
initiatives occurs when the correct confluence of technology, mathematics and business acumen is not
achieved.
Navigating around these pitfalls is key to having a successful data monetization experience. Small errors
in these areas of mis-alignment of technology, business and mathematics can lead to incorrect
recommendations. With extremely complex models today, it is difficult to “de-bug” an analysis. A given
input will typically produce an output, but the applicability of the same will depend on the validity of the
statistics behind the model in the particular business situation and judging that depends on the skill and
knowledge of the data scientist/ analyst.

Technology may cause a problem by way of constraints. For example, many retailers continue to cluster
their stores where the number of clusters is a user input, rather than the natural number of clusters
being derived from the data. This is typically causes because of a tool limitation. The same could be
mitigated by an astute statistician by running multiple clustering instances, however tools that have this
built in obviously provide an advantage.

Models obviously operate with certain fundamental statistical assumptions. For example, k-means
clustering treats all numeric columns as continuous variables. If a factor being used in this clustering is
not a continuous variable, k-means may not be the right technique to use. With large data sets, these
mistakes become more common.

Business acumen is of course of foremost importance in advanced analytics for business decisions. As an
example, a promotion ROI model that ignores a pull-forward effect will be erroneous. For many non-
perishable items a promotion only pre-pones demand i.e. later sales will decline to compensate for the
promotion lift. This effect reduces with increasing perishability and increasing impulsive nature of the
item. A pull forward effect for potato chips would be lesser than tins of cashew, for example.
Many organizations want to leverage big data for decision making but do not understand what that
truly means. The implicit aspects of flexible data structures, sustainable self-learning analytics, data
discovery, pre-processed analytics to allow real time decision support, the advanced “noise-
cancellation” needed and so on are not well grasped by many. The abilities to plan and provide all these
will come from a few organizations that have the necessary business understanding, statistical analytics,
analytics technology and data management skills.

While the analytics being discussed in the market today is around the buzz words of big data and
predictive analytics, we believe the difference will be brought about by those organizations who not
only provide big data based analytics but equally avoid the overload this can create and provide the
distillation required for this to be usable to businesses. Absence of a plan of how to prioritize,
automate, exception manage and make the insights usable will soon be seen to be the biggest lacunae
in a big data analytics plan.

The proposition:

Organizations that are able to leverage advanced analytics are those who can enable a confluence of
math & analytics, technology and business acumen, as mentioned earlier. The issue is about doing this
at scale. For which, the enterprise’ analytical capability has to be relevant to most fo their decision
makers. A manager of online business will not use data for decisions unless social and mobile data are
encompassed, a marketing manager will not use data for decisions unless the structure, culture and
incentives in the company propel him to do so.

Companies therefore need to adopt boundary-less data platforms and a progressive organization
structure and culture, to truly become smart organizations. In that context, we propose the following

Business leaders will value internal and external partners who can

1. Provide pervasive analytics

a. provide decision sciences rather than just data sciences, i.e. not just insights from data
but decisions from data.
b. provide analytics (analytics, and not simply reporting) on an industrial scale
c. provide ways to make the exponential growth in analytics manageable, either by a.
embedding into processes, b. extending into optimization rather than just analytics, and
c. enabling intelligent exception management

2. Provide self-service
a. provide visualization and depiction that is device independent and interactive
b. provide analysis not only as a service but also along with consulting, including judgment
on the extent to which descriptive and diagnostic analysis will be required vs. predictive
and prescriptive, to address a business question and support a decision

3. Provide real time analysis

a. provide big data based analytics to ensure unstructured data analysis as well as
structured
 b. provide pre-processed analytics with real time action triggers, and also provide
streaming or real time analytics

4. Provide data discovery and machine learning so that trends and patterns that are significant and
important for business decisions can be unearthed without specific questions being asked by the
decision maker

To provide pervasive analytics, the organization would need to enable decision makers in all their
interactions with stakeholders which would include suppliers, investors, regulatory bodies, employees,
partners/distributors and customers. Which leads one to therefore necessarily invest in analytics in all
functions:

1. Supply analytics
2. Operations analytics
3. Market and distribution analytics
4. Customer analytics
5. Compliance and risk analytics
6. HR analytics

The value proposition from the above functional range and complexity range together is evident in

1. Business value seen from the analytics initiative

2. Sustainability of the initiative
3. Internalization of the culture of being “smart” i.e. analytics driven
4. Business defense in the face of changing market conditions leveraging early warning of the same
5. Business expansion leveraging new product and market opportunities unearthed from data

Like any other new initiative (for example, like the wave of going online or mobile, that happened a
decade back), taking bold steps in analytics will generate returns but risks are there as well by virtue of it
being a relatively new space. To summarize, the winners will be the ones that leverage extreme volumes
of seamless data, funnel that to analytics including predictive analytics, that create usable insights, and
timely insights leveraging real time or streaming data analytics as needed.

These insights should be easily consumable through self-service portals, alerts and drill downs on mobile
devices. Insights should continuously improve through learning over time. And most importantly, the
technology prowess should exist to embed insights into optimization and automated transaction system
outputs, thereby into the business process.

Data may be the new oil, but for organizations that don’t watch their step, it’s slippery as well.