Best Practices for Business Intelligence and Predictive Analytics

By Ali Rahim(2012)
Knowing what people will do before they do it is a useful skill. Thats why Predictive Analytics
has grabbed the attention of the Business Intelligence (BI) world. Furthermore, advances in
statistical technology and computing power have made it possible to significantly improve
Predictive Analytics.
Predictive Analytics is a data-driven extension that complements other BI capabilities, such as
querying and reporting, OLAP and data visualization. Predictive Analytics provides a synergy of
technologies that let users analyze past and current performance to make predictions. The
primary benefit of using Predictive Analytics to guess the future is that it gives you the ability to
take appropriate actionable steps.
Combining BI with Predictive Analytics, in what is becoming known as Business Intelligence +
Predictive Analytics, can produce new levels of insight that werent possible before, but
successful implementation requires establishing a repeatable process. With that in mind, I am
going to share some best practices that we have derived from current and past BI + Predictive
Analytics implementations.
Following are a few best practices for BI + Predictive Analytics implementations (with a stronger
concentration on Predictive Analytics):
1. Dual Projects
To start, consider the Predictive Analytics and BI pieces as two projects, a notion that will be
relevant through the rest of this article. However, its important to not consider them as two
separate projects. Both share dependencies on each other, though the BI project encompasses the
Predictive Analytics piece.
2. Objective
Make sure you know the objective of the modeling project. Not knowing what direction you
want to go is a recipe for failure, with failure in this case being incorrect modeling output,
missing deadlines, and veering outside of project scope and budget. You must define the business
use case and what the goal is.
3. Criteria for Success and Failure
Know the criteria that define success and failure, which is vital to understanding the success of
the projects and knowing when to go back to previous steps.
4. Vision
Document the projects vision: Lock in the scope and get buy-in from stakeholders. Buy-in is
important for any project, but with Predictive Analytics, this means committing stakeholders to
aggressively applying results to decision making. This is needed at all business levels.
5. Methodology
Select a methodology. This is a requirement for both BI and Predictive Analytics projects. Keep
in mind the methodology differs from BI to Predictive Analytics projects
(RUP/Agile/Waterfall/etc. for BI and CRISP-DM for Predictive Analytics). Make sure you know
the data, as well as relevant internal and external factors.
 The CRISP-DM methodology (as shown on the diagram) is recommended for Predictive
Analytics and is summarized in the following six steps. Keep in mind a majority of the time is
spent in the first three steps. This is an iterative process.
1. Business Understanding: Once the vision/requirements are done, this should be clear.
This is a part of the BI Project as well. There may be a need for further elaboration during the
Predictive Analytics project.
2. Data Understanding: Not knowing the data is a setup for failure.
3. Data Preparation: Data specification, data cleaning and variable transformation are all a
part of this step.
4. Modeling: This is where the model is created.
5. Evaluation: This is where outputs are validated on testing data.
6. Deployment: This entails using the models in applications. It is a part of the BI project.
Consider the Predictive Analytics project complete when it is ready to deploy. You may need to
return back to the Predictive Analytics for tweaking purposes.
Here is a breakdown of CRISP-DM with tasks and roles for each iteration:
Now, here is a sample of a summarized project plan to show how a BI + Predictive Analytics
project would work. It does not account for the iterative process:

Figure 1 Sample Project Plan for BI + Predictive Analytics Project timeline shown is for
demonstration purposes
6. Model Selection
Always select models first. Get assistance from experienced data mining practitioners. If they are
not already on board, bring them in before the modeling (Predictive Analytics) project, so that
they can guide in the data collection, model selection, etc.
Knowing what models to use allows for the proper sourcing of data. For example, Logistic
Regression technique requires a binary target, and Survival Analysis requires two targets (time
and status).
 7. Model Validation
Be sure to validate models, keeping the following in mind:
1. Use the predefined success and failures criteria to validate the model.
2. When models are ready to deploy, consider this a deliverable of the Predictive Analytics
Project.
3. Understand that each modeling technique may require data that is different from other
techniques.
4. Understand that each model output will differ from another model output.
5. Understand that each model output can be evaluated differently from another model
output.
6. Understand that there are multiple approaches to solving the same problem.
7. Benchmarking modeling output is difficult; it varies by industry, the data used, the values
used, the techniques used and the business cases. Sometimes there are no benchmarks, since the
business case and solution have not been done before.
8. Dont try to do too much with little. This means sometimes more than one model output
is needed to handle multiple business cases. Try not to lump them into one model.
8. Perspective
Its important to understand each step and keep in mind you are dealing with an iterative process.
Models do not come out perfect on the first try; in fact a perfect model should be questioned as
well as the data that was used to create this perfect model. Understand that having too much or
too little data can affect the model output accuracy. It is also not necessary to use all the data you
have; i.e., all N number of years of data.
9. Misconceptions
Avoid the following misconceptions about Predictive Analytics (from the Five Myths of
Predictive Analytics by Aryng):
1. Its new. It actually has been around since 1930 when Fisher and Durand created the first
credit score model.
2. Produces a perfect prediction. That depends on the data, and models are estimates.
3. Push-button solutions. Tools cannot provide everything; mentors should select the
technique based on business context.
4. Build it and forget it. All models depend on the data that is provided. Data also has cutoff
time periods; for this reason, models can get outdated. A refresh is required but varies by
customers, industry and business case.
Success Factors
Here are three factors that lead to successful BI + Predictive Analytics implementations:
1. The availability of useful data that pertains to the business case. Sometimes there isnt
enough data
2. Knowledge of business domains and data
3. Experience
In summary, the keys to a successful BI + Predictive Analytics Implementations are
1. Data, data, data!
2. Process and documentation
3. Understanding the techniques to be used
4. Buy-in and commitment to apply the result
6 Marketing Challenges And The Shortcuts To Solve Them
By Steve Olenski(2014)

The rapid advance of technology presents a host of challenges for todays marketing
professional. Even when a marketer has found a strategy that works, innovation and competition
can cause a shift in the paradigm of their success.

These business challenges may be small hurdles or large obstacles, but a marketer can overcome
these challenges and find even greater success.

Theres Something In The Air

The improvement of indoor air quality has been a personal mission of Tom Lozanos as a result
of his own struggle with severe allergies. After installing a new indoor air technology system in
his own home, Lozano experienced life-changing relief from his debilitating allergy symptoms.

The dramatic change prompted Lozano to approach the manufacturers of the new technology and
eventually he, and his business partner Allen Brasington, took over national and international
sales and marketing of Air Scrubber Plus, launching the indoor air purification system nationally
last fall.

We faced a number of marketing challenges from the very start, says Lozano, we had to
communicate indoor air quality in a consumer-friendly way and convince HVAC vendors to
embrace a new technology that would displace many of their existing products.

Regardless of the size of the obstacle, with careful analysis and strategic planning, marketing
challenges can be used to your advantage, leading to greater clarity, creativity and customers.
And what marketer doesnt like a good challenge?

Face your biggest marketing challenges head-on with these shortcuts:

Challenge #1 Lead Generation: When youre struggling with lead generation, youre losing the
ability to attract future prospects or clients. A struggle with lead generation also means that
youre losing sales. This problem could be caused by ineffective measures used to reach and
engage your target audience.

Shortcut: Reassess which methods youre utilizing to reach your target audience. What are you
doing to reach this audience? Do you blog? Do you get social? Do you advertise? Do you count
on referrals? How are you reaching your audience? Using referrals to grow brand awareness is a
cost-effective solution that is one of the most effective methods used to generate new leads. If
youre not focusing on referral marketing, maybe now its time to get started.

Challenge #2 Target Audience: You know that you need to reach your target audience, but just
who is your target audience? If you feel as though youre struggling with the ability to find out
just who your target audience is, its time to find them.
Shortcut: Regardless of the size of your business, you need to understand the demographics of
your target audience. Start with geography. If youre a smaller business, you may focus on one or
multiple key counties, towns, or cities. For larger businesses, you may find yourself thinking
nationally or globally. Now that you know the location, you want to know the age, gender,
financial background, likes, dislikes, etc. Consider conducting surveys, polls, and census data.

Challenge #3 Content Marketing: How is your content marketing plan? Are people engaging
with your content? Is it reaching your target audience? Is it making a good first impression? If
not, then its time to reassess the quality of your content.

Shortcut: Take a look at the quality of your content and where its being shared. There are a lot
of ways to succeed at content marketing. A B2B company must take a different approach to
content than a B2C company. Once your content has been carefully crafted and curated, is your
messaging being communicated on the channels favored by your target audience?

Challenge #4 Social Media: Using social media to reach your target audience is more than just
sharing great content. Its also about making sure that you are using social media to share, not
just sell. Its making the time to sign in and actually connect with people. Sharing and engaging
with others is key. If youre not connecting with your audience, then youre failing in the social
media department, says Lozano.

Shortcut: Its time to make time for social media. Once youve found your target audience and
their favorite hangouts, its time to get engaged and to get engaged frequently. This doesnt have
to be every day, but it does need to be consistent. With 829 million daily active users on average
in June 2014 on Facebook, you may not be logging on every day, but many customers are. So
when youre not signing in, those are missed opportunities to humanize your brand and reach the
masses.

Challenge #5 Latest Marketing Trends: Times are changing and so is marketing, so its
important to stay up-to-date on the latest marketing trends. But where are some of the best spots
to do so?

Shortcut: Keep up-to-date on the latest marketing trends by following popular marketing blogs
and magazines, utilizing social media outlets to find out whats fresh and by attending webinars
hosted by powerful industry leaders.

Challenge #6 Increasing ROI: When youre ROI is high, youre generating leads and earning
profits. When your ROI is low, this is a problem. If you struggle with increasing your ROI, its
time for some action.
Top 10 Business Intelligence and Analytics Trends for 2016
By Agata Kwapien(2015)

As the year is approaching its end, its time to look towards the future. Were not foretelling the
following innovations are already in use. Check out what Business Intelligence trends will be on
everyones lips and keyboards in 2016.

1. Predictive and Prescriptive Analytics Tools

Business analytics of tomorrow is focused on the future and tries to answer the questions: What
will happen? How can we make it happen? Accordingly, predictive and prescriptive analytics are
by far the most discussed topics among the Business Intelligence professionals, as we could see
with our own eyes during this years TDWI Conference in Zrich.
Predictive analytics is the practice of extracting information from existing data sets in order to
forecast future probabilities. Its an extension of data mining which refers only to past data.
Predictive analytics includes estimated future data and therefore always includes the possibility
of errors from its definition. Predictive analytics indicates what might happen in the future with
an acceptable level of reliability, including a few alternative scenarios and risk assessment.
Applied to business, predictive analytics is used to analyze current data and historical facts in
order to better understand customers, products and partners and to identify potential risks and
opportunities for a company.
Industries harness predictive analytics in different ways. Airlines use it to decide how many
tickets to sell at each price for a flight. Hotels try to predict the number of guests they can expect
on any given night in order to adjust prices to maximize occupancy and increase revenue.
Marketers determine customer responses or purchases and set up cross-sell opportunities,
whereas bankers use it to generate a credit score the number generated by a predictive model
that incorporates all of the data relevant to asses persons creditworthiness.
Among different predicative analytics methods two attract recently the most publicity
Artificial Neural Networks (ANN) and Autoregressive Integrated Moving Average
(ARIMA). In neural networks data is being processed in a similar way like in biological
neurons. Technology duplicates biology information flows into the mathematical neuron, is
processed by it and the results flow out. This single process becomes a mathematical formula
that is repeated multiple times. As in the human brain, the power of neural networks lies in their
capability to connect sets of neurons together in layers and create a multidimensional network.
The input to the second layer is from the output of the first layer, and the situation repeats itself
with every layer. This procedure allows for capturing associations or discovering regularities
within a set of patterns with the considerable volume, number of variables or diversity of the
data.
ARIMA is a model used for time series analysis that applies data from the past to model the
existing data and make predictions about the future. The analysis includes inspection of the
autocorrelations comparing how the current data values depend on past values especially
choosing how many steps into the past should be taken into consideration when making
predictions. Each part of ARIMA takes care of different side of model creation autoregressive
part (AR) tries to estimate current value by considering the previous one. Any difference between
predicted data and real value are used by the moving average (MA) part. We can check if these
values are normal, random and stationary with constant variation. Any deviations in these
points can bring insight into the data series behavior, predicting new anomalies or helping to
discover underlying patterns not visible by bare eye. ARIMA techniques are complex and
drawing conclusions from the results may not be as straight forward as for more basic statistical
analysis approaches. But once the basic principles are grasped, the ARIMA provides very
powerful tool for predicative analysis.
Prescriptive analytics goes a step further into the future. It examines data or content to
determine what decisions should be made and what steps taken to achieve an intended goal. It is
characterized by techniques such as graph analysis, simulation, complex event processing, neural
networks, recommendation engines, heuristics, and machine learning. Prescriptive analytics tries
to see what the effect of future decisions will be in order to adjust the decisions before they are
actually made. This improves decision-making a lot as future outcomes are taken into
consideration in the prediction. Prescriptive analytics can help you optimize scheduling,
production, inventory and supply chain design to deliver what your customers want in the most
optimized way.
2. Visual Data Discovery
Big Data has reached the volume that is now insurmountable even for data scientists. When they
step inside the data, they dont know initially where it will lead. Often they begin their analysis
with visual data discovery to find patterns or structures in data sets that seem at first sight
impenetrable. With the use of different data visualization tools they try to discover relationships
between data elements across multiple data sets for subsequent data analysis. Thats the value of
visual data discovery you arrive at unexpected data insights, identified on the fly in real-time
and respond quickly and decisively to reduce risk, enhance profits or jump on short-lived
business opportunities.
Similarly to visual data discovery, explorational visual analytics tools allow you to dig into big
data with the use of visualizations and best practices in visual perception exploration. Such tools
support business agility and self-service BI through a variety of innovations that may include in-
memory processing and mashing of multiple data sources to inform your decisions.
Explorational visual analytics is based on experimentation, creativity and predefined questions,
visualizations are often created ad hoc to check different alternatives.
3. Intelligence as a Service
Self-service BI tools continue to be a blockbuster for many companies. Business professionals
take advantage of being empowered to explore new data sets without much IT support. Visual-
data-discovery tools have become synonymous with self-service BI and are growing in
popularity.
Business intelligence as a service (or BI-as-a-Service) solutions will still provide its users with
all three crucial functionalities: extracting and integrating data from various databases,
organizing it into a high-performance data warehouse or data lake, and giving business users
access to all this data and the possibility to process data through an easy-to-use interface.
But 2016 will take us a step further. Think about how much of human commerce is just about
knowing how to do something, you then realize that the next stage is to capture that knowledge
in a complex software system and sell it to anyone who needs it. Intelligence as a Service is
such a solution for the future you outsource the whole data analysis process to an expert.
Basically you purchase a package that will make decisions for you, taking into account all the
subtle variables that previously must have been analyzed by a team of data professionals to
guarantee correctness of data, compliance with rules, governmental or corporate, weather or
other external factors.
Intelligence as a Service vendors are already present on the market. An example of such
company is OPAL from Mannheim, Germany. By analyzing operative mass data OPAL can
prepare forecasts for how many fresh groceries a shop will need and thus prevent losses due to
deterioration. They developed a forecast module, the Demand Forecast Engine (DFE), which
provides precise sales projections by using complex mathematical algorithms (ARIMA), which
are automatically optimized at run-time. OPALs solutions ground on the so called native
implementation of In-Memory Technology. With this approach in the first place it is possible to
compute all requests and forecasts on a transactional level in real-time. Moreover, they
automatically integrate external factors like weather, calendrical global and local events into the
models of the DFE, so that you dont have to take care of it yourself. Founded in 2013, OPAL
with its operational analytics is definitely an example of a company of the future.
4. Innovative Data Virtualization
In the age of Big Data providing the right person with the right data in the right time poses a
major challenge. Especially as data is spread across different sources, formats, internal and
external locations. Hence, in 2016 companies will invest their efforts in data integration. New
tools will arise that will connect to each data set in its location and combine, blend, or join with
more agile tools and methods. This Business Intelligence trend is referred to as data
virtualization. Its the integration of any data in real-time from disparate, structured or
unstructured data sources, whether on premise or cloud, into coherent data services that support
business transactions, analytics, and other workload patterns. 2016 will be marked by agile data
integration approaches that see data as a virtual, abstract layer, independent of underlying
database systems, structures, and storage methods.
Data virtualization is the answer to many companies pain points. It helps to leverage all your
data and empowers your all your staff to access it. With better data insights you can respond
faster to emerging trends. Additionally, you skip the problem of data replication and
consolidation and thus your business runs more smoothly.
5. Graph Databases and Graph Analytics
As information has become more interconnected than ever, stemming from many sources and
often unstructured, the relationships between domains become as meaningful as the data itself.
Conventional relational databases may soon fall behind with scaling to the volume of big data or
processing an unstructured data search, especially in unexpected formats. Another problem lies
in the fact that in case of complex data sets, relational databases require us to use special
properties like foreign keys to infer connections between entities.
In 2016 graph databases will address these shortcomings and graph analytics will be
employed more often than ever before. Databases using graph structures for semantic queries
are already able to explore larger data fields, collecting and aggregating information from
millions of nodes and relationships, even while being armed only with a pattern and a set of
starting points. By assembling the simple abstractions of nodes and relationships into connected
structures, graph databases enable us to build sophisticated models that map closely to our
problem domain. Each node (entity or attribute) in the graph database model contains a list of
relationship-records that represent its relationships to other nodes. These relationship records are
organized by type and direction and may hold additional attributes. Whenever you run the
equivalent of a JOIN operation, the database just uses this list and has direct access to the
connected nodes, eliminating the need for an expensive search. This allows you to query and
view your data from any perspective and for any purpose.
In 2016 more and more people will decide to represent and store data in the form of graph
databases, and process it via graph analytics which is especially fit for data discovery we
mentioned before. When we start to analyze unstructured data and look for trends rather than for
an answer to a particular question, data discovery with graph analytics enables us to spot patterns
in data and relationships between certain data points, rather than arrive to a one-dimensional
answer. As patterns begin to emerge from multiple data sets, we start to gain a more complete
picture of everything that actually affects business outcomes, for example data from CRM,
logistics software, sales, weather, government, social media etc. We can see our data in context
and understand how they interrelate and impact our business.
6. Data Storytelling and Data Journalism
The recent years have witnessed a major shift from written to visual communication. The volume
of inflowing information increases, attention spans get shorter, and were used to jump from
headline to headline or from bullet point to bullet point rather than dig into the text. In order to
catch and retain our attention, journalists, or other professionals assigned with the task of passing
on information, turn to infographics. Thanks to its capability to communicate a complex set of
data on a single meaningful graph, data visualization is worth a 1000 words.
In 2016 the use of programming to gather and combine information will be an obvious necessity.
Moreover, the use of data visualizations will amplify, as more and more data presenters will
notice that its the attractive visuals rather than tables with numbers or paragraphs of text that
succeed at grabbing our attention.
7. IoT Data
Collecting of Internet of Things data is nothing new so you might wonder why we mention it
among the Business Intelligence Trends for 2016. The answer is simple the upcoming year will
bring the increase in the number of possibilities to actually make use of this data. The
aforementioned data analysis strategies, like visual data discovery and graph analytics, will allow
data scientists to leverage larger sets of unstructured data coming from multiple sources. Among
the most popular applications of IoT, insiders usually mention smart sustainable homes,
wearables, supply chains, farming or whole cities, where traffic management, water distribution,
waste management or urban security will depend on IoT data. Although this vision might still be
a bit far-fetched, in 2016 we will move a step further to the reality in which physical world and
computer-based systems will be interconnected and interdependent.
8. Cloud Analytics
The ubiquity of cloud is nothing new for anybody who stays up-to-date with Business
Intelligence trends. In 2016 the cloud will continue its reign with more and more companies
moving to the cloud as a result of the proliferation of cloud-based tools available on the market.
Moreover, entrepreneurs will learn how to embrace the power of cloud analytics, where most of
the elements data sources, data models, processing applications, computing power, analytic
models and data storage are located in the cloud. Examples of cloud analytics products and
services include hosted data warehouses, SaaS BI tools like datapine and cloud-based social
media analytics.
9. Democratization of Data Product Chain
In 2016, people will reach out for their business data to stay hands-on with all data handling
processes. Data will become democratized and available to people on various job positions
across organization. Todays dynamic market situation requires fast decisions made on the fly.
With new speed-of-thought analytical tools, users will analyze data quicker, mash it up with
other data and redesign it to create a new perspective and get more insight from their data. More
and more companies will boost their businesses with self-service data processing tools.
Rather than just consuming information, users will engage in data preparation and processing.
An obvious outcome of this information activism will be the growing use of visual data
discovery to explore not only business data, but also topics that interest people in their private
lives.
The emergence of the quantified-self movement is an example of this trend. People use
technology to acquire data on aspects of their daily lives in terms of inputs (e.g. food consumed,
quality of surrounding air), states (e.g. mood, arousal, blood oxygen levels), and performance
(mental and physical). By means of different wearable sensors, people eagerly subject
themselves to self-monitoring and self-sensing. As a result, data visualization is now becoming a
form of self-expression and a way to incorporate big data analytics into everyday life.
10. Bootstrapping
Bootstrapping is a versatile term that can be used in different contexts and will have a different
meaning for a programmer, an entrepreneur or a statistician. Bootstrapping as a Business
Intelligence trend of 2016 refers to yet another strategy of analyzing data that at the first sight
seem impenetrable. In general, bootstrapping is the process of building a complex analysis from
a simple starting point taking samples from the same data over and over again to estimate how
accurate your estimates about the entire data set are. The analysis is gradual and more advanced
with every step as the model learns from the previous data.

In 2016 big data will no longer be a flood but rather our natural environment as we will turn into
data natives. We will analyze complex data sets at work, learn from interactive infographics,
optimize our homes with smart appliances and measure our own performance in various spheres
of life with smart applications.
Applying Advanced Analytics in Consumer Companies
By Peter Breuer, Jessica Moulton, and Robert Turtle

Consumer-facing companies must be able to gather and manage the right data, apply analytics
that generate insights, and translate those insights into effective frontline action.

Retailers and consumer-packaged-goods (CPG) companies have long had access to vast amounts
of transaction data: every day, companies capture information about every SKU sold to every
customer at every store. In addition, companies regularly use sophisticated market-research
techniques to answer a variety of questions: what products should we develop and sell? How
much is the customer willing to pay? Which products should we discount and when? Which
marketing vehicles will allow us to reach the most customers?

Adding to the reams of data and market research already at their fingertips, consumer companies
now have access to social-media information and other large data sets known as big data. In this
article, we discuss the potential of big data and advanced analytics for the retail and CPG
industries and examine what it takes to turn this potential into actual value.

Immense possibilities

The combination of big data and advanced analytics offers retail and CPG companies countless
opportunities across the value chain. In portfolio strategy and product development, for example,
companies can get a more detailed understanding of consumer needs and attitudes and more
precisely identify consumer segments, improving their ability to target the highest-value
opportunities. They can measure the return on investment (ROI) for marketing spend across both
traditional and newer marketing vehicles (such as social media), allowing them to shift
marketing dollars to the most effective channels (see sidebar, Unraveling the Web: A new way
to understand the online customer). Through detailed hourly analysis of in-stock rates by store,
retailers can reduce out-of-stocks, provide a better shopping experience for consumers, and boost
sales for both themselves and their CPG partners.

Big data and advanced analytics allow companies to make better, faster decisions in their day-to-
day business and deliver improved performance. A European CPG company, for instance,
revolutionized its retailer-specific assortments and planograms: by applying advanced analytics
to consumer data, it was able to determine which SKUs were selling well in which retail formats
and which SKUs to swap in and out to best meet consumer preferences. It is now seeing 10
percent sales growth in a low-growth category. And the potential impact isnt just in sales: recent
research by McKinsey and Massachusetts Institute of Technology shows that companies that
inject big data and analytics into their operations outperform their peers by 5 percent in
productivity and 6 percent in profitability.

Making it happen: Three ingredients

We have found that fully exploiting data and analytics requires three capabilities. First,
companies must be able to choose the right data and manage multiple data sources. Second, they
need the capability to build advanced models that turn the data into insights. Third and most
critical, management must undertake a transformational-change program so that the insights
translate into effective action.

Managing the data

As they embark on a data-and-analytics journey, companies should take a decision back

approach that begins by crisply answering one question: which decisions do we want to
improve? A retailer may, for instance, seek to make better decisions about its promotional
spending. Here, the range of decisions can be quite broad: do we want to optimize the design
(number of pages, number of products on each page) of our promotional leaflets and circulars?
Do we want to reassess the distribution of our circularswhich newspapers they should be
inserted into, which addresses they should be delivered to, and so on? Do we want to rethink the
product mix in our circulars? Each decision requires different data and analyses.

Once theyve determined the specific business decisions they want to improve, consumer
companies must collect and manage the data needed to conduct insightful analysis. In our
experience, this entails three deliberate actions: creatively sourcing data, defining data-
governance standards, and establishing an IT infrastructure.

Creatively source data. Often, we find that a consumer company has the data it needs to unlock
business improvement, but the data reside in different business groups within the company. In
effect, organizational silos hide data from the analysts who can put it to work.

For example, a specialty retailer sought to redesign the promotional mailings it sends to the more
than five million members of its loyalty-card program. While the program had been successful,
revenue and profit from loyal customers was stagnating. The retailer recognized that by
combining data from various functional groupsin this case, customer loyalty, marketing, and
merchandising financeit could identify subsegments among its loyal customers and figure out
what types of direct mail resulted in the most incremental profit for each segment. Once the
retailer understood what data it needed, it developed a focused process to extract and clean the
needed data and bring it together for this new purpose. We find that such a targeted approach
often leads to success and supports quick wins.

Define data-governance standards. Given the vast amount of data that retail and CPG companies
collect and manage, ensuring the accuracy and reliability of data is a constant challenge. For
instance, product informationpackage sizes, product descriptions, or even the category in
which a product belongsisnt always up-to-date in retailers databases. If the data are needed
for analysis, the old saying garbage in, garbage out still largely applies.

The increasing value of analysis-ready data mandates a mind-set shift: in particular, a shift from
simply collecting the most easily available information and storing it as is, to establishing
rigorous data-governance standards that call for clean, consistent data to be collected and stored
in an analysis-ready state. The approach can be quite similar to the way retailers manage and
maintain stores: they have processes and checks in place to ensure that store assets are properly
cleaned, well designed, and periodically refurbished to maintain their useful life. In the same
way, data-governance standards should define what data will be collected, the processes for
checking and maintaining the data, and who in the organization is responsible for managing each
data set.

Establish the IT infrastructure. Companies need the IT infrastructure to store, easily access,
combine, and analyze large data sets. Setting up the IT infrastructure requires close collaboration
among business managers, analysts, and IT staff to ensure that data storage is designed in a way
that meets business needs and technical IT requirements.

One large retailer has taken big steps that improve data accessibility. The retailer recently
announced plans to integrate its five-plus e-commerce Web sites onto one platform and move the
corresponding data into a single in-house data repository. Whereas today sourcing of e-
commerce data requires coordination across groups and can take several weeks, the new
infrastructure will give the retailers business leaders instant transparency and access to e-
commerce data, accelerating insight generation and business impact.

Translating data into insights

Managing the data is just the first step. Companies must next make sense of the flood of dataa
task that requires sophisticated and sometimes complex analytic models. Two guiding principles
can help. First, business users should be involved in the model-building process; they must
understand the analytics and ensure that the model yields actionable results. Second, the
modeling approach should aim for the least complex model that will deliver the needed insights.

The power of these two steps became evident in a CPG companys efforts to assess its
marketing-spend ROI. An external vendor independently built a model that suggested online
paid-search ads delivered incredibly high ROI. The CPG companys marketing executives were
surprised but trusted the results, even though they didnt understand the model. Later, the
companys internal insights group partnered with the marketing function and built its own
simplified model. The results were markedly different: this model showed ROI for paid-search
ads to be 15 times lower than what the vendors model estimated. Further investigation revealed
that one of the external companys analysts, who had no knowledge of the business, had grouped
the data erroneouslya mistake that a business practitioner would have easily caught. Moreover,
the complexities of the vendors model hid the error from view.

Turning insights into effective frontline action

Gathering the right data and developing transparent models, however, wont yield impact unless
companies can also turn data-driven insights into effective action on the front line. Companies
must define new processes in a way that managers and frontline workers can readily understand
and adopt. When a large CPG manufacturer implemented a new demand-forecasting system that
generated automatic forecasts of each retail customers inventory needs, sales managers werent
told that the new process included shipping of safety stock to every customer. Worried that their
customers werent receiving adequate stock, sales managers regularly overwrote the automatic
forecasts. It took months before they changed their behavior and trusted the new system.