DATA

4 Analytics Concepts Every

Manager Should
Understand
by Amy Gallo
OCTOBER 31, 2018

JORG GREUEL/GETTY IMAGES

Like many professionals, my job doesn’t require expertise in data or analytics. I’m
a writer and editor, so I deal with words, not numbers. Still, nearly every
knowledge worker today needs to be a regular consumer of data analysis. For
example, I need to understand whether and why articles on having a mid-career
crisis outperformed ones on receiving feedback or why pieces with particular
headlines get more traffic than others.

I also need to be able to read research on the topics I cover and understand
whether the findings in those studies are valid and generalizable, and be able to
articulate the findings — and their limitations — to you, our readers.

To do all of this, I need a more-than-basic understanding of data analytics. And

while the statistics course I took in graduate school was helpful, it didn’t fully
equip me to grasp the important concepts and have the conversations I need to
around data analysis.

INSIGHT CENTER Fortunately, I had the opportunity to talk

with some of the best experts in the field
Scaling Your Team’s Data Skills
— Tom Redman, author of Data Driven:
Help your employees be more data-savvy.
Profiting from Your Most Important
Business Asset, and Kaiser Fung, who
founded the applied analytics program at
Columbia University — about several critical topics when it comes to data analysis.
Here are four refreshers from our archives on data analytics concepts that every
manager should understand.

Randomized controlled experiments

One of the first steps in any analysis is data gathering. This often happens via a
spectrum of experiments that companies do — from quick, informal surveys, to
pilot studies, field experiments, and lab research. One of the more structured types
is the randomized controlled experiment. Many people, when they hear this term,
immediately think of costly clinical trials but randomized controlled experiments
don’t have to be costly or time consuming and they can be used to gather data on
things like whether a particular customer service intervention improved customer
retention or whether a new, more expensive piece of equipment is more effective
than a less costly one. In this refresher, Tom Redman helps me understand what it
means for a test to be “controlled” and how you make sure it includes an element
of “randomization.” The article also addresses questions like: What are dependent
and independent variables? And what are the steps to designing and conducting
one of these experiments?

A/B testing

One of the more common experiments companies use these days is the A/B test
(which is a type of randomized controlled experiment). At their most basic, these
tests are a way to compare two versions of something to figure out which performs
better. Companies use it to answer questions like, “What is most likely to make
people click? Or buy our product? Or register with our site?” A/B testing is used to
evaluate everything from website design to online offers to headlines to product
descriptions. It’s critical to understand how to interpret the results and to avoid
common mistakes, like ending the experiment too soon before you have valid
results or trying to look at a dashboard of metrics when you really should be
focusing on a few. You can learn more about A/B tests here.

Regression analysis

Once you have the data, regression analysis helps you make sense of it. Of course,
there are many ways to analyze the data, but linear regression is one of the most
important. It’s a way of mathematically sorting out whether there’s a relationship
between two or more variables. For example, if you are in the business of selling
umbrellas, you might want to know how many more items you sell on rainy days.
Regression analysis can help you determine whether and how inches of rain
impacts sales. It answers the questions: Which factors matter most? Which can we
ignore? How do those factors interact with each other? And, perhaps most
importantly, how certain are we about all of these factors?

Fortunately, regression is not something you typically do on your own. There are
statistics programs for that! But it’s still important to understand the math behind
it and the types of mistakes to avoid. In this refresher, I explain how regression
works and share a common — but often misunderstood — warning against
confusing correlation with causation.

Statistical significance

Once you’ve done the analysis, you need to figure out what your results mean, if
anything. This is where statistical significance comes in. This is a concept that is
also often misunderstood and misused. And yet because more and more
companies are relying on data to make critical business decisions, it’s an essential
concept to understand. Statistical significance helps you quantify whether a result
from an experiment is likely due to chance or from the factors you were
measuring.

This is a concept I sometimes struggled to fully understand myself but,

fortunately, the average professional doesn’t need to understand it too deeply.
According to Tom Redman, who helped out with this refresher, it’s more
important to understand how to not misuse it.

While you’re boning up on these four concepts, it would also be helpful to read this
overview on quantitative analysis from my colleague, Walt Frick. It is a nice primer
on why data matters, picking the right metrics, and asking the right questions from
data. There’s also a great chart on correlation vs. causation to help you make
decisions about when to act on analysis and when not to.
Lastly, if you’re interested in analytics because you need to consume social science
research, I highly recommend this piece from Eva Vivalt, a research fellow and
lecturer at the Australian National University. She gives several tips for
determining whether the evidence from a study should be trusted.

Data analytics is ultimately about making good decisions. It doesn’t matter what
business you are in or what your role is at your company, we all want to — need to,
really — make smart, informed, evidence-based decisions.

Amy Gallo is a contributing editor at Harvard Business Review and the

author of the HBR Guide to Dealing with Conflict. She writes and speaks about
workplace dynamics. Follow her on Twitter at @amyegallo.

This article is about DATA

 Follow This Topic

Comments
Leave a Comment

Post Comment

6 COMMENTS
ann petry 10 months ago
Thanks for this great article Amy. My husband, an Economist, and I were just talking about this
over breakfast this weekend. Universities are playing catch-up on this but it is vital for everyone
to hone their skills and be conversant with statistics and data. Thanks for giving us some cliff
notes! You help us all become smarter.

 Reply 10

 Join The Conversation

POSTING GUIDELINES
We hope the conversations that take place on HBR.org will be energetic, constructive, and thought-provoking. To comment,
readers must sign in or register. And to ensure the quality of the discussion, our moderating team will review all comments and
may edit them for clarity, length, and relevance. Comments that are overly promotional, mean-spirited, or off-topic may be
deleted per the moderators' judgment. All postings become the property of Harvard Business Publishing.