H T T P : / / W W W. A N A LY T I C S - M A G A Z I N E .

O R G

DRIVING BETTER BUSINESS DECISIONS NOVEM BER/ DE C EM BER 2018

Will the real data scientist

please stand up?
• How to find the real deal among the posers
• With new apps, who needs a data scientist?

ALSO INSIDE:
Executive Edge
Dave Farmer, COO of
Harnham, on data and
analytics job market
from a recruitment view
• AI & machine learning
Everything you need to know
• Dose of humility
Five trends in health analytics
• Analytics & healthcare
Three ways industry can benefit
• Power BI vs. Tableau
Battle of visualization platforms
• Decision analysis software
Survey of vital tools for analysts
INS IDE STO RY
Analytics extravaganza
As I write this on Oct. 20, I know that
fall has arrived in the Atlanta area where
I live because overnight lows have finally
dipped into the 40s, college football domi-
nates the news, and I’m getting ready to
attend the INFORMS Annual Meeting.
Turns out the 2018 conference, set for Nov.
4-7 in Phoenix, may offer one last blast of
summer (predicted high of 86 on Oct. 29
according to today’s 10-day forecast), but
the real heat will be generated inside the
Phoenix Convention Center and Hyatt Re-
gency Phoenix where about 5,700 attend-
ees from the around the world will gather
for a four-day analytics extravaganza.
I’ll be there scouring the many plena-
ries, special events and 1,320 individual
sessions (yes, 1,320) looking for interest-
ing content and contributors for future is-
sues of Analytics magazine. In that regard,
Phoenix should be a hot, target-rich envi-
ronment regardless of the weather.
Speaking of the future, Analytics –
the magazine and website – have been
undergoing major, behind-the-scenes
renovations and redesigns leading up to
an all-new format for 2019. Stay tuned.
Meanwhile, let’s take a closer look at
this issue, which offers generous helpings
of data science, healthcare analytics and
the always-popular AI/machine learning
combo plate. Nick Pylypiw of Elicit, LLC,
2 | A N A LY T I C S - M A G A Z I N E . O R G
kickstarts the feature articles by asking the
question many have asked during these
early, Wild West days of the data science
era where wannabes roam unchecked:
“Will the real data scientist please stand
up: How to navigate the crowded data
scientist applicant landscape.”
The healthcare coverage starts with
Rajib Ghosh’s “Healthcare Analytics”
column in which he reviews the economic,
technological and political developments
over the past year that are shaping the
healthcare industry for better and worse.
Jeff Terry and Andy Day of GE Healthcare
follow with five surprising trends they’ve
observed that are impacting healthcare
now and going forward. Spoiler alert: The
first trend is “humility.” Gary Druckenmiller
of Evariant wraps up the health segment
by outlining three ways the healthcare
industry can benefit from analytics.
Of course, AI and machine learning’s
fingerprints are all over this issue, including
the articles just mentioned. Jerry W. Thom-
as, CEO of Decision Analyst Inc., goes a
step further and tells you everything you
wanted to know about artificial intelligence
and machine learning.
Is that the last word on AI and machine
learning? Not even close. This is Analytics. ❙
– PETER HORNER, EDITOR
peter.horner@informs.org
W W W. I N F O R M S . O R G
 Is the largest association for analytics in the
center of your professional network?
IT SHOULD BE.
• INFORMS Allows You to Network With Your Professional Peers and Others Who Share Your Interests

• INFORMS Connect, the Member-only, Online Community Lets You Network With Your Colleagues

• Unsurpassed Networking Opportunities are Available in INFORMS Communities and at Meetings

• INFORMS Offers Certification for Analytics Professionals

• Take Leadership Roles to Help Build Your Professional Profile

• INFORMS Career Center Provides You With the Industry's Leading Job Board

Join Online Today!

http://join.informs.org
C O N T E N T S

DRIVING BETTER BUSINESS DECISIONS NOVEMBER/DECEMBER 2018

Brought to you by

FEATURES
30 NERDS, SUITS AND GEEKS, OH MY!
By Nick Pylypiw
Will the real data scientist please stand up? How to navigate the
30 crowded data scientist applicant landscape.

36 FIVE TRENDS IN HEALTH ANALYTICS

By Jeff Terry and Andy Day
It starts with humility among healthcare leaders regarding the use of
data and AI to support real-time decisions.

42 ANALYTICS BOOSTS HEALTHCARE INDUSTRY

36 By Gary Druckenmiller
Three ways health systems can improve patients’ experience, create
differentiation and drive revenue growth.

46 ALL ABOUT AI AND MACHINE LEARNING

By Jerry W. Thomas
Everything you wanted to know but were afraid to ask about the two
hottest technologies that companies are rushing to exploit.

46 52 POWER BI VS. TABLEAU

By Dwight Taylor
Data visualization platforms battle to be the ‘king of the hill,’ but which
one is the right one for you and your needs?

58 DECISION ANALYSIS SOFTWARE SURVEY

By Justin Amoyal
Biennial survey demonstrates continuous advancement of vital tools for
decision-makers, managers and analysts.
58

4 | A N A LY T I C S - M A G A Z I N E . O R G W W W. I N F O R M S . O R G
 Welcome to Analytic Solver ®
Cloud-based Optimization and Simulation that Integrates with Excel

Everything in Predictive and Prescriptive Analytics functions; easy multiple parameterized simulations,
Everywhere You Want, from Concept to Deployment. decision trees, and a wide array of charts and graphs.

The Analytic Solver® suite makes the world’s best Forecasting, Data Mining and Text Mining.
optimization software and the fastest Monte Carlo
Analytic Solver is also a full-power, point-and-click tool
simulation and risk analysis software available in your
for predictive analytics, from time series methods to
web browser (cloud-based software as a service), and in
classification and regression trees, neural networks, and
Microsoft Excel. And you can easily create models in our
access to SQL databases and Spark Big Data clusters.
RASON® language for server, web and mobile apps.
Find Out More, Start Your Free Trial Now.
Linear Programming to Stochastic Optimization.
In your browser, in Excel, or in Visual Studio, Analytic
It’s all point-and-click: Fast, large-scale linear, quadratic
Solver comes with everything you need: Wizards, Help,
and mixed-integer programming, conic, nonlinear,
User Guides, 90 examples, even online training courses.
non-smooth and global optimization. Easily incorporate
uncertainty and solve with simulation optimization, Visit www.solver.com to learn more or ask questions,
stochastic programming, and robust optimization. and visit analyticsolver.com to register and start a free
trial – in the cloud, on your desktop, or both!
Comprehensive Risk and Decision Analysis.
Use a point-and-click Distribution Wizard, 50 probability
distributions, automatic distribution fitting, compound
distributions, rank-order correlation and three types of
copulas; 50 statistics, risk measures and Six Sigma
Tel 775 831 0300 • Fax 775 831 0314 • info@solver.com
 DRIVING BETTER BUSINESS DECISIONS

REGISTER FOR A FREE SUBSCRIPTION:

http://analytics.informs.org

INFORMS BOARD OF DIRECTORS

President Nicholas G. Hall, The Ohio State University
President-Elect Ramayya Krishnan, Carnegie Mellon University
26 Past President Brian Denton, University of Michigan
Secretary Victoria Chen, University of Texas at Arlington
Treasurer Michael Fu, University of Maryland
Vice President-Meetings Ronald G. Askin, Arizona State University
Vice President-Publications Jonathan F. Bard, University of Texas at Austin
Vice President-
Sections and Societies Russell Barton, Pennsylvania State University
Vice President-
Information Technology Marco Lübbecke,
RWTH Aachen University
Vice President-Practice Activities C. Allen Butler, Daniel H. Wagner Associates
Vice President-International Activities Sue Merchant, Blue Link Consulting
Vice President-Membership
and Professional Recognition Susan E. Martonosi, Harvey Mudd College
Vice President-Education Jill Hardin Wilson, Northwestern University
Vice President-Marketing,
Communications and Outreach Laura Albert, University of Wisconsin-Madison
Vice President-Chapters/Fora Gino Lim, University of Houston

INFORMS OFFICES
64
www.informs.org • Tel: 1-800-4INFORMS

DEPARTMENTS

Executive Director Melissa Moore
Director, Publications Matthew Walls
irector, Public Relations & Marketing Jeffrey M. Cohen
D
2 Inside Story
8 Executive Edge Headquarters INFORMS
5521 Research Park Drive, Suite 200
12 Analyze This! Catonsville, MD 21228
Tel.: 443.757.3500
16 Healthcare Analytics E-mail: informs@informs.org
22 INFORMS Initiatives
ANALYTICS EDITORIAL AND ADVERTISING
26 Newsmakers
Editor Peter R. Horner
64 Preview: WSC peter.horner@informs.org
68 Five-Minute Analyst Tel.: 770.587.3172
Advertising Sales Olivia Schmitz
Olivia.Schmitz@informs.org
Tel.: 443.757.3539
Assistant Editor Kara Tucker
kara.tucker@informs.org
Tel.: 443.757.3572
Analytics (ISSN 1938-1697) is published six times a year by the Layout & Design Alan Brubaker
Institute for Operations Research and the Management Sciences alan.brubaker@mail.informs.org
(INFORMS), the largest membership society in the world dedicated Tel.: 678.612.3833
to the analytics profession. For a free subscription, register at
http://analytics.informs.org. Address other correspondence to
the editor, Peter Horner, peter.horner@informs.org. The opinions
expressed in Analytics are those of the authors, and do not
necessarily reflect the opinions of INFORMS, its officers or the
editorial staff of Analytics. Analytics copyright ©2018 by the Institute
for Operations Research and the Management Sciences. All rights
reserved.

6 | A N A LY T I C S - M AGA Z I N E . O RG
 Welcome to Analytic Solver ®
Cloud-based Data and Text Mining that Integrates with Excel

Everything in Predictive and Prescriptive Analytics distributions, 50 statistics and risk measures, rank-
Everywhere You Want, from Concept to Deployment. order and copula correlation, distribution fitting, and
charts and graphs. And it has full-power, point-and-click
The Analytic Solver® suite makes powerful forecasting,
optimization, with large-scale linear and mixed-integer
data mining and text mining software available in your
programming, nonlinear and simulation optimization,
web browser (cloud-based software as a service), and in
stochastic programming and robust optimization.
Microsoft Excel. And you can easily create models in our
RASON® language for server, web and mobile apps. Find Out More, Start Your Free Trial Now.
Full-Power Data Mining and Predictive Analytics. In your browser, in Excel, or in Visual Studio, Analytic
Solver comes with everything you need: Wizards, Help,
It’s all point-and-click: Text mining, latent semantic
User Guides, 90 examples, even online training courses.
analysis, feature selection, principal components and
clustering; exponential smoothing and ARIMA for Visit www.solver.com to learn more or ask questions,
forecasting; multiple regression, logistic regression, and visit analyticsolver.com to register and start a free
k-nearest neighbors, discriminant analysis, naïve trial – in the cloud, on your desktop, or both!
Bayes, and ensembles of trees and neural networks for
prediction; and association rules for affinity analysis.
Simulation/Risk Analysis, Powerful Optimization.
Analytic Solver is also a full-power, point-and-click tool
for Monte Carlo simulation and risk analysis, with 50 Tel 775 831 0300 • Fax 775 831 0314 • info@solver.com
EXE CU TIVE E D G E
Data and analytics job
market: a recruitment
view
As with any buoyant
industry, demand for
skilled data specialists
outstrips supply, resulting
in an incredibly candidate-
friendly market and plenty
of opportunity.
BY DAVE FARMER
8 | A N A LY T I C S - M A G A Z I N E . O R G
As the global leader in data and analytics recruitment,
we make it our business at Harnham to always be aware
of the state of the marketplace. From the hottest roles
and technologies, to the most sought-after benefits, we
pride ourselves on our understanding of the industry.
And, in 2018, it’s looking stronger than ever.
While the political climate remains unstable, and the
uncertainty around Brexit continues, the data industry
is continuing to healthily grow. Our latest salary survey,
compiled with feedback from more than 1,600 United
Kingdom-based data and analytics professionals,
paints an optimistic picture of a marketplace with great
opportunities for specialists and businesses alike.
POSITION OF STRENGTH
As with any buoyant industry, demand for skilled
data specialists outstrips supply, resulting in an incredibly
candidate-friendly market and plenty of opportunity. We’re
already seeing that 56 percent of analysts have flexible
working hours and locations, but we’d expect to see this
number climb even higher over the next few years as this
way of working becomes more and more popular.
Unsurprisingly, this is also impacting salaries
shown in our survey, with an uncompetitive salary
W W W. I N F O R M S . O R G
the top reason for leaving a role.
Employers are having to ensure that
they fully compensate their staff with
competitive wages and, often, bonuses.
Some 58 percent of our respondents
received a bonus in 2017, averaging
14.6 percent of their salary. Moreover,
68 percent saw their bonus increase
over the previous year.
ANALYSTS CAN BE CHOOSERS
We’re also finding that data specialists
continue to be selective with the type of roles
they are choosing, with 69 percent having
been in their current role for less than two
years. With technology more widespread,
increased resources have resulted in
a greater amount of opportunities and
candidates are evaluating tools, tech and
projects more than ever as part of their
decision-making process. In particular,
there is a strong desire for end-to-end
ownership of projects.
Less employer loyalty has meant
an incredibly active job market, with 80
percent of people who changed roles in
2017 moving externally. This, alongside a
lack of available candidates, has led to a
marked increase in the number of contract
roles, with 20 percent of contractors saying
there are six or more of them in their team.
Furthermore, with 73 percent of initial
contracts being extended, there is more
permanence and less risk than before.
A NA L Y T I C S
However, many businesses appear to
have accepted a high rate of turnover, with
only 25 percent of respondents receiving
a counter offer after resigning. Given that
only 12 percent of these were accepted
and 33 percent of those have since
resigned from that role, we are seeing
companies proactively recruit throughout
the year rather than only seeking talent at
the point of need.
GENDER IMBALANCE
While the evolution of the industry
continues apace, we are still in a market
where data specialists are overwhelmingly
male. Females currently make up only
32 percent of data teams, and there is an
average gender pay gap of 18 percent.
Given that 15 percent of the teams surveyed
had no female members, it’s clear that the
data marketplace faces the same pitfalls as
many STEM disciplines, and we all need to
consider how to address this. For example,
to attract a more diverse range of applicants,
we believe that companies should consider
introducing benefits such as parental leave
and flexible working at the very first stage of
the application process so candidates don’t
have to ask for them.
EMPLOYER OPPORTUNITIES
In this competitive, candidate-driven
market, employers are having to work
harder to attract and retain the best talent.
N O V E M B E R / D E C E M B E R 2 018 | 9
EXE CU TIVE E D G E

However, by being aware of this there is salaries for your discipline, download the
the potential to stay ahead of the curve. If 2018 Salary Guide from harnham.com.
employers ensure that their salaries and
Dave Farmer is the COO and a founding partner
bonuses are competitive, are clear about of Harnham. A key figure in data and analytics
flexibility, benefits and project ownership, recruitment, he has helped hundreds of analysts
develop their careers and many businesses build
and expand their talent pool to build a more their data teams. He now leads the growth of
diverse team, they can stand out as market Harnham’s presence across the U.K., Europe and
leaders. And, with a lack of learning and the United States. ❙

development opportunities being the second

most popular reason for leaving a role, they EDITOR’S NOTE:
can improve their retention by fostering an The numbers and percentages quoted in this article
are based on a survey of 1,600 U.K.-based data
environment that fuels progression. and analytics professionals. Harnham conducted
similar surveys in Europe and the United States for
For more survey findings, including separate reports on those regions. All three survey
reports are available at harnham.com.
the average permanent and contract

INFORMS PubsOnLine Suite is your personal link to all areas of OR/MS research and discoveries, giving you
a serious competitive advantage. Subscribe to INFORMS PubsOnLine Suite and save!

ACCESS ALL 16 INFORMS JOURNALS FOR $109

INFORMATION INFORMS MANUFACTURING &
DECISION SYSTEMS INFORMS JOURNAL INFORMS JOURNAL TRANSACTIONS SERVICE SERVICE OPERATIONS MANAGEMENT
ANALYSIS RESEARCH ON COMPUTING ON OPTIMIZATION ON EDUCATION SCIENCE MANAGEMENT SCIENCE

MATHEMATICS
MARKETING OF OPERATIONS OPERATIONS ORGANIZATION STOCHASTIC STRATEGY TRANSPORTATION
SCIENCE
Volume 15, Number 1 Volume 29, Number 1 Volume 30, Number 1

RESEARCH SCIENCE INTERFACES SYSTEMS SCIENCE

Volume 20, Number 1

SCIENCE
Volume 1, Number 1 Volume 28, Number 2 Volume 10, Number 1

RESEARCH
March 2018 March 2018 Winter 2018 Winter 2019 Jannuary 2018 March 2018 Winter 2018 Volume 64, Number 1
January 2018

Volume 37, Number 1 Volume 43, Number 1 Volume 66, Number 1 Volume 29, Number 1 Volume 48, Number 1 Volume 7, Number 2 Volume 10, Number 1 Volume 52, Number 1
January–February 2018 February 2018 January–February 2018 January–February 2018 January–February 2018 December 2017 March 2018 January–February 2018

SUBSCRIBE TODAY!
CONTACT us at 1-800-4-INFORMs or 443.757.3500, or EMAIL us at informs@informs.org.
This is a members-only benefit: Access all journal issues back to vol. 1, no. 1.

www.informs.org/members-pubssuite

10 | A N A LY T I C S - M A G A Z I N E . O R G W W W. I N F O R M S . O R G
 ODH|CPLEX
$

$
$
Save money by optimizing
with ODH | CPLEX in the cloud
Cloud-based optimization is the clear choice for
customers wanting to save on infrastructure costs.
But the choice of solver can give further savings.
ODH|CPLEX runs in the cloud and because it’s faster
than many competitive optimizers, for many problems,
you can make dramatic savings on your hosting and
cloud costs.
A manufacturing client using ODH|CPLEX saved 75% of
his cloud costs versus a competitive solver.
A customer developing a workforce application uses
ODH|CPLEX and gains 90% cloud savings and still gets
near optimal solutions.
The combined innovation of Optimization Direct with the
power of ODH|CPLEX in the cloud is a clear choice for
complex optimization problems.
To find out more and challenge the Optimization experts,
find us at optimizationdirect.com

The IBM logo and the IBM Member Business Partner mark are trademarks of International Business Machines
Corporation, registered in many jurisdictions worldwide.
*IBM ILOG CPLEX Optimization Studio is trademark of International Business Machines Corporation and used
with permission.
ANALY ZE TH I S !

Who needs a data

scientist?
Whether you build or buy, ML/AI solutions are not easy.

Salesforce.com’s annual Dreamforce conference

recently descended upon downtown San Francisco
featuring 171,000 registered attendees. Because
my campus is just a few blocks from the site of the
conference, this event regularly hits my calendar. And
for the past few years, machine learning and artificial
intelligence have been a huge topic at Dreamforce.
Two years ago, after a slew of acquisitions of ML/
AI-based vendors (which also brought in a great deal
of data science talent), Salesforce announced a new
initiative called “Einstein.” Although it launched much
BY VIJAY MEHROTRA like a product, Einstein is really a set of AI technologies
and APIs that are intended to make it easy (for
Salesforce, heir customers and third-party developers)
to add predictive power and intelligence to nearly any
activity on the now-massive Salesforce platform. From
what I can tell, a big part of Einstein’s value proposition
is its ability to efficiently and effectively simplify a great
deal of data preparation, as well as much of the core
model building required to address specific business
with NATHANAEL ROSIDI
problems – all while maintaining data confidentiality and
customers’ trust.

12 | A N A LY T I C S - M A G A Z I N E . O R G W W W. I N F O R M S . O R G
 Since that initial announcement,
much has happened with Einstein.
In particular, in 2017, Salesforce
launched two major new products
called Einstein Analytics and Ein-
stein Discovery [1]. Einstein Ana-
lytics embeds predictive tools for
Salesforce’s sales, service and
With analytic apps, who needs a data scientist? Most companies.
marketing clouds through a col-
lection of context-specific applica-
tions. Meanwhile, Einstein Discovery is
marketed as a vehicle to leverage huge
amounts of data (from both Salesforce
and non-Salesforce sources) to spot
trends, identify issues and offer action-
able model-driven prescriptions (in the
form of Powerpoint presentations!). A key
tagline from the Einstein Discovery mar-
keting: “Understand your data without a
data scientist!”
As it happens, just a few days after
Dreamforce, I had a visit with my col-
league Nathanael Rosidi. Dr. Rosidi has
spent the better part of the last decade
working as a data scientist within the
healthcare space, most recently as a se-
nior marketing scientist at Genentech.
When I mentioned the Einstein initiative
to him, it was clear that I had hit a nerve.
After a brief discussion over cocktails,
Dr. Rosidi followed up in writing a few
days later:
While these kinds of products can there – and they often don’t really know
enhance the way business traditionally
A NA L Y T I C S
Source: ThinkStock
operates, they are usually off-the-shelf,
black box solutions that are made to fit
all departments across all industries.
An emerging downside I’ve noticed
since these types of software packages
have become popular is a perception
that data science is a commodity that can
be bought as off-the-shelf software (or
easily outsourced). In a lot of places, the
result is a reduction in an investment for
in-house analytical expertise. While this
isn’t true for companies where data sci-
ence is a core component of the compa-
ny’s competitive advantage (for example,
Stitch Fix, Google, Airbnb), this seems to
be increasingly true for companies where
data science isn’t perceived as mission
critical, especially in departments like
operations, sales and marketing. Execu-
tives in these departments seem par-
ticularly ripe for the kinds of marketing
messages that Salesforce is putting out
what they are and are not buying.
N O V E M B E R / D E C E M B E R 2 018 | 13
ANALY ZE TH I S !
Here are some of my thoughts as to why
one cannot automate data science com-
pletely and still expect a comparable ROI.
First of all, let’s remember that data
science is 80 percent data cleansing and
20 percent model building. The first steps
of data cleansing are fairly simple and
can be automated. It involves tasks like
dealing with missing and extreme values
in your data. However, the next steps
are more complicated and require deep
domain knowledge of your business and
industry. They often involve dealing with
feature selection (i.e., choosing the data
of interest) and creation of new and proxy
metrics that describe your business. Be-
cause each problem is unique to your
business, these tasks are subjective and
requires an understanding of the trade-
offs of every parameter in the model. Im-
portant note: Data models certainly can
and do sometimes fail, but being able to
drive discussion and consensus about
the trade-offs with the leaders of your
team will help to ensure that everyone is
on the same page regardless of how the
model performs. That just doesn’t hap-
pen if the model is a black box.
The last 20 percent of data science is
building models using a set of mathematical
equations to best describe your data and
offer predictions and recommendations.
Which model you choose is up to you and
clearly depends heavily on your business
14 | A N A LY T I C S - M A G A Z I N E . O R G
problem. But once you’ve implemented
your model, how do you know the output is
any good compared to other approaches?
How will your model and your insights
change over time as your data and your
business changes? The most effective
approach to data science is to constantly
iterate on how you cleanse the data and
implement the models. The interactive
human component ensures that both the
data and model continue to best represent
your evolving business.
I actually agree with Dr. Rosidi on
much of this stuff. But I had long ago [2]
accepted the inevitability of packaged
analytic applications becoming a big-
ger and bigger part of the marketplace.
This is partly because of the shortage of
data scientists and the high cost of cus-
tom application development, but also
because analytic software packages like
Einstein Analytics and Einstein Discov-
ery promise to reduce the time it takes
to gain business value, something that is
increasingly alluring in today’s fast paced
business environment.
For their part, the folks at Salesforce al-
ready appear to be trying to address some
of these concerns. The choice to open up
the Einstein APIs to third parties has cre-
ated opportunities for many new ML-based
application vendors [3] with specific do-
main expertise, all purporting to bring the
W W W. I N F O R M S . O R G
power of predictive analytics to bear on
business problems associated with sales,
marketing and customer service. Einstein
Analytics also includes its own self-service
model building capability, while custom-
ers who have their own data scientists can
also directly access the underlying models
generated by Einstein Discovery.
But any manager or executive relying
on a packaged ML/AI solution should
take Dr. Rosidi’s comments and concerns
to heart. Whether you choose to build or
to buy, make no mistake: None of this is
either simple or easy. ❙
A NA L Y T I C S
Vijay Mehrotra (vmehrotra@usfca.edu) is a
professor in the Department of Business Analytics
and Information Systems at the University of San
Francisco’s School of Management and a longtime
member of INFORMS.
Nathanael Rosidi is a data scientist, entrepreneur
and analytics instructor. He is currently a senior
marketing scientist at Genentech, and an adjunct
faculty member at the University of San Francisco.
He holds a Ph.D. in biomedical engineering from
Cornell University. 
NOTES & REFERENCES
1. The two products are now available from Salesforce
as a discounted bundle
2. http://analytics-magazine.org/analyze-this-
standardized-analytical-solutions/
3. See for example https://www.conversica.com/blog/
top-10-ai-companies-to-visit-at-dreamforce/
N O V E M B E R / D E C E M B E R 2 018 | 15
HEALT H CARE A N A LY T I C S

Healthcare technology
and analytics: 2018 is
a good year overall

As we are approaching the end of the year I have

tried to look back at 2018 in this article. It seemed
to me that while the healthcare space is becoming
increasingly complex and disruptive, enthusiasm
and investments are available in abundance. Is that
owing to a roaring economy? Perhaps.
On the policy front, many uncertainties are still
plaguing the industry, especially the future of the
Affordable Care Act (ACA). Various facets of the law
are now being selectively dismantled by the federal
government. Will ACA survive the test? The answer is
unknown at this time.
A new report indicates that health insurance
companies marketing their plans to the individual
marketplace have made significant profits and
hence they have lowered the cost of premiums for
the next year. Meanwhile, other insurers who left
in a hurry during the last couple of years claiming
BY RAJIB GHOSH that they were losing money are coming back to the
health insurance exchange markets.

16 | A N A LY T I C S - M A G A Z I N E . O R G W W W. I N F O R M S . O R G
Figure 1: Insurers in the marketplace enjoyed higher profit margins last year.
Source: Kaiser Family Foundation analysis of data from Mark Farrah Associates Health Coverage Portal™

So what changed? Insurers increased those states respond to the proposal of

their prices last year in response to the
uncertainty in the marketplace, as well as
the news of losing subsidy contribution
from the federal government for those who
are unable to pay the premiums. They got
the subsidy and increased their prices,
and therefore ended up with a higher profit
margin (see Figure 1).
Coverage for pre-existing conditions,
which is the most favored provision of
the ACA among consumers, was put
on the chopping block in many states. It
will be interesting to see how voters in
eliminating mandatory coverage for pre-
existing conditions in favor of decreased
insurance coverage. Of course, that
would help insurers eliminate costly
coverages, and for some consumers
it might lead to lower premiums. But it
would devastate those who go bankrupt
due to their uncovered healthcare costs
for their pre-existing conditions such as
chronic heart disease or cancer.
Congress is also contemplating
potential cuts to Medicare and Medicaid
coverages after the midterm election in

A NA L Y T I C S N O V E M B E R / D E C E M B E R 2 018 | 17
 HEALT H CARE A N A LY T I C S

Amazon, which has now

Figure 2: Companies have difficulties selling AI products into the healthcare industry.
Source: TechEmergence.com
November to balance budget deficits. If
that happens it would deliver a severe
blow to seniors and the underserved
population across the country as many
state and county governments depend
on federal funding to support those two
groups of consumers. It will also be a
blow to the healthcare industry overall
as Medicaid and Medicare together
make up a significant revenue for the
healthcare industry.
DISRUPTIONS OF BUSINESS
MODELS: STRANGE BEDFELLOWS
This year saw some interesting
corporate unions in healthcare. Health
insurance company Aetna was bought
by the retail pharmacy giant CVS. This
could be an attempt by CVS to protect
them from the growing shadow of like Amazon. It will be interesting to see
established themselves in the
pharmacy market. In a similar
move, health insurance
company Cigna acquired
retail pharmacy giant Express
Scripts.
Amazon can wreak havoc
in this space with its super-
efficient, technology-enabled
operations just like it did
in general retail business,
forcing former retail giants
such as Sears to file for bankruptcy.
Amazon also purchased the pharmacy
company PillPack this year and hinted
at rapidly scaling up that business.
Amazon’s vast data-driven retail
enterprise is a huge threat to the pharmacy
market. Mergers of health insurance and
retail pharmacy companies are geared
toward ensuring access to data; CVS
can continue to have access to a steady
stream of insured patients from Aetna that
Amazon might not be able to poach. CVS
can also direct those insured patients to
its retail clinics.
Will this limit choice for consumers and
drive up profit for the combined entity? The
American Medical Association believes
so. Also, neither Aetna nor CVS is known
to be an analytics-driven retailing power

18 | A N A LY T I C S - M A G A Z I N E . O R G W W W. I N F O R M S . O R G
what transpires in the marketplace and
for the consumers at the end.
EMERGENCE OF A NEW DISRUPTOR
Another interesting new entrant in
the healthcare space is the joint venture
company of Amazon, Berkshire Hathaway
and JP Morgan led by Dr. Atul Gawande.
The idea is to build a new nonprofit
healthcare company that would reduce
healthcare cost. So far it is not clear how
the cost problem would be addressed,
but it is anticipated that data and analytics
will play a big role in its operation and
price negotiations with Big Pharma.
CEOs of three legendary companies
will back Dr. Gawande, and each of
those organizations is proficient in deep
analytics. As healthcare cost reaches 18
percent of the GDP in 2018 and forecasted
to reach 20 percent by 2025, all eyes will
be on this new company to see what data-
driven innovations it would introduce in the
marketplace that could eventually help the
company achieve its stated goal where
many had failed earlier.
WATSON’S RUDE AWAKENING: AI
HITS THE REALITY SPEEDBUMP
This year also saw IBM Watson’s
image tarnished when news broke about
Watson dishing out inaccurate advices to
oncologists for cancer treatment, as well
A NA L Y T I C S
as large layoffs that happened within the
Watson division this year. In response, IBM
tried to defend its position with this blog,
where it mentioned that IBM Watson is
still used by 230 healthcare organizations
worldwide. It is unwise to write them off.
This year also saw Food and Drug
Administration (FDA) approve AI-driven
diabetic retinopathy detection device ­the
first of its kind. However, there wasn’t
any big movement of AI in healthcare a
report based on a survey of healthcare
executives. From any given list of 100
AI companies, only 33 percent have the
requisite or academic background in
science. Of those, only 25 percent are
engaged in some meaningful pilots in
healthcare. Figure 2 shows the difficulties
experienced by AI companies to sell their
product or services in healthcare.
RECORD INVESTMENT IN
DIGITAL HEALTH
In terms of venture capital investment
in digital health technologies and data-
driven solutions, 2018 saw about $6.8
billion year-to-date investments through
Q3 according to a Rock Health report.
That’s a significant jump compared to
2017, which registered $5.7 billion in-
vestment the whole year. The money
is coming both from traditional venture
firms as well as from corporations and
healthcare organizations. How many of
N O V E M B E R / D E C E M B E R 2 018 | 19
HEALT H CARE A N A LY T I C S
Figure 3: Digital health funding is on the rise.
these startups will survive? Are we seeing
enough “needs-driven innovation”? Those
are open questions, but funding is defi-
nitely available.
Despite the instability in the policy,
world healthcare technology and
analytics marketplaces stayed steady
overall. The industry is embracing itself
for some major disruptions of business
models and shifts in 2019 and beyond as
previously mentioned. Major changes in
the policy world such as cuts to Medicare
and Medicaid spending will surely
negatively affect the market, along with
overall investment. Any recessionary
pressure might also put a brake in growth
20 | A N A LY T I C S - M A G A Z I N E . O R G
Source: Rock Health Funding Database
and fund availability. But for now, it seems
like 2019 will be a smooth ride. So, let’s
raise our glasses; it’s time for a toast. ❙
Rajib Ghosh (rajib@health-roads.com) is the
founder and CEO of Health Roads, LLC, a consulting
company for enabling digital transformation in
healthcare organizations. He has 25 years of
technology experience in various industry verticals
where he had management roles in software
engineering, data analytics, program management,
product management, business operations and
strategy development. Ghosh spent a decade and
half in the U.S. healthcare industry as part of a global
ecosystem of medical device manufacturers, medical
software vendors, telemedicine and telehealth
solution providers. He’s held senior positions at
Hill-Rom, Solta Medical and Bosch Healthcare. His
recent work includes leading data-driven digital
transformation in the public health space, including
county-level healthcare agencies and organizations
focused on underserved populations.
W W W. I N F O R M S . O R G
CALL FOR ABSTRACTS
INFORMS/ALIO International Meeting Cancun 2019 June 9-12, 2019
meetings.informs.org/2019international

PLENARY SPEAKERS
Natashia Boland, Georgia Tech, USA
Cole Smith, Clemson University, USA
Enrique Covarrubias, Actinver, Mexico

KEYNOTE SPEAKERS
Alice Smith, Auburn University, USA
Eduardo Uchoa, Universidade Federal Fluminense, Brazil

TUTORIAL SPEAKERS
Alexandra Newman, Colorado School of Mines, USA
Mariel Lavieri, University of Michigan, USA
Andre Luiz Diniz, Eletrobras Cepel, Brazil
Ed Klotz, IBM, USA
Emilio Carrizosa, Universidad de Sevilla, Spain
Javiera Barrera, Adolfo Ibáñez University, Chile
Juan Pablo Vielma, MIT, USA
Meisam Razaviyayn, University of Southern California, USA

IMPORTANT DATES
Abstract submission deadline: March 11, 2019
Poster submission deadline: March 11, 2019
Registration deadline: April 29, 2019
Further information about submission and registration:
meetings.informs.org/2019international/submit
meetings.informs.org/2019international/register

COMMITTEE
General Chair: José Luis González-Velarde, Tec de Monterrey
PC Chairs: Roger Z. Ríos, UANL and Andrés Medaglia, U de Los Andes
Full committee at:
meetings.informs.org/2019international/organizing-committee
INFO RM S IN I T I AT I VE S

Board elections, crop

challenge, how not to
respond to critics
GEORGIA TECH’S KESKINOCAK ELECTED
PRESIDENT-ELECT OF INFORMS
Pinar Keskinocak (photo), the William W. George
Chair and Professor in the Stewart School of Industrial
and Systems Engineering at Georgia Institute of
Technology, and an INFORMS Fellow, was elected
president-elect of INFORMS following a vote of the
Institute’s membership. After a year on the INFORMS
Board as president-elect, Dr. Keskinocak will assume
the presidency in 2020.
Dr. Keskinocak served as the Secretary of
INFORMS and the Vice President of Membership
and Professional Recognition on the
INFORMS Board. She co-founded and
served as the president of the Public
Sector Operations Research Section
and the Junior Faculty Interaction
Group Forum, as well as the president
of the Health Applications Society and
the Women in OR/MS Forum.
At Georgia Tech, Dr. Keskinocak
is the co-founder and director of the
Center for Health and Humanitarian
Systems and serves as the College of
Pinar Keskinocak

22 | A N A LY T I C S - M A G A Z I N E . O R G W W W. I N F O R M S . O R G
Engineering ADVANCE Professor. Dr.
Keskinocak’s research focuses on the
applications of operations research and
management science with societal im-
pact, particularly in health and humanitar-
ian systems, supply chain management,
and logistics/transportation.
Other newly elected members of
the INFORMS Board for 2019 include:
David Hunt of Oliver Wyman (Treasurer),
Cole Smith of Clemson University
(Vice President-Publications), Melissa
“Missie” Bowers of the University of
Tennessee (Vice President-Education),
Tamas Terlaky of Lehigh University
(Vice President-Meetings), Subramanian
“Raghu” Raghavan of the University of
Maryland (Vice President-Technology
Strategies) and Laurie Garrow of Georgia
Tech (Vice President-Sections/Societies).
SYNGENTA CROP CHALLENGE TO
TEST AND AWARD ANALYTICAL SKILLS
The 2019 Syngenta Crop Challenge in
Analytics, organized by the Analytics Soci-
ety of INFORMS, is now open. By accept-
ing the crop challenge, team contestants
demonstrate how data can be used to
feed the future world for a chance to share
in prize money of $5,000 for first place,
$2,500 for second place and $1,000 for
third place. This year’s problem addresses
the tolerance of various corn hybrids to
stresses that might be encountered during
A NA L Y T I C S
the growing season. Visit
the competition website
to sign up. For more infor-
mation and to download
the submission template,
click “Learn More.” For
answers to questions, fill
in the email template un-
der “Contact.”
Submissions are due Syngenta Crop Challenge
Jan. 18, 2019. Finalists will be announced
in March 2019, and they will present their
work during the INFORMS Business An-
alytics Conference April 14-16, 2019, in
Austin, Texas, where the winners will be
announced.
WHEN MANAGERS RESPOND
TO ONLINE CRITICS, NEGATIVE
REVIEWS ENSUE
A new study in the INFORMS journal
Marketing Science found that when
managers respond to online reviews
it’s possible that those responses could
actually stimulate additional reviewing
activity and an increased number of
negative reviews.
The study, “Channels of Impact: User
Reviews When Quality is Dynamic and
Managers Respond,” is authored by
Judith Chevalier of the Yale School of
Management and NBER, Yaniv Dover of
the Hebrew University of Jerusalem and
Dina Mayzlin of the Marshal School of
N O V E M B E R / D E C E M B E R 2 018 | 23
 INFO RM S IN I T I AT I VE S
Mangers should reconsider responding to negative online reviews.
Source: ThinkStock
Business at the University of Southern
California. Their research focused on on-
line reviews of “dynamic-quality” goods
and services, such as hotel and travel/
hospitality services, as compared to “stat-
ic-quality” goods and services, where the
consumer experience is not as influenced
by direct managerial involvement.
The purpose of the research was to
learn about the role and effectiveness of
managers responding to online reviews
and the motivations of consumers who en-
gage in online reviewing. The researchers
studied online reviews of hotels and ana-
lyzed reviews and data from portals that
included TripAdvisor, Expedia and Hotels.
com. They focused on the drivers for word-
of-mouth that are influenced by the pres-
ence of a managerial response, or lack of
response, to online reviews. In the end, the
authors found that managerial responses
are likely to stimulate reviewing activity.
Study author Yaniv Dover says that
one of the key differences between con-
sumers who review dynamic-quality and
static-quality goods and services is that
24 | A N A LY T I C S - M A G A Z I N E . O R G
there are two audiences for the re-
views of dynamic-quality goods and
services. “In the case of … a review
of a product purchase on Amazon,
the reviewer’s audience is more
than likely other consumers,” he
says. “But when reviewing dynam-
ic-quality goods and services, the
reviewer could be motivated by an intent
to impact the manager, and not just other
consumers.”
The entry of the firm’s manager into
the conversation potentially changes the
nature of the discourse, which in turn af-
fects the customers’ incentives to post
reviews,” Dover adds. “The response
functionality transforms a peer-to-peer
review system into a hybrid system that
includes both peer review and a custom-
er feedback avenue.”
In working to understand the consum-
er profile of those who engage in online
reviewing, particularly, in response to
managerial posts, the authors said that
having the ability to influence the target-
ed firm could be a major factor. “Based on
our research, we believe that managerial
response activity disproportionately stim-
ulates negative review production since
the negative feedback may be seen by
reviewers as particularly impactful,” Do-
ver says.
For the full study click here. ❙
W W W. I N F O R M S . O R G
 ASSOCIATE CERTIFIED
ANALYTICS PROFESSIONAL
Analyze How aCAP Can Help
Launch Your Analytics Career

The Associate Certified Analytics Professional (aCAP) is an

entry-level analytics professional who is educated in the
analytics process but may not have practice experience.

This prestigious program begins a career pathway

leading to the CAP designation.

www.certifiedanalytics.org
NE W S M AK E R S

Salary survey, cloud

fraud, immigration
numbers and more
SALARY SURVEY PAINTS OPTIMISTIC PICTURE
FOR ANALYTICS PROFESSIONALS
Harnham, a global leader in data and analytics
recruitment, recently released the 2018 editions of its
salary guides for the United Kingdom, the United States
and Europe. The surveys are based on feedback from
thousands of data and analytics professionals across
the globe.
“Our surveys are created for analysts, by analysts
and offer a detailed, on-the-ground look at what’s con-
cerning talent in the industry,” says Harnham COO
Dave Farmer. “The data industry continues to grow
and shows no sign of
slowing, with demand for
analysts still easily out-
stripping supply.”
All three salary surveys
are available to download
from www.harnham.com
and include salary and
trend analysis across five
Harnham released 2018 salary guides for the United Kingdom, United States key specialisms: data and
and Europe. technology, data science,

26 | A N A LY T I C S - M A G A Z I N E . O R G W W W. I N F O R M S . O R G
digital analytics, marketing and insight,
and risk analytics.
Farmer further discusses the salary
surveys and key findings in this issue’s
Executive Edge column.
YALE RESEARCH ON IMMIGRATION,
AGING RUNNERS GRABS
HEADLINES
A recent study by Yale professor and
former INFORMS President Edward H.
Kaplan (photo), Yale colleague Jonathan
Feinstein and Mohammad M. Fazel-
Zarandi of MIT suggests that the number
of undocumented immigrants in the
United States is nearly twice as many
as experts previously thought. Since its
publication last month, the study, which
estimates the number of such immigrants
at 22.1 million instead of 11.3 million, has
garnered worldwide attention from major
media outlets including the Los Angeles
Times, the Boston Globe, Fox News,
Bloomberg News and the Daily Mail.
The study, which provides an
estimated range of undocumented
immigrants between 16 million and 29
million with 22.1 million as the mean, is
based on demographic modeling with
data from 1990 to 2016. The authors are
quick to point out, as Kaplan did in an
appearance on “Fox and Friends,” that
their research does not indicate that the
number of undocumented immigrants
in the U.S. is suddenly exploding. To
A NA L Y T I C S
the contrary, the research
shows: 1) that the greatest
growth of undocumented
immigrants during that
timeframe occurred in the
1990s through the mid-
2000s, 2) that the growth
was vastly undercounted Edward H. Kaplan
at the time, and 3) that the number has
remained relatively stable over the past
decade.
As the Yale School of Management’s
Insights notes, the approach in the new
research was based on operational data,
such as deportations, visa overstays and
demographic data, including death rates
and immigration rates. “We combined
these data using a demographic model
that follows a very simple logic,” Kaplan
explains. “The population today is equal
to the initial population plus everyone
who came in minus everyone who went
out. It’s that simple.”
Kaplan adds, “The analysis we’ve
done can be thought of as estimating
the size of a hidden population. People
who are undocumented immigrants are
not walking around with labels on their
foreheads. Neither are populations of
homeless people, neither are populations
of drug users, and neither are populations
of terrorists. Yet for policy, it is very
important to know the size of these
hidden populations because that sets
N O V E M B E R / D E C E M B E R 2 018 | 27
NE W S M AK E R S
the scale of the problem in each of these
different policy areas.”
Another study co-authored by Kaplan –
this one co-authored with Yale economist
Ray Fair and focused on running and
aging – served as the basis for an Oct.
3 article (“We Slow as We Age, but May
Not Need to Slow Too Much”) in the New
York Times by Gretchen Reynolds.
“The new analysis, which refines
famous past research by one of the
scientists [Fair], finds that, although
declines in running performance with
age are ineluctable, they may be less
steep than many of us fear,” Reynolds
writes. “And, perhaps most important,
the new research updates a popular
formula and calculator that runners
past the age of 40 can use to determine
how fast we can expect to slow down
and provides us with reasonable, age-
appropriate finishing-time targets for
ourselves.”
As part of their research, Fair and
Kaplan analyzed data on world masters
running records for men ages 35 to 95 for
the 5K, 10K, half marathon and marathon
events. (The data on women runners was
limited and statistically suspect.)
“The complicated calculations indicate
that current world records for older runners
theoretically could drop by as much as
8 percent in the future, Dr. Kaplan says,
28 | A N A LY T I C S - M A G A Z I N E . O R G
providing all of us new benchmarks for
our own aging performance,” the Times
article concludes.
REPORT: ONE IN FIVE CLOUD-BASED
USER ACCOUNTS MAY BE FAKE
According to the Q2 2018 DataVisor
Fraud Index Report, more than one in
five user accounts set up through cloud
service providers may be fraudulent. The
report, based on information gathered
between April and June, analyzes 1.1
billion active user accounts, 1.5 million
email domains, 231,000 device types
and 562 cloud hosting providers and
data centers, among other indicators.
DataVisor, a fraud detection platform,
found that 22 percent of accounts
originating from cloud service IP ranges
appear to be fraudulent. Malicious
accounts are eight times more likely to
originate via cloud services than normal
users. Some cloud services and data
centers can have more than 75 percent
fraudulent accounts, the study found.
More than 21 percent of fake ac-
counts targeting online and financial ser-
vices originated from the United States,
and 17 percent originated from China. In
attacks targeting North American online
services, more than 45 percent originat-
ed in the U.S. Interestingly, crime rings
leverage different cloud service providers
W W W. I N F O R M S . O R G
More than 90 percent of fake account registration in social platforms involves coordinated attacks.
depending on the attack. Fraudsters tar-
geting social platforms largely use Ama-
zon Web Services; DigitalOcean appears
to be preferred by fraudsters targeting
mobile apps and financial services, ac-
cording to the report.
Coordinated attacks – a group of
fraudulent accounts controlled by the
same attacker – represent the majority
of fraudulent activity in both social plat-
forms and financial services, the report
found. More than 90 percent of fake
account registration in social platforms
involves coordinated attacks; in the fi-
nancial sector more than 40 percent of
A NA L Y T I C S
Source: ThinkStock
application fraud comes from coordi-
nated attacks.
While most fraudulent attacks occur less
than a day after accounts are established,
some “sleeper cell” accounts can lie in wait
for months or years before being used. On
average, fraudulent accounts incubate for
35 days before attacking.
“[This report]” demonstrates that
the increased adoption of the cloud
has unintended consequences for the
financial well-being of online businesses,”
says Yinglian Xie, CEO and co-founder
of DataVisor. To read this and other
DataVisor reports, click here. ❙
N O V E M B E R / D E C E M B E R 2 018 | 29
NE RDS , S U IT S & G E E KS
Will the real data scientist
please stand up?
How to navigate the crowded data scientist applicant
landscape.
BY NICK PYLYPIW
D
ata science has seen a
dramatic rise in the last de-
cade. The LinkedIn 2017
U.S. Emerging Jobs Report
revealed the two fastest growing jobs as
“machine learning engineer” and “data
scientist.” Universities are struggling to
keep up with this trend, assembling new
programs to address the growing need for
data science professionals. Many schools
are even offering accelerated, online
programs to make the investment more
manageable and appealing for potential
candidates.
30 | A N A LY T I C S - M A G A Z I N E . O R G
Meanwhile, companies of all shapes
and sizes are competing for the attention
of these newly popular “nerds,” offering
large sums of money and flexible work
schedules. With the university system
churning out talent at breakneck speed,
it would seem that the industry’s appetite
for data scientists would be satiated.
However, the gap between supply and
demand seems to be growing.
One problem seems to be lack of qual-
ity, not quantity, but determining an individ-
ual’s data science qualifications is a very
difficult task. This ambiguity allows anybody
W W W. I N F O R M S . O R G
One problem is lack of quality, not quantity, but determining an individual’s data science qualifications is a difficult task.
to put “data scientist” on their business
card with little to no accountability. Though
there are imposters in every field, the cur-
rent bumper crop of data scientists seems
to be littered with applicants citing only a
few Kaggle submissions as evidence of
technical experience. To be clear, Kaggle
competitions are hotly contested affairs
and routinely generate high-quality work.
However, the problems are generally well-
defined with relatively limited data sourc-
ing and preparation. This leaves only the
modeling piece which, at the risk of sound-
ing blasphemous, is arguably the easiest
part of the process.
In addition to the folks above, there
are those who have been stretched by
their employers into uncomfortable terri-
tory. As a way to cope with the difficulty
A NA L Y T I C S
Source: ThinkStock
and expense (and culture shift) associ-
ated with standing up a data science
team, some companies choose to cut
corners by rebranding their best busi-
ness analysts and IT professionals as
data scientists. Without the proper train-
ing and guidance, these makeshift brain
trusts quickly find themselves out of their
depth, resulting in missed deliverables,
unanswered business questions and, ul-
timately, the loss of critical support from
stakeholders. This fairly common and in-
tentional practice of putting square pegs
into round holes is not fair to the individu-
als and is rarely successful. Furthermore,
it has the added effect of diluting the field,
fostering further skepticism around the
benefits that a properly structured data
science team can provide.
N O V E M B E R / D E C E M B E R 2 018 | 31
REAL DATA S C I E N T I ST
Of course, the growing mountain
of resumes is not all imposters. There
are plenty of capable and talented data
scientists in the pile as well. In addi-
tion to the hordes of newly minted data
scientists emerging from university
programs, there is a large group of sta-
tistical modelers who have been doing
“data science” for longer than it has
been called that. These analytics prac-
titioners have the technical skills and
business experience to be successful
in any industry. However, they can be
difficult to spot, partially due to the lack
of the resume buzzwords that recruiting
algorithms typically search for.
So what, exactly, does a real data
scientist look like? How can you find a
ruby in a mountain of rocks? Is there a
Coupe de Ville hiding at the bottom of
that Cracker Jack box? At Elicit, we have
an approach that we believe in, called
“Geek Nerd Suit.” This framework is in-
tegral to how we build our teams and ap-
proach our work. Aspiring data scientists
are assessed on three core competen-
cies: technology and data (Geek); in-
sights and modeling (Nerd); and strategy
and communication (Suit). In our expe-
rience, balance in these three pillars is
directly correlated with success, though
we typically see data scientists slightly
over-index in the “Nerd” category.
32 | A N A LY T I C S - M A G A Z I N E . O R G
NERD
There is an ever-increasing number
of software options that make machine
learning extremely easy. Even outside
of these point-click interfaces, the actual
construction of a machine-learning model
is incredibly simple, often a one-liner in R
or Python. The ease of use can create
an inflated sense of confidence in the
results, and in the process itself, even
prompting some companies to question
why the data science skillset is needed
at all.
There is real danger in this sort of
“drive-thru” analytics, however, as the
out-of-the-box models have limited
options and flexibility. Anybody can copy
a line of code from Stack Overflow and
build a model. The things that happen
before and after that one line of code are
what makes a data scientist valuable.
In addition to being able to know why
a certain algorithm is appropriate for
a specific business problem, a data
scientist must be able to explain how
the model works, as well as how to tailor
the model specifications for the given
use case. This requires some significant
knowledge in mathematics and statistics.
Without proper context, a data scientist
could recommend business action based
on egregious misinterpretations of a
model’s output.
W W W. I N F O R M S . O R G
SUIT
The technical skills discussed above
are obviously crucial, but they are
worthless without the ability to synthesize
business needs and goals into questions
that can be answered with data science.
Business partners can be skeptical of
machine learning, especially when the
insights generated do not validate their
existing views. Convincing them that the
data-driven insights produced by the
data science team are likely to be more
reliable than anecdotes and the status
quo can be challenging.
An effective data scientist under-
stands how to weave the model insights
into a story, allowing business partners
to see the strategic benefit and, perhaps
more importantly, feel comfortable con-
tributing to the conversation. Without
the necessary communication skills, a
data scientist could be interpreted as
condescending or dismissive.
GEEK
Data is rarely collected, organized,
stored and shaped in a model-ready
state. In large organizations, the IT
team is sometimes tasked with creat-
ing the appropriate tables for reporting
and data science projects. Deliberately
stepping through the processes of ticket
creation, prioritization, delegation and
A NA L Y T I C S
ETL (extract, transform, load) creates
delays that the business cannot afford.
These processes are important and
necessary, of course, but not suited for
the iterative work required to properly
implement a machine-learning algo-
rithm. As the team works toward a so-
lution, they might pivot several times,
deciding that weekly is a better aggre-
gation than monthly, or that the mar-
keting data is needed after all, or that
there is another source of that account
info that has fewer missing values.
Rather than rely on IT to build these
tables over and over, many companies
prefer to build a sandbox environment
where the data science team can build
their own tables, adding columns and
sources as appropriate, before passing
these plans to IT for production. This
requires data scientists who are com-
fortable working with large and varied
data sources, and who have the cod-
ing skills (generally SQL) needed for
exploring and joining data in the envi-
ronment. A poorly written SQL query
can bring down a server and impact all
of the other processes in the environ-
ment. Perhaps worse, the results re-
turned from said poorly written query
may be built on incorrect sources, joins
and filters, essentially producing mis-
leading data.
N O V E M B E R / D E C E M B E R 2 018 | 33
REAL DATA S C I E N T I ST
An interview is usually not enough
time to screen for competence in these
areas, so how can we make sure we are
hiring quality data scientists, those with
the skills to be successful? One method
that some companies rely on is a mini
project. If executed properly, this is an
exercise that can tell you much about the
capabilities of potential candidate.
MINI PROJECT
A week or so before the interview, send
the applicant a few data sets and a rela-
tively vague business context. Ask them to
prepare analysis relevant to the business
question, as well as a 15-minute business
presentation. During the conversation, use
specific, probing questions to dig for the
“Geek Nerd Suit” competencies. Here are
the things to look for:
Strategy (Suit). What were the ques-
tions being answered and were they
relevant to the overall business con-
text provided? How was the solution
organized?
Data preparation (Geek). What steps
were taken to prepare the data? Were ta-
bles joined together or transposed? Were
there any missing or anomalous obser-
vations? If so, how were these handled?
Were new fields created from existing
ones? Most importantly, why were these
steps taken?
34 | A N A LY T I C S - M A G A Z I N E . O R G
Modeling (Nerd). What algorithms/
techniques were used? Were these su-
pervised/unsupervised? Why were these
particular techniques chosen? How is the
output from these models interpreted?
Presentation (Suit). Are the insights
accessible for someone without the tech-
nical vocabulary? How were the model/
insight/data discussions handled? Were
the findings clear?
Vision (Geek/Nerd/Suit). What are
the next steps for this analysis? What
other questions were born as a result of
this work? What additional steps could
improve the results? Are there addition-
al data pieces that could strengthen the
model?
A candidate who can demonstrate
competence in the above areas is likely
to be successful in your organization. Of
course, mastery in all three areas isn’t a
requirement, but it’s a strong indicator of
the level of performance the individual
would bring to a business setting. ❙
Nick Pylypiw is a manager on the Data Science
team at Elicit, LLC, a customer science and strategy
consultancy that helps clients uncover latent insights
about their customers, and apply those insights to
business, marketing, product, loyalty, brand and
customer experience strategy. The company’s
Fortune 500 clients include Southwest Airlines,
HomeAway, Fossil, GameStop, Sephora, BevMo!
and Pier 1 Imports.
W W W. I N F O R M S . O R G
 TA L KING ALL ABOU
A R E T
LE
OP W H AT I S UP- A
ND
PE

-CO
MING
MEMBERSHIP

AT I N F O
RENEWAL !

M R
S

READ THE
PRESIDENTʼS
INVITATION
Thank you for TO RENEW
supporting your
profession in 2018
with your INFORMS
membership.

?
CAN WE COUNT
ON YOUR
CONTINUED
Online renewals available at SUPPORT
renew.informs.org FOR 2019

443.757.3500 or
1.800.446.3676

RENEW
informs@informs.org

ONLINE
TODAY
HALT IN G TH E H Y P E
Humility and other trends
in health analytics
Scenario planning with simulation analytics and
problem-back machine learning make the trend list.
BY JEFF TERRY (left) and ANDY DAY
O
ur team at GE Healthcare
uses analytics to help
caregivers improve outcomes
and efficiency. Our work
includes retrospective analytics to assess
performance and monitor KPIs, simulation
analytics to align leaders with a strategy by
testing alternative futures, and actionable
analytics to support real-time decisions
with predictive and prescriptive information.
Here are some recent trends we observe in
health analytics:
1. Humility. We notice growing humility
among healthcare leaders regarding the
use of data and artificial intelligence (AI)
to support real-time decisions. It’s quite
36 | A N A LY T I C S - M A G A Z I N E . O R G
striking. A few years ago, the CIO, CTO
or CMIO of many major institutions would
hold forth about their big data strategy and
growing army of data scientists. Vendors
spoke of data lakes, AI platforms and
digital envelopes; it was an international
broken record of buzzwords and futures.
If only we had all the world’s data, then
we’d find something to do with it.
Fast forward to present day, and
something is different. Much of that old
drumbeat continues of course, but a
contrarian cadence has risen to meet
it. Leaders talk about moving from
“PowerPoint to proof points.” Consider
recent headlines:
W W W. I N F O R M S . O R G
• “How to Cut Through
the Artificial Intelligence
Hype” – TechEmergence
• “Fake Artificial
Intelligence vs. the Real
Thing” – Enterprise Tech
• “Artificial Intelligence in
Business: Separating the
Real from the Hype.” –
McKinsey & Company

You see it up close with

healthcare leaders too. One
of the authors recently had
the honor to facilitate a panel
with three CIOs from major
Data scientists, change leaders and clinicians are getting down to the difficult work of
health systems in New York solving problems with real-time data.
City, each a great leader from Source: ThinkStock

a great institution. What struck us most retail, its complexity dwarfs all other do-
was their humility. They see the potential mains combined. In other words, a new
of predictive analytics and AI, but they appreciation of the chasm from imagin-
were also upfront about the relatively ing to actually implementing a real-time
small steps their teams have made so health analytic. And it’s into that chasm
far. “Very basic” use of machine learning that healthcare analytics leaders must
to route ambulances. Tracking refrigera- dive.
tor temperatures. Recognition that even For us, this is the most positive
though all sites may be on a single EMR trend of all. As unrealistic optimism
instance, it still takes rework to imple- fades, data scientists, data translators,
ment an analytic at two sites in the same change leaders and clinicians are get-
system. And so on. ting down to the intensely difficult work
We see this humility all over the world. of solving problems with real-time data
A new appreciation that while health in service of caregivers, patients and
data is “small” compared to telecom and families. Humility. That’s good news.

A NA L Y T I C S N O V E M B E R / D E C E M B E R 2 018 | 37
HEALT H AN A LY T I C S T R E ND S
2. Scenario planning with simulation
analytics. We notice accelerating adoption
of digital twins and other simulation analyt-
ics to test alternative futures and eventually
align leaders with a strategic plan. Using
simulation analytics to test alternative sur-
gical block schedules, bed complements,
bed algorithms, winter-surge strategies
and unit-level staffing plans is not exactly
new. Our team has been doing this work
for more than 10 years. But acceptance
of these methods, and even eagerness
to adopt them, has undoubtedly turned
a corner. Ten years ago, our initial client
presentations had to have an educational
flavor. Today, many healthcare leaders
understand the concepts and instead ask
pointed questions about the scalability and
durability of our technology.
Simulation analytics take many
forms. Given the complexity of hospitals
and health systems, not to mention
the human factor of patient decision-
making, it’s hard to argue that discrete-
event simulation and agent-based
modeling are the most appropriate
tools for healthcare strategic planning.
Growing awareness of these tools is
one factor driving their use.
The other factor is increasing utili-
zation of tertiary and quaternary facili-
ties driven by consolidation, an aging
population and the relative infancy of
38 | A N A LY T I C S - M A G A Z I N E . O R G
population health efforts. We see this all
over the world. Healthcare executives
in the United States, Canada, United
Kingdom, France, Brazil, Mexico, Ger-
many, Saudi Arabia, Australia and virtu-
ally everywhere else that we work face
budgets that have grown more slowly
than patient demand, not to mention
patient expectations.
There are no easy answers, but
more and more healthcare leaders are
using simulation analytics to test alter-
native investment strategies and, in the
process, align divergent stakeholders
to an agreed-upon path forward. The
key is that there are many good ideas.
The risk is to launch many small proj-
ects rather than the few crucial high-
impact programs. The challenge is that
each project affects the others. Simu-
lation analytics are increasingly valued
for their ability to measure the overlap,
understand the potential ramifications,
and align the team.
3. Analytics career paths. We notice
growing appreciation of a simple fact:
“data scientist” must be a career and not
just a job. To be viable, an analytics ca-
pability must offer employees a long-term
career path. This seems so obvious, and
yet organizations have spent years and
large sums to learn it. “We have a team of
W W W. I N F O R M S . O R G
20 data scientists,” or “we hired an amaz-
ing data scientist with [amazing clinical
and data science credentials]” just isn’t
enough. Imagining, building, validating,
deploying, refining, supporting and ex-
panding real-time decision support ana-
lytics is not a year’s work. It’s decades
of work.
In recent discussions with CIOs and
CMIOs there is a growing appreciation of
this reality. In short, an appreciation that
the capability they seek requires career
paths to attract, retain and grow great
employees in stable roles over decades
so those employees can invest in their
lives and families.
The good news is that the macro-
trend toward fewer, larger systems makes
such career paths possible in more and
more large health systems. Even so,
much work remains to craft those career
paths, compensation structures, etc. By
the same token, we work with leaders
of smaller institutions who have spotted
this trend and choose to outsource their
analytics capability.

OR What
 

  

  



 
 


 


 ­

/ 

 // /



A NA L Y T I C S N O V E M B E R / D E C E M B E R 2 018 | 39
HEALT H AN A LY T I C S T R E ND S
4. Problem-back machine learning.
We notice that while machine learning
(ML) is the most accessible rung on the
AI ladder, the technology alone is insuf-
ficient. Fully harnessing the power of ML
to predict census and bottlenecks re-
quires three competencies: technology,
nuanced local knowledge of healthcare
delivery, and a data translator to bridge
the two. As with humility, there seems to
be a growing appreciation that it is the
combination of these competencies that
is essential to impact care delivery. To-
gether, staff can work “problem-back” to
first pinpoint the problem, then the de-
sired response and finally the analytic
that will trigger it.
5. Early warning explosion. We notice
a proliferation of early warning algorithms
to predict inpatient deterioration. There’s
HEWS, MEWS, CEWS, TEWS, PEWS
and 50 more. This trend is related to the
democratization of machine learning. It
also demonstrates a wonderful aspect of
modern patient care: the data exists in the
EMR and patient-monitoring equipment to
Subscribe to Analytics
It’s fast, it’s easy and it’s FREE! Just visit: http://analytics.informs.org/
40 | A N A LY T I C S - M A G A Z I N E . O R G
identify with some measure of accuracy
those patients who are likely to have
problems tomorrow. A remarkable reality.
That’s the good news; it is increasingly
possible to anticipate patients who will
decline. It’s not yet routine, but no longer
science fiction either. The harder part is
putting this into practice. Why might that
patient deteriorate? Who should see that
information? What to do? Answering
these questions is the work ahead, and
many great caregivers are sorting them
out. We’re trying to help through our
command center work. The challenge is
to incorporate new methods into an acute
care setting that is already strained,
already delivering great patient care, and
(sometimes rightly) resistant to change.
It’s a wonderful time for analytics in
healthcare and always an honor to serve
caregivers. ❙
Jeff Terry is CEO of Healthcare Command Centers,
GE Healthcare. Andy Day is a principal, advanced
analytics, with GE Healthcare Partners where he
leads the global analytic consulting team and the
design of GE’s real-time data analytics for hospital
command centers.
W W W. I N F O R M S . O R G
 brought to you by

Participate in the 2019 Syngenta Crop Challenge in

PUT YOUR Analytics brought to you by the Analytics Society of
ANALYTICAL INFORMS and use your analytical and mathematical skills
SKILLS TO to help solve a global challenge: growing more food to
THE TEST meet the needs of a growing population.

WIN UP TO Enrolled participants receive a real-world data set to use

$5,000 in developing a methodology that addresses a research

question relating to crop productivity.

REGISTER NOW TO ENTER

THE 2019 CROP CHALLENGE IN ANALYTICS
www.ideaconnection.com/syngenta-crop-challenge

@SyngentaUS @INFORMS
#CropChallenge

All photos are either the property of Syngenta or are used with permission. ©2018 Syngenta. The Syngenta logo is a registered trademark of
Syngenta Group Company. Other trademarks are the property of their respective owners. Syngenta supports but is not a sponsor of this
INFORMS challenge. Entrants must read and agree to terms and conditions of the challenge, found with the registration materials.
VA L U E- BAS E D CA R E

Three ways the

healthcare industry can
benefit from analytics

BY GARY DRUCKENMILLER

the midst of an industry fact, the amount of patient data is estimated

In shift toward value-based

care and an increasing
demand for “retail
healthcare experiences,” health systems
are struggling to remain competitive. How
can organizations provide high-quality,
personalized patient experiences that
differentiate their facility from the hospital
across town?
The answer? Health analytics. There
is an enormous amount of patient data
waiting at the fingertips of providers,
among clinical, financial, consumer and
behavioral information (to name a few). In
to reach roughly 25,000 petabytes by the
year 2020. Unfortunately, the majority of
this data goes unused, because 90 percent
of health systems don’t have the capacity
to collect and analyze this information and
produce actionable clinical insights.
With the right data hub to collect data
from disparate sources and synthesize
this information, health systems can use
analytics to influence patient outcomes,
create differentiation and drive revenue
growth.
Let’s take a look at a few ways that
analytics can benefit the healthcare industry:

42 | A N A LY T I C S - M A G A Z I N E . O R G W W W. I N F O R M S . O R G
Creating a holistic view of patients is essential to creating personalized healthcare experiences.
1. Create a holistic view of patients.
Creating a holistic view of patients
is essential to creating personalized
healthcare experiences. With access to
clinical, financial, consumer and behavioral
data, health systems can develop an
engagement strategy that is more likely to
be effective for each unique consumer or
patient.
Specifically, this data can be used to
create targeted healthcare marketing
campaigns that reach the right consumers
and patients with the right care information,
through the right communication channels.
That being said, with the massive amount
of patient data being collected from many
different sources, it’s difficult to create a
holistic view of each consumer or patient.
A NA L Y T I C S
Source: ThinkStock
This is where health analytics come into
play.
With a health analytics engine,
organizations are able to optimally leverage
patient data using technology that can
incorporate multiple data feeds and analyze
the information to produce insights. Without
this technology, it would be much more
complicated to aggregate all the patient
data sets available and make sense of
the information to drive actionable, timely
decision-making that improves patient
experiences.
2. Improve patient engagement.
Consumers have endless options when
it comes to selecting a healthcare pro-
vider. Now, patients can independently
N O V E M B E R / D E C E M B E R 2 018 | 43
HEALT H CARE : A NA LY T I C S B E NE F IT S
search the Internet for information
about their health (including risk fac-
tors, symptoms, treatment options,
etc.) and browse nearby healthcare fa-
cilities. This makes it difficult to attract
and retain patients.
What’s the solution? Healthcare
organizations can utilize a data-driven,
consumer-centric approach to emphasize
personalized communication and
engagement throughout the customer
journey. The customer journey refers to
the steps a patient goes through as they
interact with a healthcare organization.
Typically, consumers and patients will
follow these steps:
• Awareness: Self-assessment of
conditions and symptoms that leads
to online research and education,
including posing questions on social
media, etc.
• Help: Initial contact with health
system via call center, e-mail or
mobile.
• Care: Assessment of health
condition in medical facility
(physician’s office, hospital, etc.).
• Treatment: On-site and follow-
up care (medications, physical
therapy, etc.).
• Behavioral/lifestyle change:
Changes to reduce readmissions
and promote proactive health.
44 | A N A LY T I C S - M A G A Z I N E . O R G
• Ongoing care/proactive health:
Ongoing care management between
patient visits, fostering engagement
between the patient and physician
and enabling the patient to better
manage his/her own care.
Patient engagement is dependent
on delivering the right information to
the right patients at the right time.
Healthcare organizations can take
advantage of customer journey mapping
and health analytics to determine what
messaging will be most relevant during
each stage of the patients’ journey.
Health analytics also allow healthcare
organizations to be more proactive.
With the right patient and consumer
data, health systems can leverage
predictive models to identify ailments or
conditions that may affect patients and
offer strategies to mitigate risk, ultimately
driving engagement.
3. Improve population health
outcomes. Health analytics are an
essential factor in achieving triple
aim, which the Institute for Healthcare
Improvement defines as “improving the
experience of care, improving the health of
populations, and reducing per capita costs
of healthcare.” As previously discussed,
health analytics can improve the care
W W W. I N F O R M S . O R G
experience by consolidating patient data
from a variety of sources, making it easier
to engage patients, deliver personalized
care and predict a patient’s future needs
to proactively help them.
An example about flu season
summarizes how health analytics can
improve the health of populations overall.
Say it is flu season, and healthcare
marketers want to make sure the
population gets their annual flu shot to
reduce widespread illness. Analytics
allows organizations to determine which
patients are at highest risk for the flu, who
has or has not received a flu shot, analyze
which messages are more or less effective
for different groups of people (based on
historical data), and make predictions
about which communication channels will
result in the best response rate from each
group (i.e., predictive modeling).
Healthcare marketers can then
create a targeted campaign to reach
these patients with information about the
flu shot, which clinics offer it nearby and
other information that may urge them to
get the shot.
Finally, health analytics reduces the
cost of healthcare by eliminating the need
for expensive generalized marketing
efforts and by encouraging patients to
be proactive about their health. By using
analytics, healthcare marketers can
A NA L Y T I C S
target and personalize patient outreach,
thereby creating more successful
campaigns without wasting time and
money. Health analytics can also reduce
future healthcare costs by helping health
systems provide tailored proactive
communication. This way, providers
can prevent and treat health problems
before they become more serious and
subsequently more expensive.
FINAL THOUGHTS
In most organizations, the majority
of collected patient data goes unused.
This means that most organizations
are not taking full advantage of the
opportunities health analytics provides.
Healthcare organizations need analytics
to harness all available patient data,
boost engagement strategies and
improve population health initiatives.
As a result of data-driven engagement
and population health initiatives, health
systems can maintain a competitive
edge, improve patient retention and
boost health outcomes overall. ❙
Gary Druckenmiller Jr. is vice president and
marketing practice lead at Evariant. He functions
as lead strategist, digital marketing thought leader
and C-level executive sponsor for all of Evariant’s
enterprise clients, primarily focused on advising
health system leadership of opportunistic methods to
improve their digital presence and interactive growth
potential.
N O V E M B E R / D E C E M B E R 2 018 | 45
TR EN DY TE C H N O LO G I E S
Everything you wanted
to know about AI and
machine learning
… but were afraid to ask.
BY JERRY W. THOMAS
he business news is cluttered
T with stories about new artificial
intelligence (AI) and machine
learning (ML) startups. Major
corporations are rushing to set up internal
teams and divisions to exploit AI and ML.
Graduate schools are turning out data
scientists and business analysts with training
in AI and ML.
The big technology companies
are creating AI software and systems.
Marketing executives are rushing to apply
AI and ML to optimize marketing and
46 | A N A LY T I C S - M A G A Z I N E . O R G
advertising processes and programs. It’s
as though the gods have descended from
the heavens to share ultimate truth with
the poor human race.
But what is “artificial intelligence” and
how does it relate to “machine learning?”
What do these terms mean?
“Intelligence,” whatever it is, is presumed
to reside inside of biological creatures
(cells, bacteria, plants, animals, insects,
humans). We don’t think of intelligence
as something possessed by a rock, or a
mineral, or other nonliving substances. All
W W W. I N F O R M S . O R G
The simulation of human thinking is a much higher standard for what AI is, or could be.
biological “creatures” can make decisions,
or choices, that increase their chances of
survival. Let’s define “intelligence,” then,
as an ability to make a decision, to choose
among alternative paths or possibilities in
order to achieve some objective.
“Artificial,” in this context, means
nonliving or nonbiological. So, AI is an ability
of some nonbiological entity (machine,
computer, software, system, algorithm) to
make choices, or trigger actions, that help
solve a problem or achieve an objective.
We’ll come to “machine learning” later.
It is possible that AI devices have
been around for a very long time. The
spring-loaded “trigger” snare might be
considered one of the very earliest AI
devices. A tree limb is bent to create stored
energy, a trigger device is activated by
A NA L Y T I C S
Source: ThinkStock
the movement of an animal, and a wire or
rope jerks and entraps the animal. Spring
snares date back at least 10,000 years
(and more likely 25,000–50,000 years),
but their origins are lost in the mist of time.
This might be stretching the concept of AI
a bit, but I do think these types of devices
meet the above definition of AI or come
pretty close.
The first definite mechanical AI devices
could be credited to Christiaan Huygens,
the famous Dutch scientist and mathema-
tician. He perfected the pendulum as a
device to improve the accuracy of clocks
in 1656, and he invented the centrifugal
governor for use in windmills around the
same time.
James Watt applied Huygens’centrifugal-
governor concept to his revolutionary steam
N O V E M B E R / D E C E M B E R 2 018 | 47
AI & M AC H I NE LE A R NI N G
engine in 1788 to control its speed. Both
of Huygens’ inventions (pendulum clock
and centrifugal governor) meet the stated
definition of AI. They are nonbiological.
They make decisions, or trigger actions,
that regulate the measurement of time or
the run speed of a windmill or engine. These
decisions are made automatically – without
human or other biological intervention.
The beginning of modern artificial
intelligence, as it is now commonly
thought of, traces its origins to the
development of computers during and
following World War II and the possibilities
spawned by those machines. The arrival
of these powerful machines gave rise to
much thinking about what intelligence is
and whether machines might be able to
think “like humans think.”
Virtually all computer languages and
programs, with their ability to compare
variables and values and change the
flow of logic to achieve some objective,
or trigger certain outputs, meet the above
definition of AI. These programs are
nonbiological and make “decisions” to
achieve objectives.
The definition of AI, however,
continues to evolve and expand. The
current definition and understanding tends
to mean machines (broadly defined) that
simulate or mirror human thinking. The
simulation of human thinking is a much
higher standard for what AI is, or could be.
48 | A N A LY T I C S - M A G A Z I N E . O R G
The recognition and translation of human
speech into text is a good example of
AI-derived models that closely simulate
a human’s mental capability. Image and
pattern recognition are, likewise, human
feats that AI can increasingly derive
models to mimic. Voice-to-text translation
and image recognition, however, are only
the tips of the iceberg.
AI is, or will be, applied to developing
and improving models to perform medical
diagnoses, conduct legal research, do
data mining and predictive analytics,
analyze business processes, detect
fraud, predict market trends, forecast
sales and so on. The possibilities are
endless.
MACHINE LEARNING: THE LATEST
ITERATION OF AI
Machine learning is closely related
to AI, and the two terms are often used
as synonyms. With machine learning,
computers can “teach themselves” how
to simulate processes and decisions. It is
the arrival and development of machine
learning that offers such great promise
and hope.
If computers can “program” themselves
or create solutions, then AI can be applied
to an array of problems and processes
very economically and very quickly.
Machine learning is the linchpin
technology that could open up many
W W W. I N F O R M S . O R G
new applications and allow AI to spread
rapidly. So, what is machine learning and
how does it work?
Machine learning requires some goal
or objective; that is, a dependent variable
(or multiple dependent variables). The
more narrow and specific the dependent
variable(s), the greater the chance(s) that
machine learning can derive a formula,
equation or mathematical algorithm
that helps optimize (or maximize) the
dependent variable(s).
The dependent variable could be
something as simple as the response
rate to a direct mail promotional offer,
or the dependent variable could be
classifying photos into two groups: those
containing an image of a chair and those
without an image of a chair. Regardless
of the type of dependent variable(s), we
must have some way to determine if the
dependent-variable prediction is better
(or correct) during each iteration of the
model derived by machine learning.
Machine learning also requires a
substantial and relevant database of
independent variables that might predict
or explain the dependent variable(s).

Funding Excellence
in O.R. & Analytics
Saving Lives | Saving Money | Solving Problems

DEVELOPMENT
Enriching our Profession

www.informs.org/donate

A NA L Y T I C S N O V E M B E R / D E C E M B E R 2 018 | 49
AI & M AC H I NE LE A R NI N G
The better the independent database in
terms of completeness, relevance and
accuracy, the greater the chances that
machine learning will be able to build a
good mathematical model.
In the response-rate prediction example,
the independent variables could include
things such as household demographics,
weather data, mail-delivery days and times,
economic data, marketing research data,
and historical details about the various mail
pieces (claims, type of graphics, colors,
length of copy, type font, etc.) used in the
past and the resulting response rates.
The bulk of the work and most of
the costs related to machine learning
revolve around creating a high-quality
database of independent variables with
data for each variable over a substantial
number of cases or over a substantial
period of time. In some instances this
is weeks or months, but in other cases
years or decades. Once this database
is assembled and cleaned, we have
a dependent variable (response rate),
an objective (maximize response rate),
and a database of potential explanatory
(independent) variables with extensive
and clean historical data for each variable
including past results. But, we are not
yet ready to push the “go” button on the
computer.
Machine learning also requires a
computational strategy. Computers are
50 | A N A LY T I C S - M A G A Z I N E . O R G
dumb and thoughtless. Without a strategy,
computers could easily grind away on a
dataset for thousands of years without
achieving anything. There are many
possible models that might be employed
as the machine-learning algorithm,
including: regression analyses, decision
trees, support vector networks, ensemble
models, gradient boosting methods,
neural networks, Bayesian networks and
deep learning.
The computational algorithm could
be one of these statistical techniques,
or it could be combinations of these
techniques (i.e., hybrid models), or it
could be completely different techniques,
but the human mind has to give the
machine-learning system some type of
strategy. There are easily more than 100
existing statistical routines or techniques
that might form this computational
strategy – and no doubt thousands more
to be created in the future.
Now it’s time to put the machine
learning to work. The computer begins
to run calculations following the assigned
strategy or strategies. In each iteration
the model estimates the response rate
(or other outcomes), compares the pre-
dicted outcomes to actual outcomes, and
tweaks the model to improve its predic-
tive accuracy. This iterative process of
model improvement is continuous so that
the predictive model becomes better and
W W W. I N F O R M S . O R G
better over time. Figure 1 shows
a simplified diagram of a ma-
chine-learning system.
One other point: During the
early stages of model develop-
ment, how can the accuracy of
the new machine learning model
be tested? Back when the original
database of dependent and inde-
pendent variables was created,
you were very clever and ran-
Figure 1: A simplified diagram of a machine-learning system.
domly chose a “holdout” data set
(also called a “validation” data set), a ran- determine which photos in the validation
dom subset of the original database. The data set pictured chairs. The machine-
holdout data set allows you to test your new learning model(s) would ultimately be
machine-learning model to see how well it judged by how well each model identified
works. photos with images of chairs.
The actual response rate for this There you have it. Now you can pre-
holdout data set is known, so you have tend to be an expert on AI and machine
a benchmark, a measuring stick, to learning. As is evident, artificial intelligence
determine how good your new machine- and machine learning offer promise for the
learning model is at predicting the actual future, but they are not magical solutions
response rate. In the other example – to all the world’s questions and problems.
classifying photos into those with a picture We must wait a while longer for the gods to
and those without – the same process reveal ultimate truth to the long-suffering
would be followed. The original set of human race. ❙
photos would be divided into a training
Jerry W. Thomas is president and chief executive
data set and a holdout or validation data of Dallas/Fort Worth-based Decision Analyst, Inc.
set, and a huge database of possible (www.decisionanalyst.com), one of the nation’s
oldest and largest marketing research and analytics
predictive or explanatory variables would firms. Decision Analyst serves the Fortune 500 and
be created. similarly significant companies, governments and
organizations around the world. Thomas welcomes
Human judgment would determine comments and suggestions. He may be reached by
which photos contained chairs in the train- email at jthomas@decisionanalyst.com or by phone
at 1-817-640-6166.
ing data set, and human judgment would

A NA L Y T I C S N O V E M B E R / D E C E M B E R 2 018 | 51
MARKE T D OM I N AT I O N
Power BI vs. Tableau
Data visualization platforms battle to be ‘king of the hill’
BY DWIGHT TAYLOR
a kid, we used to play a
As game on the playground
called “king of the hill.” We
played it in the winter when
the playground was plowed, leaving large
snowbanks at its perimeter. Someone
would race to the top of the snowbank and
claim to be king, willing to take on all who
dared to take him down. In our bulky win-
ter coats and gloves, we would attempt to
push, pull, drag and tackle the king off the
summit so we could lay claim to the throne.
The game may no longer be a mainstay
of the playground, but it’s played frequently
in corporate America where one company
dominates a market until another comes
along and attempts to take it down.
Tableau is finding itself in that position.
The data visualization platform has been
52 | A N A LY T I C S - M A G A Z I N E . O R G
the industry’s gold standard for nearly
two decades. For the sixth straight year,
Tableau is classified as a leader in the
Gartner Magic Quadrant for Analytics and
Business Intelligence Platforms.
Similarly, Gartner recognizes Tableau’s
closest competitor – Microsoft’s Power BI –
in the leadership quadrant (there are only
three with Qlik being the third). According
to the most recent Magic Quadrant study,
Tableau and Power BI are equally on par in
their ability to execute. However, in 2016,
Power BI leaped past Tableau in the com-
pleteness of its vision.
I am a firm believer that Tableau is
a much better platform than Power BI,
but I also believe Tableau has to keep a
close eye on its competitors or risk hav-
ing Power BI become king of the hill.
W W W. I N F O R M S . O R G
 The vast majority of
data analysts I engage
firmly believe that Tableau
is the better platform. It’s
not even a contest in their
eyes. I find it almost sur-
real that nobody believes
Power BI is a threat to Tab-
leau. I liken it to the 2016
presidential election where Is Power BI a threat to Tableau’s “king of the hill” throne?
all the pundits and pollsters
were handing the White House keys to Hill-
ary Clinton. As we now know, the experts
were wrong.
How did we get here?
THE RISE OF TABLEAU
In 2003, a small unknown company
from the San Francisco Bay Area came
into the data visualization market with a
tool that could help the most nontechni-
cal person see and understand data with
ease. At the time Tableau debuted, the
data market was owned by goliaths like
IBM, Microsoft, Oracle, SAP and others.
Tableau’s objective was to disrupt busi-
ness intelligence incumbents with tech-
nology that made the democratization of
data and self-service analytics possible
without dependence on IT.
Tableau had a “land-and-expand”
sales strategy. It targeted selling its data
analysis visualization tool to the person
A NA L Y T I C S
at a company who had the biggest pain
point. It often was a data analyst who had
to synthesize files and data from a myriad
of teams and divisions – an exhausting
task. From that single initial sale, Tableau
expanded its footprint inside each custom-
er’s company, and it didn’t take long for it
to become a disruptor to the Goliaths.
POWER BI’S ASCENT TO
PROMINENCE
Microsoft took a different approach.
First, Microsoft targeted data analysts
who were most comfortable using its
Excel spreadsheets. Second, Micro-
soft incorporated smart analytics into its
strategy. In the early 2000s, Excel was
the No. 1 data visualization, data dis-
covery and data analysis software pro-
gram in the world so it was a no brainer
for Microsoft to target those users. Be-
tween 2006-2012, Microsoft sold add-on
N O V E M B E R / D E C E M B E R 2 018 | 53
POW E R B I V S . P LAT E AU
software that allowed analysts to give a
different look and feel when presenting
data visualization.
Simultaneously, Microsoft was pouring
millions of dollars into its own standalone
product and unleashed Power BI in 2013
to go head-to-head with Tableau. The
Power BI black box offering included
Microsoft Excel add-ins, SQL Server
Report Builder components, machine
learning and artificial intelligence. The
first release of the tools fell far short of
competing with Tableau, and thus was
ignored by the king of the hill.
From inception, Power BI was present-
ed with the perception that it was the more
cost-effective data analysis and visualiza-
tion tool. That perception has caught on,
and the majority of customers I’ve spoken
with believe Power BI is cheaper than
Tableau. Microsoft has done a masterful
job masking the overall cost of its Power
BI platform, especially for on-premise de-
ployments. Perception can trump reality.
MICROSOFT POWER BI:
AGILITY IS STRENGTH
Power BI is a de facto option for
companies with Microsoft platforms like
Office 365, Azure and Microsoft Excel
that have Enterprise SQL Server agree-
ments for reporting. According to Gart-
ner’s Magic Quadrant respondents, the
54 | A N A LY T I C S - M A G A Z I N E . O R G
platform is easy to use and has a free
option, as well as an attractive entry
point for the Power BI Pro version. How-
ever, this version requires that data are
stored on Microsoft Power BI servers in
the cloud. Many large companies and
enterprises are reluctant to store data in
the cloud, so the only viable option for
prospective Power BI customers is the
more expensive one.
“Microsoft’s customer reference
scores place it in the top quartile for ease
of use, with 14 percent of customers cit-
ing this as the main buying criterion,” ac-
cording to the February Magic Quadrant
comparison. The report states that “BI’s
reference scores also place it in the top
quartile for visual appeal. Winning cus-
tomers within the first few minutes has
been part of Microsoft’s ‘five by five’ strat-
egy – five seconds to sign up and five
minutes to wow the customer.”
It does all of that and more.
Power BI’s automated analytics, chart-
ing, custom visual platform and pre-built
dashboard templates for products such as
Google Analytics and Salesforce give it an
interesting market position when placed
next to Tableau, the current king of the hill.
However, Power BI’s highly technical na-
ture of the data models is a barrier to entry
for data analysts unfamiliar with complex
data modeling concepts.
W W W. I N F O R M S . O R G
EASE OF USE AND SELF-SERVICE
ANALYTICS IS GREAT, BUT . . .
Tableau also comes in three ver-
sions – free, low-cost cloud-based and
enterprise premium, allowing users to
see and understand their data. Tableau
specializes in the art of muting the com-
plexity of the software and intelligently
guiding the user’s attention to what’s
most important, the data visualization.
Tableau is ideal for the nontechnical
data analysts who need to create in-
teractive data visualizations. The free
version is a fully functional version of
Tableau Desktop with data volume and
data connectivity limitations.
With nearly a quarter century of busi-
ness intelligence and programming expe-
rience, what I have witnessed and hear is
that data analysts love Tableau.
Tableau earns impressive customer
reference scores, with 94 percent of cus-
tomers scoring the platform as “excellent,”
according to the Magic Quadrant report,
and 55 percent “using it to empower cen-
tralized teams to provision content for con-
sumers in an agile and iterative manner.”
Additionally, 64 percent use it to enable
decentralized analysis by business users,
according to Gartner.
Tableau invested heavily in integra-
tions and connections to a wide vari-
ety of data sources. Its strong ability to
A NA L Y T I C S
obfuscate the complexity of data model-
ing and data integration remains one of
its top selling points.
With its solid customer scores and a
continued focus on reducing the user’s
time to insight, no question that Tableau
is currently king of the hill, but Power
BI’s “five by five” strategy has this of-
fering making its way to the pinnacle of
the hill.
THE POWER BI COST PERCEPTION
When Tableau brought on Adam
Selipsky as its CEO in 2016, the former
Amazon web services executive changed
the pricing model to a subscription-based
model. The pricing conversion reduced
the entry price point, which made Tableau
a plausible option for companies where
high initial costs were a barrier.
I use and help clients achieve their
objectives with both platforms. They are
equally impressive and deserve to be
the top two leaders in the Gartner Magic
Quadrant for Analytics and Business
Intelligence. No other platform is even
close.
Now let’s look at the perception sur-
rounding cost.
In Figure 1, I’ve presented a side-by-
side, on-premise deployment cost com-
parison of the two platforms for 50 users
(10 content creators and 40 content
N O V E M B E R / D E C E M B E R 2 018 | 55
 POW E R B I V S . P LAT E AU

Figure 1: Cost comparison of the two platforms for 50 users.

ASSUMPTIONS:
1. Labor costs not included (consulting, solution architects, etc.)
2. On-premise server deployment is a customer requirement
3. For high availability and redundancy, the customer has requested a 3-server cluster
4. No vendor discounts have been applied
5. Subscription pricing (not enterprise agreements) is preferred by the customer
6. Customer requirements do not include embedding needs
7. Server hardware and operating system licensing costs not included
When you crunch the numbers, Power BI is clearly the more expensive option for an on-premise deployment. Not to mention
the fact that the cost grid does not include the cost of the SQL Server cluster necessary for an on-premise highly available
Power BI Server deployment. I encourage prospective data visualization customers to also take a look at the TrustMap on
TrustRadius; written by platform end users.

consumers). Please make note of the as- TABLEAU MAKING MOVES

TO STAY AHEAD
sumptions following the graphic.
Smart analytics is the incorporation
When you crunch the numbers,
of machine learning and artificial intel-
Power BI clearly is not the cheaper op-
ligence into platforms to automate data
tion, and the previous analysis does not
prep, data analysis, data visualization
include the cost of the SL Server clus-
and data driven decisions. As king of
ter necessary for an on-premise, highly
the hill, Tableau has established itself
available Power BI Server deployment.
as the self-service analytics disruptor to
I encourage prospective data visualiza-
every incumbent business intelligence
tion customers to also take a look at the
vendor. However, smart analytics is the
TrustMap on TrustRadius, which is writ-
disruptive force on the horizon. The
ten by platform end-users.

56 | A N A LY T I C S - M A G A Z I N E . O R G W W W. I N F O R M S . O R G
vendor that successfully delivers smart
analytics will ultimately emerge as king
of the hill.
On the playground, retaining king of
the hill status demanded vigilance and
a healthy dose of speculation. Vigilance
because one slip could end a reign; spec-
ulation because identifying formidable
contenders from mere wannabes doesn’t
always yield expected results. Tableau
finds itself at this exact point of decision.
As a consultant on the ground with a
wide variety of clients, I believe Power
BI is that formidable contender. With a
clear edge in ML & AI analytic capabili-
ties, Power BI has its sights on the throne
. . . and Tableau is playing catch-up.
Tableau needs to do three things well:
1. Flawlessly and quickly integrate
ML & AI analytic capabilities into
its platform with the August 2017
acquisition of ClearGraph and the
August 2018 acquisition of MIT
startup Empirical Systems. Also keep
a close eye on ThoughtSpot.
2. Arm every sales resource with an
ironclad total cost of ownership
narrative making it crystal clear that
Tableau ultimately is cheaper than
Power BI.
3. Don’t slip, trip or falter while
executing on the existing Tableau
product roadmap.
A NA L Y T I C S
Power BI is a good example of how
marketing impacts perception, and how
that perception can create a window of time
long enough to reshape reality. I believe
Tableau clearly trumps Power BI when it
comes to those telling stories with data.
For now, Tableau continues to reign
as king of the hill but for how long? ❙
Dwight Taylor is the founder & CEO of VIZYUL
LLC, a Tableau and Power BI solutions provider.
VIZYUL helps companies understand self-service
Tableau and Power BI usage patterns and optimize
their data analytics environments. He has more than
20 years of experience in delivering data discovery
and data visualization services in the healthcare,
banking, non-profit, scientific research facilities
and technology industries. Connect with Dwight on
Twitter@Vizyul.
RESOURCES AND REFERENCES
1. https://www.tableau.com/learn/webinars/smart-
analytics-data-decisions
2. https://www.tableau.com/learn/whitepapers/tableau-
advances-smart-analytics
3. https://www.tableau.com/about/blog/2018/2/tableau-
named-leader-gartner-magic-quadrant-six-years-
row-82534
4. https://www.tableau.com/reports/gartner
5. https://www.tableau.com/about/press-releases/2017/
tableau-acquires-natural-language-query-startup-
cleargraph
6. https://www.tableau.com/about/press-releases/2018/
tableau-acquires-empirical-systems
7. https://info.microsoft.com/ww-landing-gartner-bi-
analytics-mq-2018-partner-consent-test.html
8. https://powerbi.microsoft.com/en-us/blog/gartner-
recognizes-microsoft-as-a-leader-in-analytics-and-bi-
platforms-for-11-consecutive-years/
9. https://www.trustradius.com/business-intelligence
10. https://www.trustradius.com/static/about-trustradius-
scoring#question2
11. https://www.forbes.com/sites/petercohan/2018/09/08/
thoughtspot-raised-145m-more-to-target-tableau-as-
it-barrels-to-ipo/#393f3deb474f
12. https://powerbi.microsoft.com/en-us/calculator/
13. https://www.tableau.com/pricing/teams-orgs
N O V E M B E R / D E C E M B E R 2 018 | 57
SO FTWARE S U R VE Y
Decision analysis
Biennial survey demonstrates continuous advancement of
vital tools for decision-makers, managers and analysts.
BY JUSTIN AMOYAL
2018, the introduction and
IN rapid adoption of once fu-
turistic technologies has
become the norm, unfor-
tunately and frequently leaving the numer-
ous hours of design, testing, fabrication
and analysis – most importantly decision
analysis ­ without much or any consider-
ation. Prior to becoming realities, these
technologies were items of imagination,
so much so that they were utilized in mov-
ies to add to the “futuristic” element. Going
back to “Space Odyssey” (2001), Face-
time-like video and Siri-like artificial intel-
ligence were introduced. Facebook took a
page out of “The Minority Report” (2002)
58 | A N A LY T I C S - M A G A Z I N E . O R G
by using individual’s interests and history
to drive personalized advertisements. Per-
haps the most important technological ad-
vancement of our time, predicted in “The
Net” (1995), was online pizza ordering and
delivery [1].
These technologies have driven the
development of new fields and domains
to study, and implicitly with them, experts,
models, data and analytics to continuously
advance them. Yet, it is rare that a product or
system today solely requires the expertise
of only one individual or domain. The Ency-
clopedia of Information Science and Tech-
nology expands on this by explaining a core
problem is bringing multiple professional
W W W. I N F O R M S . O R G
communities together: “More
specifically, to design effec-
tive serious games, the ex-
pertise of educators, content
experts, and learning experts
will need to be utilized on top
of all other experts who are
typically involved in game
design” [2].The complexity
Project teams frequently struggle with optimally combining the inputs from stakeholders,
inherent with the desired per- experts and models to reach decisions.
formance of today’s systems Source. ThinkStock

requires the combined expertise of many,
thus continuously growing the need for col-
laboration amongst multiple experts when
attempting to make decisions.
Project teams frequently struggle
with optimally combining the inputs from
stakeholders, experts and models to
reach decisions. It is typical for the model
builders to side with the model results,
the experts to believe in their research,
and the stakeholders to strive to advance
their portion of the project or their lead-
ership’s interests. What is the remedy to questionnaire sent to vendors who previ-
effectively managing these conflicting ob-
jectives? In addition to the complexities
associated with the problem itself, “an
effective decision analyst must under-
stand the challenges of decision-making
in organizations, the mathematical foun-
dations of decision analysis, and the soft years, vendor responses are provided ver-
skills required to work with the decision
makers, stakeholders, and experts” [3].
What tools can we use as thought
leaders, decision-makers, managers and
analysts to define these decision frame-
works in our own projects? The results
of this year’s biennial decision analysis
software package survey are meant to
assist in providing the tools and numer-
ous capabilities available to us.
THE SURVEY
In line with previous years, the 2018
survey was made available via an online
ously participated or those recommended
by staff. Vendors who did not respond by
the deadline can still provide their soft-
ware’s information by submitting a ques-
tionnaire and it will be added to the online
version of the survey. Just as in previous
batim and do not imply quality or cost ef-
fectiveness. Rather, the list serves to raise

A NA L Y T I C S N O V E M B E R / D E C E M B E R 2 018 | 59
SO FTWARE S U R VE Y
awareness of the variety of tools avail-
able. Approximately 60 multiple choice
and short answer questions made up this
year’s questionnaire. The primary focus
was to define the current capabilities and
any updates or advancements associated
with each software package, enabling a
high-level perspective on decision analy-
sis software capabilities to our audience.
The primary goal of the software sur-
vey was to ask questions focused on the
decision analysis capabilities of each tool.
These include but are not limited to mul-
tiple competing objectives, probabilistic
dependencies, stakeholder collaboration,
value functions, strategy tables, elicitation
and decision algorithms. Moving forward,
vendors were asked questions to define
the usability of their applications. To do so,
they provided feedback on their software
interfaces, visualization capabilities, mod-
el and data protection, user interfaces,
and other usability related features. Types
of training offered, as well as the recog-
nized certifications associated with each
software tool, were also investigated in the
2018 survey.
2018 RESULTS
This year’s survey includes 28
software packages from 19 different
vendors, domestic and international, who
responded by the deadline. Pricing for
licenses varied, ranging from free products
60 | A N A LY T I C S - M A G A Z I N E . O R G
to a max of $100,000, depending on the
user/license type and elements of the
software package. Specific industries and
markets for these software applications are
widespread, including defense (military,
aerospace and shipbuilding) healthcare
(pharma, hospitals and clinics), energy-
related domains (petroleum and nuclear
energy), as well as many others.
Decision analysis capabilities: Each
of the decision analysis-focused capabili-
ties proved to be available by at least one
of the software tools reviewed. Uncertainty
existed in every tool, yet implementation
likely differed within the applications, and
it is therefore worth investigating to find
the one best fit for an individual project.
Unfortunately, there were two minimally
available features, an evidential reasoning
(e.g., Bayesian belief networks) capabil-
ity and portfolio decision-making, which
do not exist in 79 percent and 48 percent,
respectively, of the tools reviewed. Value
functions/scores and risk tolerance existed
in 75 percent of the tools, while strategy ta-
bles were available in 43 percent of them.
Seven of the tools offered AHP, while 13
offered MODA/MAUT.
Usability: Usability features were fairly
consistent among the vendor responses.
XML integration and API (embedded
decision support system) proved to be the
W W W. I N F O R M S . O R G
capabilities most limited. Yet the inclusion
of an API has increased since 2016 from
50 percent of the tools to 68 percent
this year. Only eight of the tools offered
simultaneous data input and viewing.
When considering data visualization,
it is difficult to rely solely on the decision
analysis software tool. Inherent in the name,
these are decision analysis tools and there-
fore they provide the ability to output deci-
sion analysis-related visualizations such
as graphical sensitivity analysis, expected
value tornado diagrams, decision trees and
influence diagrams, but stakeholders and
WHICH ORGANIZATION HAS THE BEST O.R. DEPARTMENT IN THE WORLD?
The Institute for Operations Research and
the Management Sciences annually awards
the INFORMS Prize for effective integration
of operations research/management
science (OR/MS) and advanced analytics
into organizational decision making. The
award is given to an organization that has
repeatedly applied the principles of OR/MS
and advanced analytics in pioneering,
varied, novel, and lasting ways.
A NA L Y T I C S
decision-makers are usually not decision
analysts. Thus, it is important to consider
the exporting capabilities of these tools, par-
ticularly related to the export of data which
existed in 86 percent of the tools. Exporting
the data provides the ability to develop ad-
vanced visualizations, customizable to the
desires and experience of project stakehold-
ers, which may better convey results than
the standard visualization existing within the
software tools.
Licensing and training: The majority
of the vendors (80 percent) reported that a
call for
nominations
DEADLINE TO APPLY IS DECEMBER 1, 2018
FOR MORE INFORMATION PLEASE VISIT
www.informs.org/informsprize
CONTACT:
Dr. Mark Gallagher
USAF Studies & Analyses and Assessments
2019
571-256-2111 or email: markgallagherJMJ@gmail.com
TELL US WHY THE TITLE SHOULD BE YOURS.
N O V E M B E R / D E C E M B E R 2 018 | 61
SO FTWARE S U R VE Y
limited use (run-time) or demo version of the
tool is available. Nearly all of the products
are available for purchase for educational
(96 percent) and/or commercial (100
percent) use. As expected, many of these
tools are offered free or at a discounted
rate for educational use. The number of
tools offering enhanced/high performance
capabilities with these software packages
showed a significant increase since 2016,
from 35 percent to 75 percent this year.
Pricing for these advanced features
varies significantly, typically based on the
client’s needs and options selected. Mul-
tiple options for training were offered in re-
lation to the software tools, mostly offered
by the vendors themselves (79 percent).
Online resources including training, tutori-
als and open source collaboration was of-
fered for 75 percent of the tools, a nearly
25 percent increase since 2016.
CONCLUSION
This year’s decision analysis software
survey demonstrates that software capa-
bilities continue to advance, enabling users
to conduct decision analysis regardless of
industry, interfacing and elicitation needs,
and desired visualizations or operating
systems. When solving difficult problems,
it is important to understand the resources
available, interfacing requirements, desired
decision analysis techniques to be utilized,
required outputs and other problem-specific
62 | A N A LY T I C S - M A G A Z I N E . O R G
elements to select the tool that will best fit
the analysis.
It is no surprise that every tool reviewed
in this survey will not be the “perfect fit” for
each decision analysis problem. Thus, the
percentages mentioned earlier associated
with nonexistent features is not detrimen-
tal to a tool’s potential performance for a
specific project. A project on a tight timeline
may not have the opportunity for iterative
analysis; therefore, a tool offering this ca-
pability would provide no additional value.
In closing, this year’s software survey
includes an impressive list of applications
available to decision professionals, and
the survey results can assist profession-
als in selecting the best decision analysis
package for their particular needs. ❙
Justin Amoyal (jamoyal@innovativedecisions.
com) is an analyst with Innovative Decisions, Inc.,
and a member of INFORMS. A version of this
article appeared in OR/MS Today, the membership
magazine of INFORMS.
REFERENCES
1. Pahle, Rebecca, 2016, “15 Movies That Predicted the
Future,” Mental Floss, Sept. 15, mentalfloss.com/
article/86080/15-movies-predicted-future.
2. “Gaming,” 2015, Encyclopedia of Information
Science and Technology, Information Science
Reference, pp. 3,291-3,291.
3. Parnell, Gregory S., and Bresnick, Terry A, 2013,
“Introduction to Decision Analytics” in “Handbook of
Decision Analysis” (by Parnell, Bresnick, Steven N.
Tani and Eric R. Johnson), John Wiley & Sons.
SURVEY DIRECTORY AND RESULTS
For a directory of decision analysis software vendors
who participated in this year’s survey and the
complete survey results, click here.
W W W. I N F O R M S . O R G
INFORMS JOURNAL ON
APPLIED ANALYTICS
formerly INTERFACES

Interfaces has a new name

in 2019 – Same Great
Content, Brand New Look!
Interfaces is changing its name to
INFORMS Journal on Applied Analytics to
more accurately reflect its editorial content
for an ever-expanding audience. For almost
half a century, our journal has demonstrated
the value of O.R., and management science
by highlighting practical applications in
business, military, healthcare, nonprofit, and
public sector communities, and others that
shape our world. It also featrures world-class
competitions, as well as short practice
summaries that highlight the experiences
of students and entrepreneurs.

For more information please visit:

https://pubsonline.informs.org/journal/inte

Volume 49, Number 1

January–February 2019
CO N FERE N C E P R E V I E W

WSC 2018: ‘Simulation

for a Noble Cause’
WSC is the central meeting place for simulation researchers, practitioners
and vendors spanning all disciplines and sectors.

The Winter Simulation Conference (WSC) is the

premier international forum for disseminating recent
advances in the field of dynamic systems model-
ing and simulation and has been for 50 years (2017
was the 50th anniversary!). In addition to a technical
program of unsurpassed scope and quality, WSC is
the central meeting place for simulation researchers,
practitioners and vendors spanning all disciplines and
working in industry, government, military, service and
academic sectors. WSC 2018 will be held Dec. 9-12
in Gothenburg, Sweden, at the Swedish Exhibition
and Congress Centre. The theme for WSC 2018 is
“Simulation for a Noble Cause.”
One great aspect of the simulation field is the wide
range of interests it attracts. The academic tracks include
analysis methodology, modeling methodology, simula-
tion optimization, agent-based simulation and hybrid
BY BJÖRN JOHANSSON simulations. The applied uses of simulation showcased
at the conference include homeland security enterprise,
modeling and analysis of semiconductor manufacturing

64 | A N A LY T I C S - M A G A Z I N E . O R G W W W. I N F O R M S . O R G
(MASM), healthcare applications, logistics,
manufacturing applications, aviation mod-
eling and analysis, networks and commu-
nications, military applications, gaming,
complex, intelligent, adaptive and autono-
mous systems (CIAAS), project manage-
ment and construction, environmental and
sustainability applications, simulation stan-
dards and reproducibility, and cyber-physi-
cal systems.
Additionally, the conference will in-
clude introductory and advanced tutorials,
simulation education, case studies, Ph.D.
colloquia, poster sessions and vendor tu-
torials. Whatever your specialty, WSC will
have something for you.
Simulation has been found useful for a
range of scientific, engineering and busi-
ness applications as evident by the papers
presented at Winter Simulation Confer-
ences over the past 50 years. Simulation
has been recently employed to help noble
causes, but such efforts have received lim-
ited attention. The 2018 conference seeks
to highlight applications of simulation for
these noble causes in addition to continu-
ing to report leading developments and ap-
plications in other fields. The track sessions
will feature uses of simulation in efforts
analyzing and addressing issues facing
humanity including, but not limited to, re-
ducing poverty and world hunger, social
causes, social problems, improving natural
environment and disaster response.
A NA L Y T I C S
Dr. Jean Baderschneider, CEO of
the Global Fund to End Modern Slav-
ery, will deliver an opening keynote on
applying simulation for analyzing so-
cial issues such as human trafficking, a
reflection of the theme for WSC 2018,
“Simulation for a Noble Cause.”
The military keynote speaker is Dr.
Reiner Huber, emeritus professor, Univer-
sity of the German Federal Armed Forces,
Munich, Germany. He will speak about
the role of modeling and simulation from
a WWII recollection, as well as onward to-
ward perspectives of today. On a related
note, the homeland enterprise keynote will
be delivered by David M. Wulf, acting dep-
uty assistant secretary, U.S. Department of
Homeland Security, who will discuss secu-
rity and resilience of critical infrastructure.
The “Titans of Simulations” plenary
talks feature two excellent speakers:
Dr. Peter Frazier, associate professor
at Cornell University and data science
manager at Uber, and Dr. Russell Cheng,
emeritus professor at the University of
Southampton.
Professor Stewart Robinson from
Loughborough University will pres-
ent the Ph.D. colloquium keynote on a
topic that will challenge Ph.D. students:
building and using the right model for
the right purpose.
WSC 2018 also incorporates the
Modeling and Analysis of Semiconductor
N O V E M B E R / D E C E M B E R 2 018 | 65
CO N FERE N C E P R E V I E W

Manufacturing (MASM) conference-with- presentations to the meetings of various

in-a-conference. This conference features
a keynote by Professor Lars Mönch, Uni-
versity of Hagen, Germany, who will talk
on “Reflections on Reference Modeling,
Simulation Testbeds and Reproducibility,”
which is timely because testbeds and dig-
ital hubs are now increasingly populating
our world.
The Winter Simulation Conference is
designed for professionals and academ-
ics from all backgrounds and across a
broad range of interests. From the ex-
tensive group of exhibitors and vendor
professional societies and user groups,
along with various social gatherings, WSC
provides all attendees the opportunity
to become involved in the ever-expand-
ing activities of the international simula-
tion community. We hope that you have
the opportunity to join us in Gothenburg,
Sweden, to actively participate in continu-
ing the great tradition of WSC.
For more information, visit http://win-
tersim.org/2018/. ❙
Björn Johansson is general chair of WSC 2018.

2019 INFORMS
HEALTHCARE CONFERENCE
July 27–29 | Cambridge, Massachusetts

The goal of INFORMS Healthcare 2019 is to bring together academic researchers in

“healthcare analytics” and industry stakeholders who are applying this research to
share information and improve the delivery of effective healthcare. The health sector is
catching up fast but still lags behind other industries such as airlines, hotel chains, and
retail in the adoption of systematic and quantitative analysis used in decision support,
and this conference will help to bridge the informational gap. The conference aims to
bring a cross-cultural view of healthcare systems worldwide and the research and
applications designed to improve these systems in areas such as:

• Medical decision making • New methods associated with defining value

in healthcare
• Computational optimization
• Incorporation of O.R. & industrial engineering
• Patient wait time guarantees in healthcare delivery
• Effective use of electronic
• Intersection of healthcare & public health to
health records
provide a more “population” perspective on value
• Delivering healthcare in
developing countries

Abstract Submission Deadline: May 10, 2019

meetings.informs.org/healthcare2019

66 | A N A LY T I C S - M A G A Z I N E . O R G W W W. I N F O R M S . O R G
Submit an
Abstract

REGISTER BY
DECEMBER 31, 2018
Using promo code
“AUSTINHOTEL”
to be entered into a drawing for a
COMPLIMENTARY THREE NIGHT STAY
during the conference at the JW Marriott Austin

EXHIBIT & SPONSOR

AT ANALYTICS 2019
For details, contact Olivia Schmitz at
Olivia.Schmitz@informs.org

meetings.informs.org/analytics2019
Austin, Texas | April 14–16, 2019
FIVE- M IN U T E A N A LYST

World economic
growth
I am not an economist, but I am interested in the
performance of the economy. At work last week, my
colleague wanted to know if we could build a model
How to ‘squeeze’ a bit that considered the growth and interaction of some of
more knowledge out of a the world’s biggest economies, based on data from
limited data set. the USDA Economic Research Office. Specifically, he
wanted to see if he could extrapolate the data that
was in the set out to the future. A plot of the data is
shown in Figure 1.

MODELING THE WORLD ECONOMY

BY HARRISON SCHRAMM
Disclaimer: serious work is done in this area by
economists, which I am not!
In order to build a model of this process, I de-
cided to take a differential equation approach and fit
the parameters by minimizing the residual deviance.
This uses ideas at the crossroads of statistics and
optimization, which I have been fascinated by since
seeing Dimitris Bertsimas’ Philip McCord Morse Lec-
ture at the 2014 INFORMS Annual Meeting. The de-
velopment here follows a general class of differential
equation models, which have specific use cases in

68 | A N A LY T I C S - M A G A Z I N E . O R G W W W. I N F O R M S . O R G
epidemiology (the Kermack-
McKendrick SIR model) and
modeling combat situations
(Lanchester systems). With
these ideas in mind, we pro-
pose a model of the form:

Where X is the vec-

tor representing the world
economies at time t, and
is the (stationary) matrix of Figure 1: Plot of the USDA World GDP data from the U.S. Department of Agriculture. The
GDP growth. Because the data set includes both actual numbers and projections that go to 2030.

data is presented annually,

the natural “scaling” of this
model is 1.
Before we discuss the so-
lution methodology, we show
the model result in Figure 2.

SOLUTION METHODOLOGY
As the charts imply, this
was done in Microsoft Excel,
using the installed Solver. The
NLP formulation is:

Figure 2: Fit of world economic data using the vector model proposed.

s.t. When looking to solve this model,

I originally used a full rank matrix with
values restricted to be greater than
one. My general NLP instinct was that
this would work better, as depicted on

A NA L Y T I C S N O V E M B E R / D E C E M B E R 2 018 | 69
 FIVE- M IN U T E A N A LYST

Figure 3: Two different solution approaches. On the left, the matrix is “full,” with entries restricted to be greater than zero. On the right, the
matrix is upper triangular, with the members unrestricted in sign. While mathematically these approaches are similar, the model on the
right is far more amenable to analysis.

the left side of Figure 3. However, when
it came time to actually solve the opti-
mization problem, the solver worked far
better as an “upper triangular” matrix.
This is both because of fewer variables
in Solver, as well as making the solu-
tion more unique.
using time only as a variable. Because
of the way the problem is postulated,
we know that the endpoints will “fit”
(the problem was posed this way). One
way to think about the performance of
a model like this is to plot the residuals
or “miss” amounts.

ASSESSING PERFORMANCE SUMMARY

This is a very simple analysis In summary, this short article shows
performed with an “adequate” tool how to “squeeze” a bit more knowledge

Figure 4: Matrix of growth /decline variables across the economic dataset. The rows are “from” and the columns are “to,” with the
interpretation that a nation’s own growth is “to itself” (the on-diagonal elements).

70 | A N A LY T I C S - M A G A Z I N E . O R G W W W. I N F O R M S . O R G
 out of a limited data set, which
is always an implied subtask to
any analysis. It was also a fun
way to exercise both differen-
tial equation models as well as
getting MS Excel’s GRG Non-
linear Solver to produce an in-
teresting result. ❙

Harrison Schramm (harrison.

schramm@gmail.com), CAP, PStat,
is a senior fellow at the Center for
Strategic and Budgetary Assessments
(CSBA) and a longtime member of Figure 5: Under-prediction amounts (actual – predicted) for the GDP model. Note the
INFORMS. The author thanks Adam large under-prediction of the U.S. GDP during the 2000s and of China in 2010-2030
Lemon of CSBA for piquing his interest
in this problem. (projected).

2018 DECISION ANALYSIS SOFTWARE SURVEY

October 2018
Volume 45 • Number 5
ormstoday.informs.org APPLICATIONS INVITED FOR A FACULTY POSITION IN
INDUSTRIAL ENGINEERING & MANAGEMENT SCIENCES
Also Inside:
• Proper government We invite applications for a full-time, tenure-track faculty appointment
response to AI issues
at the assistant to associate professor level to begin September 2019.
• Ethics guidelines for
analytics professionals
The search is broad, with a preference for candidates in the areas of
operations and logistics, including service operations management,
• The problem with U.S.
university rankings supply chain management and sharing economy; computational social
• Thomas Saaty: intellectual science and networks; and computational methods in operations
giant’s final work research, including machine learning. Applicants should hold an

Wargaming
earned Ph.D. or be near completion of their doctoral studies with
demonstrated research potential in industrial engineering, operations
research or management science. A commitment to rigorous and

cybersecurity
relevant research is essential.

Submit applications electronically at www.mccormick.northwest-

ern.edu/industrial/career/. Materials to be uploaded include a cover
letter and a curriculum vitae detailing educational background,
How O.R., data science and research and work experience, statement of current and future research
wargaming empower analysts program, a teaching statement and one research paper. Candidates
to do more with less in cyber will be asked to provide contact information for three references on the
conflicts. application site. To receive full consideration, all materials should be
received by December 1, 2018. To be considered for a meeting at
INFORMS, please submit materials by October 26, 2018.

Chair, Faculty Recruiting Committee

Department of Industrial Engineering and Management Sciences
Northwestern University
2145 Sheridan Road, Room C210
Evanston, IL 60208-3119
Check out the October 2018 Issue of ORMS Today facultysearch@iems.northwestern.edu
Now Available at: www.ormstoday.informs.org

A NA L Y T I C S N O V E M B E R / D E C E M B E R 2 018 | 71
 ING
OPERAT GS
AVIN
COST S
O 9%
OF 5% T
OPTIMIZING THE DISTRIBUTION OF REFINED
PETROLEUM PRODUCTS BY TANKERS
A GAMS APPLICATION IN THE PETROLEUM INDUSTRY
Empresa Nacional del Petroleo (ENAP) is the
state-owned petroleum company in Chile. Its
main business is transforming crude oil into
refined petroleum products to be distributed
and sold to large retail companies of the Chilean
industry. The annual volume of sales is around 66
million barrels.
One of the toughest tasks at the Logistics
Department is the maritime distribution of refined
petroleum products. The combinatorial nature of
the decisions that must be made can be modeled
using MIP techniques. The Operations Research
group at ENAP formulated and implemented the
model in GAMS/CPLEX, resulting in a large scale
model.
Given a monthly demand, the decisions that the
GAMS optimization model makes include the
schedule, the stowage configuration, the volume
of each product to be loaded at the starting
WWW.GAMS.COM
Join our Community
www.gams.com/community
www.youtube.com/GAMSlessons
www.facebook.com/GAMSofficial
point and the volume of each product to be
unloaded in each demand port for each trip of
each tanker. This model minimizes the total cost
and satisfies all the operational constraints.
IMPACT AT ENAP
This approach is saving between 5 % to 9 % of
the operational cost for the maritime distribution
system at the company. In addition, ENAP sees an
interesting potential in linking this tool with other
simulation and optimization models in GAMS
that are being used in the company. For instance,
in parallel with this research project a model for
optimizing the distribution of refined petroleum
products by pipelines at ENAP was developed. A
model that integrates the maritime and pipeline
distribution system could be a natural extension
of this work.
SEND US AN
EMAIL
General Information and Sales
GAMS Development Corporation
Phone: (+1) 202 342-0180
sales@gams.com