7 CONSIDERATIONS FOR

DATA LAKE MIGRATION

EBOOK
3 Why large enterprises are investing in data lakes
4 Define your company’s vision
5 Assess your current state
6 Set your goals
7 Understand costs
8 Focus on key capabilities
9 Be prepared
10 Think ahead
11 Conclusion
12 About Snowflake
WHY LARGE ENTERPRISES
ARE INVESTING IN DATA LAKES
Heading into 2020, many companies were already generating
or accumulating more data than they could use. The pandemic
exacerbated this problem by forcing some industries to accelerate
their digitalization efforts and shift more of their operations to the
cloud, which resulted in members of the C-suite having more data
than they knew what to do with.
It’s clear that the ability to harness data and turn it into actionable
insights will determine which companies are competitive over the long
term—and which are likely to fall behind. According to a prediction
from Gartner®, “by 2023, organizations that promote data sharing will
outperform their peers on most business value metrics.”1
The need for data organization and accessibility became more acute
during the COVID-19 pandemic, leading many C-suite executives to
explore comprehensive data migration efforts in order to bring data
together in a single platform where it’s accessible and usable by a
variety of users.
Data lakes are an increasingly popular way of accomplishing this.
Indeed, the global data lake market (estimated at $3.74 billion in
2020) is expected to expand at a compound annual growth rate of
29.9% over the next five years, landing at $17.6 billion by 2026,
according to Research and Markets.2
That said, the decision to fully migrate a company’s data is just a
start, and the work of corralling all of its disparate data sets can be
daunting. At its best, the process can be complex; at its worst, it can
be labor-intensive and messy.
Before diving into a data lake migration, it’s crucial for business and
technology leaders—whether they be CEOs, CFOs, or CIOs—to ask
the right questions up front and make the appropriate preparations.
Given the importance and sensitivity of such an endeavor, it’s also
vital to partner with the right experts.
In the pages that follow, we’ll outline seven parameters every company
should consider when approaching a data lake migration.

DEFINE YOUR
COMPANY’S VISION
WHAT DOES YOUR DATA-DRIVEN IDEAL
LOOK LIKE?
Before getting into the details of how your
company would execute a data lake migration,
step back and explore some of the more
fundamental questions regarding your data use
and its potential. Here are a few to start with:
• Why are you considering this move?
• What is your dream data usage scenario?
• I f neither bandwidth nor legacy systems
were an issue, how would your company best
employ data to empower decision-making
across the organization?
CHAMPION GUIDES
Asking these questions ahead of time will help you
frame the goal and better outline the right solution
for your business. The answers should also help steer
decision-making throughout the process or show that
your data problems are due to issues unrelated to your
data lake.
Having a data role model to emulate can also be
helpful. For example, after a successful data lake
partnership, 95% of the employees at the energy firm
Devon have access to the same complete database
and can make decisions quickly and autonomously.
Is this type of data democratization important to your
company? If so, what do you need to do to get there?
The more leaders can identify their ultimate rationale
for considering a data migration, the closer they will be
to achieving their goals.
4
 CHAMPION GUIDES
ASSESS YOUR
CURRENT STATE

PERFORM A COMPLETE ANALYSIS OF DATA In addition to documenting their company’s current

STORAGE, USAGE, AND SYSTEMS
Once you know where you want to go,
take stock of where you are. For companies
looking to break down silos and pull together
many forms of data via multiple systems and
protocols, this requires some honest scrutiny.
Among the questions leaders need to ask are:
data storage and usage, leaders should have a full
understanding of who in the company has access to
which information, where gaps exist, and who makes
decisions about these hierarchies and rules. Only
then can leaders determine where bottlenecks may
form and how decision-making should be managed
in order to better organize and streamline adoption
after a migration.

• Where is all of our data now?

• How do we use it?
•  hat is holding us back from achieving our
W
data ideal?

5

SET YOUR GOALS
IDENTIFY AND QUANTIFY PAIN POINTS
YOU AIM TO SOLVE
Now is the time to get more specific and a complete data migration. For example:
map out issues you are trying to fix and how
you plan to address them. It helps to put
this in writing in the form of measurable,
achievable results.
One of the main reasons companies invest in
data lakes is to battle “data sprawl.” This refers
to data that is housed all over the place, often
inconsistently, with no single group or leader
having full access to or control over its usage.
Data sprawl results in a limited view of the
customer and business
The more scattered a company’s data is, the harder
it is to act swiftly and with the best intelligence.
A 2021 IDC Technology Spotlight found that typical
data pipelines are processing up to 10 different
types of data coming from up to nine distinct
sources.3 For example, marketers’ efforts to deliver
personalized content, ads, and offers can be stymied
by disjointed data sets that yield incomplete views
of consumers. More broadly, businesses can end up
with incomplete views of their true revenue drivers,
leading them to misinterpret opportunities or focus
on the wrong markets.
Inefficient data pipelines slow access Companies should draft a set of quantifiable, achievable
and querying
The movement of data via temporary or imperfect
pipelines can also contribute to data usage
breakdowns—in part because these makeshift
processes tend to be slow.
Take Scripps Health, for example. By moving to
Snowflake’s platform, Scripps Health achieved a 50%
reduction in full-time equivalent (FTE) staff dedicated
to database administration, and it saved 60% for
software licensing. Previously, users retrieved data
and analyzed it on their own systems, creating data
silos. Now, users stay entirely in Snowflake, and silos
are eliminated.
Decentralized data collection and housing
can create governance challenges
A lack of centralized data control can lead to disjointed
decision-making and security vulnerabilities since
there is no holistic view or rule for managing access.
This can compromise the security of sensitive
customer or business data while also making it
difficult for individuals to do their jobs. IT leaders
require the ability to oversee and make decisions on
data centrally, which siloing naturally inhibits.
CHAMPION GUIDES
KPIs related to the problems associated with data
sprawl. These KPIs can be used to gauge the payoff of
•  roject duration: How long will the actual migration
P
take? Which processes are likely to take longer,
and why?
•  isruption: How much will this migration temporarily
D
cut off critical systems or slow down vital processes?
What workloads will be increased or put on hold?
Will IT be strained during this transition?
•  ost: Various parties will need to be compensated
C
both to complete the migration itself and to steer
operations during any disruptions, including current
employees and contractors. Companies may also
need to pay for additional monitoring or tools to
provide visibility into the migration’s progress.
•  sage: Companies can set usage estimates based
U
on expected workloads, but the time spent with
the data lake and/or the number of users involved
can fluctuate. Companies should take this into
account ahead of time since many may benefit
from a consumption-based pricing model via
which they pay only for what they use. Read more
about the importance of pricing decisions and
why consumption pricing is a good choice on the
Snowflake blog.
6

UNDERSTAND COSTS
ASSESS YOUR POTENTIAL FOR SAVINGS
As part of their assessment, companies should
fully account for current costs of their data
platform. Their list should include software
and server bandwidth the firm is paying for
currently, what siloing or gaps are costing in
terms of lost opportunities—and, of course,
what price a migration will ultimately incur.
Here are five less-obvious costs to include in any
evaluation. These can help clarify the value of a
data lake:
•  xcess capacity: All systems have excess capacity
E
for load spikes and growth, which can be amortized if
merged in a data lake. For a company that maintains
an excess capacity of 20% per system, this could
amount to considerable savings.
• I T, accounting, and engineering time: Resources
that are currently being allocated to support a variety
of silos can be streamlined and merged to greatly
reduce human costs and complexity.
•  raining and user productivity: When you have a
T management, logging, and access control, can seem
diversity of systems, users need time and training
to get up to speed and remain productive on each
separate data silo. This cost is often distributed
across the entire organization.
•  econdary system costs: Integrations with other
S and more centralized. Teams can more easily run
technologies can easily be overlooked but are often
quite expensive. Consolidating the workloads these
many systems support into a single platform can
significantly reduce costs. Quantifying these costs
will depend greatly on the business in question.
•  pportunity costs: Many companies have a “wish
O performing audits and investigations.
list” of things they’d like to do if they had more
complete data sources or potential revenue streams
they could tap into if they had greater data access.
Gauging the value of what you might be missing
out on is challenging yet crucial, as calculating
opportunities is complex. It often requires estimating
not just missed revenue but accounting for potential
delays and bottlenecks.
CHAMPION GUIDES
Remember that moving to a new platform may
present some up-front costs while you are running
multiple systems at once. The benefits of a migration
will increase with each subsequent silo that you
merge. It often helps to start small and expand over
time so that it becomes easier to showcase the
benefits of migration to decision-makers.
Finally, thinking through the worst-case scenarios—
such as a data security breach—is a difficult but
worthwhile exercise. When data is distributed
throughout an organization, the cost of attempting
to secure multiple silos, along with governance
prohibitive. This leads some companies to adopt
a more lax approach, potentially exposing them to
massive harm from a breach, leak, or lawsuit.
In a data lake, management can be much simpler
audits to ensure that data control is being tightly
maintained. They can also identify a breach much
more quickly if it occurs; centralizing everything in
a data lake provides a more comprehensive view of
their data, and there are fewer places to search when
As an example, global investment firm EQT Group
uses Snowflake’s data lake product to “mask”
certain data that includes personal information from
consumers. This enabled far better governance and
compliance, helping EQT Group move quickly to a
privacy-first way of operating and avoid any potential
fines or interest from regulators.
7

FOCUS ON KEY
CAPABILITIES
WHAT DOES YOUR COMPANY NEED IN A
DATA LAKE SOLUTION?
There are numerous providers in the market
touting a range of capabilities under the banner
of a data lake. When vetting potential partners,
it’s important to be aware of the capabilities
that will do the most to address the pain points
you outlined in Section 3—and drive the most
impact for your business.
CHAMPION GUIDES
Here are a few areas to look for: • Concurrency: How many users and divisions will
need to access the data lake? (Bear in mind that
•  now and protect your data: A common problem
K
once data silos are broken down and access is
with data lakes is that they are cluttered with so
simplified, a surprising number of people in your
much data that they become unusable. If you
organization can potentially make use of a data lake.)
don’t know what’s in your data lake, how can you
Ensure you pick a provider who can provide the
confidently control who has access to that data? A
sign-on capabilities, auditing, and, most importantly,
good solution should help you understand what’s
elasticity to ensure the data lake doesn’t degrade as
in your data lake, easily enforce access policies, and
more users are onboarded.
monitor usage to ensure only the right people are
accessing the right data. It should also help you • I nteroperability: Your business will grow and evolve,
generate easy-to-run reports that allow for a better which means that new partners, data systems, and
understanding of usage and currency, enabling tools are inevitable. The platform supporting your
smarter decision-making. data lake should be flexible and adaptable enough
to satisfy your needs as they change.
•  cale: For most companies, it’s reasonable to expect
S
that scale requirements will increase over time. This •  ontinuity: Ideally, companies should not notice
C
makes it important to ensure that the data lake’s any differences in access or performance if there
capabilities can grow with your business but remain is a need to fail over across regions or clouds. In
cost-efficient when operating at a smaller scale. In the event of a disruption, the right platform makes
addition, the scalability of storage and computing failover and failback seamless.
resources needs to be independent so companies
don’t end up paying for more than they need.
•  roductivity: It’s important to consider which types
P
of users in your business will need to interact with
the data lake today or in the future. Are real-time
or batch interfaces required? Which languages
(for example, SQL, Python, Java) do analysts and
developers use and prefer?
8

BE PREPARED
GET READY FOR THE MIGRATION
JOURNEY BY THINKING EVERYTHING
THROUGH
By now, you’ve conducted a deep, far-reaching
examination. You have your data north star,
and you’ve calculated all the pertinent costs
and tradeoffs. You’re convinced that a data lake
will be greatly beneficial to the business. It’s
time to prepare for the actual migration.
CHAMPION GUIDES
Process mapping is a great place to start. The best When going through these questions, it’s helpful to
way to proceed is by asking the right pre-migration think about today, tomorrow, and next year. Initially,
questions: your company’s focus will be on the immediate impact
• How is your data structured today? of centralized storage and accessibility, but you’ll
eventually need to think about interoperability. It’s a
• Where does it live?
good bet that your storage needs will evolve over time,
•  hat are your company’s language
W so anticipate and prepare for such scenarios.
requirements?
Whichever direction your company decides to take,
•  hat is the most crucial workload currently?
W mapping the order of migration steps is crucial. The
What elements do you need to have working exact path a migration takes can shape the data
on Day One?
lake’s future usability. For example, establishing data
•  hich processes are currently being replicated
W security protocols for a migration is crucial for most
and can be streamlined? companies, and thinking through initial use cases and
•  hich processes are no longer necessary and
W requirements should usually happen first.
can be eliminated? The way a migration unfolds, and the order in which
•  ow will your data scientists’ and analysts’
H different functionalities of a data lake are implemented,
upcoming projects be impacted by this change? will have a direct impact on employees—some of whom
Are they still supported? will require new skills and intensive training. Ideally,
such training should be provided in parallel with the
rollout of relevant phases of a data lake adoption.
9

THINK AHEAD
CONSIDER YOUR POST-MIGRATION
ARCHITECTURE AND WHERE YOUR
INDUSTRY IS HEADED
Just as a company is never “done” transforming
into a data-driven organization, data lake
adoption doesn’t end once a migration is
complete. As the past few years have taught
us, industries can change abruptly, consumer
preferences and habits are forever fluid,
and the march of technological innovation
never ceases.
CHAMPION GUIDES
While business leaders look to complete a data lake
migration in a way that best unlocks efficiency and
effectiveness today, it’s likely they’ll find themselves
in different or entirely new categories in a few
short years. This means that leaders should plan
for different eventualities while considering how
their data needs might change over time. For some
helpful direction, take a look at this TDWI Checklist
for companies looking to future-proof their data
lake strategies.
Given the pace of change and innovation, some
companies’ organizational structures—or even their
business models—could be unrecognizable in a few
years. The most successful enterprises will have a
flexible data platform that can adapt to meet their
business needs wherever they land.
10

CONCLUSION
There’s no doubt that executing a data
migration is a major decision and that
choosing the right data lake partner and
strategy can be daunting. But like most major
operational transitions, challenges can be
mitigated with the right preparation. These
seven considerations are designed to help
steer leaders toward data lake success.
CHAMPION GUIDES
It starts with making sure you and your organization’s Learn more about Snowflake data lake solutions
leaders are aligned on why they want to invest in a and how you can streamline a migration project at
migration in the first place. Mission clarity will help snowflake.com/workloads/data-lake
guide the rest of the process. With this in place, leaders
must conduct a comprehensive analysis of their current
data practices, determine where they’d like to make
changes and why, and put lofty but achievable goals
down on paper. Of course, a full examination of current
and future costs is essential.
Once the decision to move forward has been made,
technology leaders and their migration teams/partners
must think through what kind of cloud partner they
need, which criteria and tasks are most important,
and how their needs are likely to evolve over time.
The data centralization payoff should be measurable
and immediate. Over time, the companies that are
most diligent about this process will be the ones best
prepared for the inevitable next wave of (data) change.
11
 ABOUT SNOWFLAKE
Snowflake delivers the Data Cloud—a global network where thousands of organizations mobilize data with near-unlimited scale, concurrency,
and performance. Inside the Data Cloud, organizations unite their siloed data, easily discover and securely share governed data, and execute
diverse analytic workloads. Wherever data or users live, Snowflake delivers a single and seamless experience across multiple public clouds.
Snowflake’s platform is the engine that powers and provides access to the Data Cloud, creating a solution for data warehousing, data lakes,
data engineering, data science, data application development, and data sharing. Join Snowflake customers, partners, and data providers already
taking their businesses to new frontiers in the Data Cloud. snowflake.com

© 2022 Snowflake Inc. All rights reserved. Snowflake, the Snowflake logo, and all other Snowflake product, feature and service names mentioned herein
are registered trademarks or trademarks of Snowflake Inc. in the United States and other countries. All other brand names or logos mentioned or used
herein are for identification purposes only and may be the trademarks of their respective holder(s). Snowflake may not be associated with, or be
sponsored or endorsed by, any such holder(s).

CITATIONS
1
Gartner, “Data Sharing Is a Business Necessity to Accelerate Digital Business,” Laurence Goasduff, May 20, 2021, www.gartner.com/smarterwithgartner/data-sharing-is-a-business-necessity-to-accelerate-digital-business.
GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.
2
Research and Markets, “Data Lakes Market - Growth, Trends, COVID-19 Impact, and Forecasts (2021 - 2026),” https://bit.ly/3roSSRW, October 2021.

IDC Technology Spotlight, sponsored by Matillion, “Calming the Storm: Cloud-Native Data Integration,” doc #US47518521, March 2021
3