What is Change Data Capture (CDC) in Snowflake ? | by Santhosh K Lingappa | Feb, 2024 | ... https://medium.

com/@sanusa100/what-is-change-data-capture-cdc-in-snowflake-19ee4c9c1e64

Search Write Sign up Sign in

What is Change Data Capture (CDC)

in Snowflake ?
Santhosh K Lingappa · Follow
2 min read · Feb 9, 2024

Change Data Capture (CDC) might sound complex, but it’s super important
for keeping track of changes in your data.

1 sur 12 23/02/2024, 08:11

What is Change Data Capture (CDC) in Snowflake ? | by Santhosh K Lingappa | Feb, 2024 | ... https://medium.com/@sanusa100/what-is-change-data-capture-cdc-in-snowflake-19ee4c9c1e64

Snowflake, a cool cloud data platform, makes CDC easy. Let’s break it down:

CDC Basics:

2 sur 12 23/02/2024, 08:11

What is Change Data Capture (CDC) in Snowflake ? | by Santhosh K Lingappa | Feb, 2024 | ... https://medium.com/@sanusa100/what-is-change-data-capture-cdc-in-snowflake-19ee4c9c1e64

CDC helps you spot and grab any changes made to your data, like when you
add, edit, or delete stuff in a database. This is super helpful for things like
keeping track of your recipes over time or managing business data.

Snowflake has two cool tricks to make CDC a breeze:

• Time-Travel and Versioning:

Ever wish you could time-travel? Well, Snowflake lets you do that with your
data! You can check how your data looked at any point in time. This helps
you see the history of all changes — it’s like having a time machine for your
data.

• Stream and Tasks:

Snowflake also has Streams and Tasks to keep things rolling smoothly.
Streams catch changes in real-time, while Tasks automatically apply those
changes to where they need to go. It’s like having a team working behind the
scenes to keep everything up to date.

3 sur 12 23/02/2024, 08:11

What is Change Data Capture (CDC) in Snowflake ? | by Santhosh K Lingappa | Feb, 2024 | ... https://medium.com/@sanusa100/what-is-change-data-capture-cdc-in-snowflake-19ee4c9c1e64

Putting it into Action:

Implementing CDC in Snowflake is as easy as 1–2–3:

Step 1: Create a Stream

CREATE STREAM my_stream ON TABLE my_table;

This SQL statement creates a stream named my_stream on the my_table

table, capturing changes as they occur.

Step 2: Create a Task

4 sur 12 23/02/2024, 08:11

What is Change Data Capture (CDC) in Snowflake ? | by Santhosh K Lingappa | Feb, 2024 | ... https://medium.com/@sanusa100/what-is-change-data-capture-cdc-in-snowflake-19ee4c9c1e64

CREATE TASK my_task

WAREHOUSE = my_warehouse
SCHEDULE = '5 MINUTE'
AS
ALTER TABLE my_target_table
SWITCH TO my_stream;

a task named my_task is created to switch changes from the stream to the
target table (my_target_table) at a specified interval.

Step 3: Monitor the CDC Process

SELECT SYSTEM$STREAM_HAS_DATA('my_stream') AS has_data;

Use this query to check if the stream has data, indicating new changes to be
applied.

Best Practices for CDC in Snowflake

5 sur 12 23/02/2024, 08:11

What is Change Data Capture (CDC) in Snowflake ? | by Santhosh K Lingappa | Feb, 2024 | ... https://medium.com/@sanusa100/what-is-change-data-capture-cdc-in-snowflake-19ee4c9c1e64

• Set Up Your Warehouse Right: Efficiently configure the Snowflake

warehouse to handle the CDC workload, considering factors like size,
concurrency, and performance.

• Regularly Monitor Streams: Keep a close eye on stream activity to ensure

timely processing of changes and identify any issues promptly.

• Implement Error Handling: Incorporate error handling mechanisms

within your CDC tasks to address any issues that may arise during data
replication.

• Explore the Past with Time-Travel: Leverage Snowflake’s time-travel

feature for in-depth analysis of historical data and identifying trends or
patterns over time.

• Write it All Down: Maintain thorough documentation of your CDC

processes, including stream configurations, task schedules, and any
changes made to the data.

Conclusion:

CDC might be a big deal in the data world, but Snowflake makes it a walk in
the park. With Streams, Tasks, and other cool features, Snowflake helps you

6 sur 12 23/02/2024, 08:11

What is Change Data Capture (CDC) in Snowflake ? | by Santhosh K Lingappa | Feb, 2024 | ... https://medium.com/@sanusa100/what-is-change-data-capture-cdc-in-snowflake-19ee4c9c1e64

keep tabs on your data changes, ensuring accuracy and unlocking insights
into your data’s evolution. Easy peasy, right?

Written by Santhosh K Lingappa Follow

7 Followers

Data, Database, DataWarehouse and Cloud.

More from Santhosh K Lingappa

7 sur 12 23/02/2024, 08:11

What is Change Data Capture (CDC) in Snowflake ? | by Santhosh K Lingappa | Feb, 2024 | ... https://medium.com/@sanusa100/what-is-change-data-capture-cdc-in-snowflake-19ee4c9c1e64

Santhosh K Lingappa Santhosh K Lingappa

All You Need to Know to build Snowflake Python Worksheets:

Snowflake Alerts and Notifications <HOW TO>
Snowflake provides features that allow you to Snowflake Python Worksheets release is a
send notifications and automate actions… real game-changer, simplifying the process …

4 min read · Jun 30, 2023 6 min read · Aug 6, 2023

8 sur 12 23/02/2024, 08:11

What is Change Data Capture (CDC) in Snowflake ? | by Santhosh K Lingappa | Feb, 2024 | ... https://medium.com/@sanusa100/what-is-change-data-capture-cdc-in-snowflake-19ee4c9c1e64

Santhosh K Lingappa Santhosh K Lingappa

Streamlit app by using Snowsight Choosing the Right Data Tools:

and accessing Snowflake data Informatica vs. Matillion vs.…
Streamlit in Snowflake provides a Python When it comes to handling data, there are
editor in Snowsight where you can write, edi… several tools available, each with its own…

3 min read · Jan 28, 2024 3 min read · Aug 29, 2023

See all from Santhosh K Lingappa

Recommended from Medium

9 sur 12 23/02/2024, 08:11

What is Change Data Capture (CDC) in Snowflake ? | by Santhosh K Lingappa | Feb, 2024 | ... https://medium.com/@sanusa100/what-is-change-data-capture-cdc-in-snowflake-19ee4c9c1e64

Dave Melillo in Towards Data Science Vu Trinh in Data Engineer Things

Building a Data Platform in 2024 I spent another 6 hours

How to build a modern, scalable data understanding the design…
platform to power your analytics and data… All insights from Snowflake academic paper
in 2016

9 min read · Feb 6, 2024 17 min read · Feb 10, 2024

1K 20 359 6

Lists

Staff Picks Stories to Help You Level-Up

587 stories · 767 saves at Work
19 stories · 488 saves

Self-Improvement 101 Productivity 101

20 stories · 1378 saves 20 stories · 1261 saves

10 sur 12 23/02/2024, 08:11

What is Change Data Capture (CDC) in Snowflake ? | by Santhosh K Lingappa | Feb, 2024 | ... https://medium.com/@sanusa100/what-is-change-data-capture-cdc-in-snowflake-19ee4c9c1e64

What I learned this week | Piet Jan de Bruin Mikkel Dengsøe

Can Snowflakes Melt and Data ownership: A practical guide

Databricks Crumble? A toolkit for defining and activating
Snowflake and Databricks are the two biggest ownership across the data team, upstream…
names in the B2B data space. Over the past…

7 min read · Jan 22, 2024 11 min read · Feb 12, 2024

103 3 159

11 sur 12 23/02/2024, 08:11

What is Change Data Capture (CDC) in Snowflake ? | by Santhosh K Lingappa | Feb, 2024 | ... https://medium.com/@sanusa100/what-is-change-data-capture-cdc-in-snowflake-19ee4c9c1e64

Dave Flynn in In the Pipeline Eylon Steiner in Snowflake

dbt best-practices are in, but Snowflake New DataOps Related

merge-times are up Features (2024)
Have you noticed that your time-to-merge How to manage Snowflake resources with
has increased since moving your data proje… Snowflake’s Git Integration, EXECUTE…

3 min read · Feb 7, 2024 7 min read · Feb 5, 2024

96 48 1

See more recommendations

Help Status About Careers Blog Privacy Terms Text to speech Teams

12 sur 12 23/02/2024, 08:11