Scribd Logo
Upload

Welcome to Scribd!

  • Delta Lake Landscape 2022 Target State Architecture 1.5

    Uploaded by

    Le Viet
    8 views1 page
    The document describes a software as a service (SaaS) and platform as a service (PaaS) data architecture that ingests data from various source systems through replication and messaging. The data is then curated using Delta Lake and Spark Streaming and stored in a data lake for business consumption and publication to downstream systems.

    Original Description:

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    The document describes a software as a service (SaaS) and platform as a service (PaaS) data architecture that ingests data from various source systems through replication and messaging. The data is then curated using Delta Lake and Spark Streaming and stored in a data lake for business consumption and publication to downstream systems.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    8 views1 page

    Delta Lake Landscape 2022 Target State Architecture 1.5

    Uploaded by

    Le Viet
    The document describes a software as a service (SaaS) and platform as a service (PaaS) data architecture that ingests data from various source systems through replication and messaging. The data is then curated using Delta Lake and Spark Streaming and stored in a data lake for business consumption and publication to downstream systems.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    of 1

    Costco Confidential Wenatchee Data Center (WDC) Software as a Service (SaaS) Platform as a Service (PaaS)

    Source Systems Data Ingestion Data Management/Curation

    Replication
    Databrick’s Delta Lake - Lakehouse Architecture
    Spark Streaming
    INA400 Spark Streaming

    CDC Data/ Real Time Certified Datasets,


    Replication Event events
    WHB400 Event Based Data from ready for Business
    Data Replication Hub Source Consumption
    (Messaging) Data Publication
    Replication
    CAMP01US Topic 1 Schema 1
    Topic 1
    Reader Loader Server_schema 1 sales
    Replication
    OMS

    Replication Topic 2 Topic 2 Schema 2 inventory


    KRONOS Loader Server_schema 2 Views
    Reader

    raw
    Supply Chain
    Replication ODP API Topic 3 Topic 3 Schema 3
    SAP Core SAP Server_schema 3
    Reader Loader
    (ECC,CRM, SLT
    HCM)
    marketing

    Other Event Messaging


    Sources
    Messaging Ecomm
    Data Management
    Stibo Schema n
    Loader Server_schema n
    Messaging
    Data Hub Synapse (Data Warehouse)
    N…
    Change
    Azure Messaging
    CosmosDB
    Feed Function Cosmos DB
    (MGLO)
    pos_rts_schema
    POS TLog POS
    IBM Retail Topic n
    Log Topic n Azure Search
    Controller Reader
    Pack

    External_Schema n
    Delimited Files
    Axway (SFTP) Other Downstream System

    Third Party API & Batch


    Sources, Azure Data Lake Storage (ADLS)
    APIs, IOT

    You might also like