Introduction to Data Mesh

Complete Overview of Data Mesh and its Benefits (xenonstack.com)

Table of Contents

1. Introduction to Data Mesh

2. What is Anthos Service Mesh?
3. What is Azure Service Fabric Mesh?
4. What is AWS App Mesh?
5. How is Data Mesh different from Data Lake?
6. How is Data Mesh different from Data Fabric?
7. What are the Benefits Of Data Mesh?
8. Conclusion

Introduction to Data Mesh

Data mesh builds a layer of connectivity that takes away the complexities of connecting,
managing, and supporting data access. It is a way to fasten the data together that is held across
multiple data silos. It combines the data distributed data across different locations and
organizations. Data mesh provides data that is highly available, easily discoverable, and secure.
It is beneficial in an organization where a team generates data from many data-driven use cases
and access patterns in data mesh.
Cloud technology used in conjunction with the “data mesh” concept provides a promising
approach to solving regulatory reporting problems. Source: Cloud-based Data Mesh
Technology
We can use data mesh, like when we need to connect cloud applications to sensitive data that
lives in a customer's cloud environment. Also, when we need to create virtual data
catalogs obtained from various data sources that can't be centralized. There is also a situation in
which data mesh is used, for instance, when we create virtual data warehouses or data lakes for
analytics and ML training that can be done without consolidating data into a single repository.

What is Anthos Service Mesh?

It is a fully managed service mesh that is used for complex microservices architectures. It is a
suite of tools that monitor and manage a reliable service mesh on-premises or Google Cloud. It's
powered by Istio, which is a highly configurable and one of the powerful open-source service
mesh platforms that have tools and features that enable industry best practices. It defines and
manages configuration centrally at a higher level. It is deployed as a uniform layer across the full
infrastructure. Service developers and operators can use a rich feature set without making a
single change to the application code. Anthos Service Mesh relies on Google Kubernetes Engine
(GKE ) GKE On-Premise Observability features. Microservices architectures provide many
benefits, but on the other hand, there are challenges like added complexity and fragmentation for
different workloads. It solves the problem like it unburdens operations and development teams
by simplifying service delivery across the board, from traffic management and mesh telemetry to
securing communications between services.

What are the features of Anthos Service Mesh?

Here are some of the features of Anthos Service Mesh

Deep visibility built-in [beta]

Anthos Service Mesh is integrated with Cloud Logging, Cloud Monitoring, and Cloud Trace that
provides many benefits, such as monitoring SLOs at a per-service level and setting targets for
latency and availability.

Easy authentication, encryption

Anthos Service Mesh ensures easy authentication and encryption. It transport authentication
through MTLS (Mutual Transport Layer Security) has never been more effortless. It secures
service-to-service as well as end-user-to-service communications with just a one-click mTLS
installation or incremental implementation.

Flexible authorization

It provides flexible authorization like we only need to specify the permissions after that grant
access to them at the level that we choose, from namespace down to users.

Fine-grained traffic controls

Anthos Service Mesh opens up many traffic management features as it decouples traffic flow
from infrastructure scaling and includes dynamic requests. Routing for A/B testing, canary
deployments, and gradual rollout, and that also all outside of your application code.

Failure recovery out of the box

It provides many critical failure-recovery features out of the box, to configure dynamically at
runtime.

What is Azure Service Fabric Mesh?

Azure Service Fabric Mesh helps the developers deploy microservices applications, and there is
no need to manage virtual machines, storage, or networking. The applications hosted on Service
Fabric Mesh run and scale without worrying about the infrastructure powering it. Service Fabric
Mesh has clusters of many machines, and every one of these cluster operations is hidden from
the developer. You only need to upload the code and mention the resources we need, availability
requirements, and resource limits. It automatically allocates the infrastructure and handles
infrastructure failures as well, and we need to make sure the applications are highly available.
We need to take care of the health and responsiveness of the application and not the
infrastructure. Azure Service Fabric has three public offerings: Service Fabric Azure Cluster
service, Service Fabric Standalone, and Azure Service Fabric Mesh service.Read more
about Top 6 Big Data Challenges and Solutions to Overcome

What is AWS App Mesh?

AWS App Mesh helps to run services by providing consistent visibility and network traffic
controls. For services built across multiple computing infrastructure types. App Mesh abolishes
the necessity to update the application code. To vary how monitoring data is collected or traffic
is routed between services. It configures each service to export monitoring data and implements
consistent communications control logic across your application. When any failure occurs or
when code changes must be deployed, therein situation makes it easy. To pinpoint the precise
location of errors quickly and automatically reroute network traffic.

What are the advantages of AWS App Mesh?

Following are the advantages of AWS App Mesh: Provides End-to-end visibility because it
captures metrics, logs, and traces from all of your applications. We can combine and export this
data to Amazon CloudWatch, AWS X-Ray, and community tools for monitoring, helping to
quickly identify and isolate issues with any service to optimize your entire application.

Ensure High Availability

App Mesh gives controls to configure how traffic flows between your services. Implement easily
custom traffic routing rules to ensure that every service is highly available during deployments,
after failures, and as your application scales.

Streamline Operations

App Mesh configures and deploys a proxy that manages all communications traffic to and from
your services. This removes the requirement to configure communication protocols for every
service, write custom code, or implement libraries to control the application.

Enhance Any Application

Users can use App Mesh with services running on any compute services like AWS Fargate,
Amazon EKS, Amazon ECS, and Amazon EC2. App Mesh can also monitor and control
communications for monoliths running on EC2. Teams running containerized applications,
orchestration systems, or VPCs as one application with no code changes.

Hybrid Deployments
To configure a service mesh for applications deployed on-premises, we can use AWS App Mesh
on AWS Outposts. AWS Outposts could be a fully managed service that extends AWS
infrastructure, AWS services, APIs, and tools to virtually any connected site. With AWS App
Mesh on Outposts, you'll provide consistent communication control logic. For services across
AWS Outposts and AWS cloud to simplify hybrid application networking.

How is Data Mesh different from Data Lake?

Given below are the differences between Data Mesh and Data Lake.

 The data lake is a storage repository. That holds a vast amount of raw data
in its native format. The hierarchical data warehouse stores data in files or
folders. Whereas the data lake uses a flat architecture to store data.
 The advantage of the data lake is that it is a Centralized, singular, schema-
less data store with raw (as-is) data as well as massaged data.
 The Mechanism for fast ingestion of data with appropriate latency
 It helps to map data across various sources and give visibility and security to
users
 Catalog to find and retrieve data
 Costing model of centralized service
 Ability to manage security, permissions, and data masking
 The main difference between data mesh and data lake is that data mesh is
decentralized ownership in which domain teams usually consider their data a
byproduct that they don't own because a data lake is centralized ownership
of that raw data.

Click to explore Open Source Big Data Tools and Frameworks

How is Data Mesh different from Data Fabric?

 Data Fabric integrates data management across cloud and on-premises
to accelerate digital transformation. It helps deliver consistent and integrated
hybrid cloud data services that help data visibility and insights, data access
and control, and data protection and security.
 Data Fabric and Data Mesh's difference is that Data fabric allows clear
access of data and sharing of data across distributed computing systems by
means of a data management framework that is single, secured, and
controlled.
 But Data Mesh follows a metadata-driven approach and is a distributed data
architecture supported by machine learning capabilities. Data Mesh is a
tailor-made distributed ecosystem with reusable data services, a centralized
governance policy, and dynamic data pipelines.

What are the Benefits Of Data Mesh?

 1. Data Mesh provides agility. In this, each node works independently. The
node is containerized and can be deployed as soon as any changes are
ready.
2. Construct and deploy new nodes to the mesh, whenever new data arises.
Many portals and teams can access the same node, allowing the
organization to scale the data mesh. This way, data mesh provides
scalability.
3. Use data mesh under various circumstances, like connecting cloud
applications to sensitive data that lives in a customer's on-premise or cloud
environment. Use it while creating virtual data catalogs from various data
sources. We need to create virtual data warehouses or data lakes for
analytics and machine learning training without consolidating data into a
single repository.

Conclusion
A data mesh allows the organization to escape the analytical and consumptive confines of
monolithic data architectures and connects siloed data. To enable machine learning and
automated analytics at scale. The data mesh allows the company to be data-driven and give up
data lakes and data warehouses. It replaces them with the power of data access, control, and
connectivity.