How to use Azure Data Explorer in Azure

Synapse Analytics
In this article we will seen how to use Azure Data Explorer to get insight from your Data.
Azure Data Explorer is a fast and complete data analytics service for real-time exploring,
especially on large volumes of streaming data from logs sources and more.
To use this service, need to first create Data Explorer pool and an explorer database.
Azure Data Explorer is available in Azure Synapse Analytics in a preview version.
We are going to those steps:
- Launch Synapse studio
- Create Azure Data Explorer pool
- Create Data Explorer database
- Ingest data and create tables, files are using in this article
- Querying the ingested data

Launch Synapse Studio

From your Synapse Analytics service (Azure portal) select open to open Synapse Studio

Create a new Data Explorer pool

1. On the Synapse Studio home page, navigate to the Management Hub by selecting
Management icon
2. Once you are in the Management Hub page, navigate to the Data Explorer pools section and
select New button to create a new one
3. Enter the details in the Basics tab like the pool Name, Compute specification etc...:
4. Click Next to get access to Additional settings
Set the setting to strict needed for POC purpose or test to reduce costs. For this article I choose
to enable them (streaming ingestion, purge)
5. Select Create to create the pool.

6. Check in the Data Explorer pools list the existence of the new created
You can pause it or configure it.
 7. Navigate to the Data Hub and in the Workspace, section collapse the new pool created

Now we need to create a Database Explorer to ingest data into it

8. Select the three dotes at the Explorer pool level and click on Open in Azure Data Explorer to
launch the Azure Data Explorer
 9. In the Azure Data Explorer, you will have three Tab:
a. Data Tab to quick ingest data or create table (Standard or Externale)

Data tab

b. Query Tab allow adding cluster, exploring data by querying it

Query tab
 c. Dashboard Tab to build visualizations, KPIs etc...
Dashboards tab

Create an explorer database

10. Go back to the Azure Synapse Analytics Studio at Data Hub workspace section, select
the plus button, and form the drop list select Data Explorer database (preview) Button

Then click create button

You can also create a database from Azure Data Explorer Studio at Data Tab page level
All actions tab, select Manage at Database section

Add the Cluster name, database name , data retention period policy, caching period.
At Azure portal you can add a database.

11. After created the database, refresh the Explorer pool to see the new database add her folders.
 You can add databases as needed to ingest, explore , build indicators.

Publish your work to save changes.

We have now the cluster and the database successfully created, it’s time to ingest data and
build queries.
From Azure Data Explorer Studio, you can ingest data with One-Time mode (back mode) It enables you
to quickly ingest data in JSON, CSV, and other formats into a table and easily create mapping structures.
You have also Continuous ingestion mode (near real-time) for streaming data.

One-time ingestion: Ingest data from local file, blob, blob container , data lake

Continuous ingestion: Azure Data Lake, Event Hub , blob container

To ingest in this article, I use ADLS as a source. Data I already loaded.

You can use Microsoft Azure Storage Explorer to navigation between your ADLS accounts, containers ,
folders , files…..

12. From One-Time ingestion section, select ingest from ADSL Gen2
In the ingest new data window, the Destination step is activated. Select the Cluster, Database,
and the table if it is already created. You can create a new one on the fly by select create new
table
13. Select Next Source to get access to the source information. The source tab is activated at this
step.
14. Select container option if you have an ADLS setup, add the Subscription, the Account and the
Container or Add URL if you have generated an SAS to grant access to you container.
 15. At the Schema defining file you have the list of files listed. Select the file you need to load.

16. Select Next Schema to open the Schema tab

At this step we have an overview of the schema and a preview of the data in the source file.

Live Compression type as it is and select in the Data format field, the right file format. In this article we
use CSV.
17. Select Next Summary

18. In the tiles below the ingestion progress, we have a Quick Queries for analysis and
exploration

19. Repeat the same process until all files gets loaded.
20. Load 3 files at one time
We can go back to the Azure Data Explorer and querying the tables ingested

21. From the left panel in Azure Data Explorer navigate to the Query Tab to

22. Tip some queries to explore your data.

Select count *
Select All

Select Top 10
Filtering on OrderDate and OrderNumber

Take 5
TOP 20 OrderLineItem

Sort By
Aggregation
Conclusion:

With Microsoft Azure Data Explorer, you can quickly and easily ingest your in-rest data as well as in
motion (logs, IOT, Steaming). You can Also manage, monitor Clusters from Synapse Analytics Monitor
Tab in terms of instances, CPU use, Cache utilization Query results