Identify event streaming techniques and their uses ~ Differentiate between types of mashups in order to determine the best mashup for a dashboard presentation ~ Analy trusted data in order to derive information. a I. EVENT STREAM PROCESSING a Introduction — Stream Processing Stream processing is the practice of taking action on a series of data at the time the data is created. Stream processing often involves multiple tasks on the incoming series of data, i.e., the data stream which can be performed serially, in parallel, or both. This workflow is known as a Stream Processing Pipeline, and it includes the generation of stream data, data processing, and data delivery to a final destination. Actions that stream processing takes on data include sum, mean and standard deviation), analytics (e.g., predicting a future event based on patterns in the data), transformations (e.g., changing a number into a date format), enrichment (eg, combining the data point with other data sources to create more context and meaning), and ingestion (e.g., inserting the data into a database). aggregations (e.g., calculations such as Ceci Input Data Stream Processing Engine Fig. 3.1. eam ProcessingMash¥Ps occ real engi ream. the opt m Processing VS. Batch Processing isto ically, data was typically processed in 1d (c.g every night at a specified time, batches based on a sel every hundi r two mesabytes). Batch processing is currently not en fon OF EVE time the volume ches ‘ ly not a useful option si fe ata are being created constantly and data needs to be processed immedingiy ae be useful. hedule or some predefined Stream processing is mandatory for modern applications, Ente technologies that respond to data at the time at which it is created for a variety of use cases and applications. Stream Processing allows applications to Tespond to new data events at the moment they occur. Rather than grouping data and collecting it at some Predetermined interval, like batch processing, stream Processing applications collect and Process data immediately as they are generated. : Stream Processing — Working Stream processing is most often applied to data that is generated as a series of events, such as data from IoT sensors, payment processing systems, and server and application logs. Common models include publisher/subscriber (commonly referred to as pub/sub) and source/sink. Data and events are generated by a publisher or source and delivered to a stream processing application, where the data may be cleaned, augmented, tested and then sent to a subscriber or sink. In terms of technology, common sources and sinks include Apache Kafka, Big Data repositories like Hadoop, TCP sockets, and in-memory data grids. erprises are using many Stream processing =] ey Fig. 3.2: Stream ProcessingAdvanced Data Visualization, 94 Real-time stream processing is the process of taking action on data at the time the data j, generated or published. Historically, real-time processing simply meant data was “processed frequently as necessary for a particular use case.” But as stream processing technologies an frameworks are becoming ubiquitous, real-time stream processing now means processing in rey time. Processing times can be measured in microseconds (one millionth of a second) rather tha, in hours or days. Example Use Cases Use cases typically involve event data that is generated by some action and upon which some action should immediately occur. Common use cases for real-time stream processing include: © Real-time fraud and anomaly detection: One of the biggest losses for credit carg companies is because of fraud transactions. Credit card providers have been able to reduce their fraud write-downs by huge amounts in the range of $800 M per year, due to fraud and anomaly detection powered by stream processing. Credit card processing delays are troublesome to both the end customer and the store attempting to process the credit cards. Historically, credit card providers performed time-consuming fraud detection processes in a batch manner post-transaction. With stream processing, as soon as a user’s swipes a card, they are able to run algorithms to recognize and block fraud transactions, and trigger alerts for abnormal transactions, while processing regular transactions immediately without any delay. Internet of Things (loT) edge analytics: Companies in manufacturing, oil and gas, and transportation, as well as those constructing smart cities and smart buildings, make use of stream processing to handle data from billions of “things”. An example of IoT data analysis is detecting anomalies in manufacturing that indicate problems that need to be fixed to improve operations and increase profits. With real-time stream Processing, a manufacturer may recognize that a production line is turning out too many defective products while they are being manufactured instead of recognizing that an entire batch is defective at a later point of time. They can recognize huge savings and prevent massive waste by pausing or stopping the manufacturing line for immediate repairs. Real-time personalization, marketing, and advertising: With real-time stream Processing, companies can deliver personalized, context-based experiences for theif customers. This can include a discount for an item that a customer has added to a catt on a website but did not immediately purchase, a recommendation to connect with 4 newly registered friend on a social media site, or an advertisement for a product similar to the one just viewed.fe time VS Streaming Data 9s eal! 7 ‘ast enough in real time for the buy/no- buy/sell decision to be executed at a profit or the lowest possible loss, Streaming data is data that is continuously generated and delivered rather than processed in batches or micro-batches. This is often referred to as “event data,” since each data point describes something that occurred at a given time. Stock ticker data is @ good example of stream data. If day traders collect and process a batch of data every 15 minutes, or when they complete some other interval or cross some threshold, they may miss many opportunities to buy and sell for a profit. Real-time Data Streaming Tools and Technologies Traditional data tools were built around disk-based processing and batch data pipelines, making them insufficient for streaming real-time data as required in the use cases above. While a variety of real-time data streaming tools and technologies have entered the market in the last few years, they have some common features: ¢ Inmemory processing: RAM is critical to stream processing. Disk-based technologies simply are not fast enough to process streams in real-time, even in massively parallel architectures where the resources of many computers are joined together. The decreasing price for RAM has made in-memory computing platforms the common standard for building applications to support real-time data. streaming. Open source: The open source community has been responsible for laying the groundwork for many real-time data streaming tools and technologies. Technology Providers will often build features and components to make real-time stream Processing technologies enterprise-ready, but there is quite often an open source element underneath. Cloud: Many of the applications that generate data for real-time stream Processing are cloud-based. Most real-time data streaming technologies are often based on cloud. Event Stream Processing Event stream processing (ESP) is the practice of taking action on a series of data points * originate from a system that continuously creates data. The term event refers to each data Point in the System, and stream refers to the ongoing delivery of those events. A series of events it Also be referred to as “streaming data” or “data streams.” Actions that are taken on those “ents include aggregations (e.g., calculations such as sum, mean and standard deviation), eAdvanced Data Visualtzation lata), transformations (ep int with other da, mbining the data poin : ds, esto (eg., inserting the data into ; 96 analytics (e.g., predicting a future event based changing a number into a date format), enrichment (e: aning), and sources to create more context and mei database). g ‘An Overview of Event Stream Processing Event stream processing is often viewed as complementary to batch processing. Batch processing is about taking action on a large set of static data (data at rest), while event stream processing is about taking action on a constant flow of data (data in motion). Event stream i processing is necessary for situations where action needs to be taken as soon as possible. This is j why event stream processing environments are often described as real-time processing. There are many related and synonymous terms pertaining to event stream processing. For example, stream processing is considered an equivalent term. Complex event processing (CEP) is also viewed as a synonym although it implies an older class of technologies in contrast to newel, high-speed technologies that are popular today. The term message is often used to refer to events and thus the entire system can be referred to as a messaging system. Fig. 3.3: Event Stream Processing Components Event streaming is an implementation of the publi: il i : nt strean np publish/subscribe architecture pattern (pub/sub) with certain specific characteristics. The pub/sub pattem involves the following elements:Mashups 97 @ Message: Discrete data a Publisher wi, . Wishes to communicat il ° replat Puts messages in a message broker in Ape te a subscriber. 7 icular topic. . oe = waa Messages from 4 Message broker's ee ae © Message broker: A system with i a " tl it make them available to subseribers he ability to store messages from publishers and @ Channel or Topie: A 5 ubset of eve; channel are used interchat nts that. , share a category. . ingeably in the indust ‘egory. The terms topic and ry. @5 es 1e > e-0-0 ~» a 0 Pre-Processing OHCo c--0 + @ a Processing Sars aan @ Perit —@ susicder oon toi Fig. 3.4: Event Stream Components Event Stream Processing ~ Working An event stream is a sequence of events, meaning order between events matters, Events flow through the system in a specific direction: from producers to consumers. Event stream processing works by handling a data set by one data point at a time. Rather than view data as a whole set, event stream processing is about dealing with a flow of continuously created data. This requires a specialized set of technologies. In an event stream processing environment, there are two main classes of technologies: (1) the system that stores the events, and (2) the technology that helps developers write applications that take action on the events. The former component pertains to data storage, and Stores data based on a timestamp. For example, you might capture outside temperature every minute of the day and treat that as an event stream. Each event in this case is the temperature Measurement accompanied by the exact time of the measurement. This is often handled by technology such as Apache Kafka. The stream processors or stream, processing engines is the ral “event stream processing” component that allows us to take action on the incoming data. A arty of processor options are available inthe market today, Though mos steam a ®Dpear similar in capabilities, the in-memory stream processors stand out because 0 ‘Process large amounts of streaming data very quickly.os Advanced Data Visuan, PRODUCERS —EvENTs—— , —vers— [ i) iRecoRos nt Stream Platform Example: Use Cases of Event Stream Processing Use cases such as payment processing, fraud detection, anomaly detectioy ’ ene Handi des In, predic inantenance, and IT analytics depend on immediate action on data. All of thee es oar with data points in a continuous stream, each associated with a specific Point in time. These a \ classic event stream processing examples because the order and timing of the data Points bey with identifying patterns and trends that represent an important insight for users, > Event stream processing is also valuable when data granularity is critical. For example, te changes to a stock price are often more important to a trader than the stock price itself, ang stream processing allows users to track all the changes to make better trading decisions. The ice of change data capture (CDC), in which all individual changes to a database are event stream processing use case. In CDC, downstream systems can use the stream of idual updates to a database for purposes such as identifying usage pattems that can help fine optimization strategies, as well as tracking changes for auditing requirements, vent Streaming Architecture Components An end-to-end ESP stack consists of an assembh specific functions such as storage, analytics, shape an event streaming architecture: 1. Stream Processor ly line of various platforms designed for and processing. There are four main components that The stream processor, commonly referred to as the stream processing engine, is the agent that extracts data from the source. Once data is captured, it is translated-to a standard message format, which it streams continuously. These processors rely on a fast performing messaging Platform to enable massive streaming capacity and data persistence. Though such brokers mostly focus on streaming, these also apply queries to events from message queues and generate results. Some popular stream processors in use today include the Hazelcast Platform, Apache Flink, Apache Spark Streaming, and Confluent ksqIDB.Mashups . 2, ETL ETL (Extract-Transform-Load downstream actions by integrating Joading. ETL is the capability in an multiple sources into an enriched, provisioning of historical context ) is the series of functions that prepare data for specific data through three stages: extraction, transformation, and ESP system used to Process large amounts of raw data from Consolidated view. This cnables easier analysis and reporting, and business performance, and the maintenance of an accurate i , which gives business users an all-round understanding of business trends and their underlying raw data, 3. Query Engine Once the data goes through the ETL stages and is ready for consumption, the query engine processes and analyzes this data further to Provide business insights. Event stream processing utilizes a list of passive queries to analyze and sequence data for Storage or use by other processors. Some popular query engines that enable real-time data analytics for event streams include: © Elasticsearch @ Amazon Athena @ Amazon Redshift © Cassandra 4, Data Storage Certain use cases require organizations to store streaming event data. While many ESP applications only work within a local control loop, some applications also require events to be recorded by default. To do so, organizations can extend traditional IT storage devices and databases to enable the post-processing of events, Some data storage options for event streams include: () Data lakes: A data lake option is affordable and flexible since there is no need to tabulate data, making it agile and highly efficient for enormous volumes of data. Data lakes, however, typically experience high latency, which presents a challenge for real- time analytics. (i) Databases or data warehouses: Primarily used to store data for analysis that most commonly support SQL queries. (iii) Message brokers: Requires no additional components as it uses the inherent query engine for storage and is, therefore, the easiest to set up and scale. Benefits of Event Stream Processing © It offers the ability to build event stream pipelines to serve advanced streaming use cases, For example, if event data needs to be enriched with metadata and thenyo Advanced Data View 100 ake use of an ev transformed into a JSON object for storage, we can m ent, Streep Pipeline. ; ime, civin : © It processes and analyzes large amounts of = oe a giving users the ability, filter, categorize, aggregate, or cleanse data be! on ie i y when data volume ; © It scales infrastructure seamlessly w shi t © It enables continuous event monitoring, which allows users to create alerts to dete pattems or anomalies. ; © It allows for real-time decision-making. ins Where It is Used Event Stream Processing — Domal | rocessing is used to handle large amounts of continuous data, However, eve Evert See useful when users want to leverage its real-time nature, ie., when ing is ing is i ne reeks immediate action on events. Event Stream Processing is used commonly in the a 7 following industries: E-commerce © Fraud detection ¢ Financial industry, especially the banking industry ¢ Intelligence and surveillance © Marketing @ Analytics er, the use of event stream processing is not limited to these industries, like New York Times also make use of Event Stream Processing using Apache distribute published content to their readers in real time through various appl applications in various domains are as follows: Kafka ty ications, Ithcare Sector: Monitoring patients in hospital care and predicting the changes in their condition to ensure a timely treatment in cases of emergencies. 9 tics and Automotive Sector: Trackin, 1g and monitoring cars, trucks, fleets, and Hotel, and Travel Sector: ctions and orders. nufacturing Sector: “ices or other equipm Collecting and immediately responding to customer To continuously capture and analyze sensor data from IoT ent, li Event Streams can deliver numerous benefits like: ‘© Systems can closely mimic real-world processes, © Increased utilization of scal Services can remain idle unt © Improved flexibj ity. le-to-zero functions (serverless computing), because more il they are needed.ly Apache Kafka i 101 Most common is used oA Ht also allows you to build Evcen or eee PE 9 store evens et poe to different Event Streams for further re Pipelines where a pe The various aspects of Event Stream Processing that are ; "publishing and subscribing to Event Streams APA Kalin clade: ¢ Storing Streams of Events reliably without any time . e Processing Streams of Events as and when they : — Event Stream Processing vs. Batch Processing with major advances in technology, organizations are dealing with a far greater volume of gustan they did few years ago. So, more advanced data processing tools are required to keep up ‘gin tis rapid change. A traditional application undertakes the intake, storege, processing of dota with storage of the processed results. These processes usually occur in batches wherein the ion will have to wait until it has sufficient data before it can start processing. The tine == jon has to spend waiting for data is not acceptable for reabtime or tine-critical sppleations that would need immediate data processing.102 Advanced Date Vitaliy, Batch Processing vs : Stream processing Comparison Chart Pascoe et ekarccs batch withth) Pere yia aed pep batch process! Pre merlluacres: Pests eae Cagis ‘g platforms like | platforms «suc ‘Spark, Graphx, | Spark streaming, ee eee! |e cist lanl ceca! EY Peery verter Prt Mikes Pieces esas Fig, 3.6: Batch vs. Stream Processing Therefore, to overcome this challenge, Event Streams are used, In Event Stream Processing, each single data point or event will get processed immediately which means there is no queue of data points which makes it ideal for real-time applications. In addition to this, Stream Processing allows users to detect patterns, look at various levels of focus and take a look at data from multiple streams simultaneously. Compared to the problems of Batch Processing, Event Stream Processing can work with a lot less hardware by spreading the processing over time. Delivery Model of Event Streams An event stream is a sequence of events, meaning order between events matters. Even's flow through the system in a specific direction: from producers to consumers. As producer events7 are published to ” icers, they ublish opi = Published to a topic, a topic is 1 Hac iype of event. Subscribers interested sn events of a a 2Dically a categorization for a type s parti icular category subscribe to that top! er events are Published to the topic, the broker identifies Se te’ events available to them, Publishers and Subscribers, need not know abot h othe ‘an event is always published by a single publisher, but it might need ee il 5 uibscribers. Handling subscriber failures and determining WARE meses a big part of the protocol Implementation. Because of that one a Because of ai to is i ae factors of these protocols is how they handle broker/s bscriber communica ia Connectio! The connection between broker and subs. the subscribers for the topic and n Initiation criber can be either Subscriber-initi tion 7 initiate ~ initoted. Broker-initiated models require the broker to know about existing Secueenes be achieved either through static registration or a service discovery mechanism. In subscriber- inated models, subseribers need to be aware ofthe broker's address These monte convenient when there are many dynamic subscribers, especially in environments when service discovery is not feasible (e.g., a WAN). push vs. Pull Model In the pull model, subscribers send a Message to the broker when they want to receive events, and only then the broker sends them data. Subscribers need to maintain the State to know what topics exist and what data to ask for. For example, Kafka Consumers pull data from brokers. In the push model, the broker determines when to send data to subscribers based on its own logic. Subscribers do not need to maintain state or implement special logic to coordinate amongst themselves. Message Delivery Another essential aspect of technologies implementing the pub/sub pattern is determining which subscribers get each event. The options are: © All subscribers to a topic get all events for the topic, which is useful for use cases where there is no shared state across consumers. Chat applications running on client devices are good fits for this model. Event processing requires updating different states whenever a new chat message is sent. © Only one of the topic subscribers gets an event. This model is commonly used for cases when event processing involves shared state, as multiple subscribers processing an event would result in inconsistencies. A common example would be subscribers that send an email when an event occurs (like a new user sign up). If multiple Subscribers were running and all subscribers processed the event, multiple emails Would be sent. * Having subscribers grouped into “groups” and having all groups and He Subscriber per group get an event. Kafka Consumer Groups support this model.mee Advanced Dats Visuatcng, Il, MASHUPS IN DASHBOARD PRESENTATIONS Mashup A mashup is a combination of separate, 'Y compound, and useful system. The data sources for mash combination, The end goal is to produce a resource that 'S mi is currently available, in order to ¢1 rate a routirie business process. i tly avaitabte, it nhance of automate # FU ine , systems, mashups : be defined as a process OF combining heterogeneous data frog, In BI syst shups can be defined as dat . ee ; multiple sources. Data Mashing is the proces of integrating a related heterogeneous ang cat 5 “e-a more unified view: application data from numerous sources to give am ¢ . ; Mashing involves ageregating similar elements from different data sources, ‘tis class into Bice ie me Ga ‘and Data Mashing. Data ‘mashing involves on ination of mi data eozues) andl also known ‘as Enterprise Mashing or Business . lashing. t helps a consolidate the data on a single page OT mobile device and offers real-time business opportunities for organizations around the world. ; ; A business dashboard is a good example of data mashing. At any time, dashboards can pul the dats from multiple sources to create multiple reports {1 & single view and support operation become a common method for dé iness i ce. This isplay has business intellicence. This type of data display & qepreseneatioa ‘as it allows a full view of data from various sources. A good example of mashup is ‘an API. Developers have is to build a new application. The concept of the ability to plug in APIs ion mashups bas reduced the development overhead of rewriting already existing code, as te developers can reuse the code in the form of APIs or web services. As the use of technology and | Sr media have increased, mashups have gained in popularity. a ‘Types of Mashups ‘There are many types of mashups. Some of them are business, gencous elements into a sing, aly ups arise ean be internal, External, cy “ire efficient and effective than yj." consumer and data mashups. 1. Business or Enterprise Mashup Business (or enterprise) mashups define applications that combine their own resources, application and data, with other external Web services. They focus data into a single presentation and allow for collaborative action among businesses and developers. _An enterprise mashup is the integration of heterogeneous digital data and applications from smultiple sources for business purposes. An enterprise mashup is also sometimes known as # siness mashup. Enterprise mashups are secure, visually rich Web applications that expos actionable information from diverse internal and external information sources.10s End-Users (Koowiedge Worker) & @ @Q Developers et Joma cataborsio gy Mashup Web Application Developers integrate . Fig. 3.7: Enterprise Mashup The term enterprise mashup is often used to different Wed mashup, which is usually for the general public. modular components that the end-user can assemble and needs, In an enterprise mashup, the product is typically a and applications with externally sourced data, SaaS (so! Enterprise mashups also include integration with the govemance, and built-in business intelligence (BI) programming tools and stringent security measures. tiate a business-related mashup from a In general, a mashup is created from Teassemble as desired to serve current combination of internal corporate data fiware as a service) and Web content. business computing environment, data capabilities, and require sophisticated Jor Widgets (Wiring) to enterprise-class. applications }+ User-friendly Bullding Blocks |* Provide graphical and simple ‘user interaction mechanism J+ Abstracting from the complexity Of the undertying resources, Core Building Blocks ictual content, data or ‘pplication functionality + Encapsulatedvia well-defined interfaces (APIs) Fig. 3.8: Enterprise Mashup Stack106 Advanced pata Vitaly Enterprise Mashup Stack and User Roles hese FESOUTCES are Resources represent : A actual contents, data or application functionality: ay 4 existing enterprise systems or Web providers (i... Amaz0™ Google, 6.) and are reated by developers iliar wit ic pers who are familiar with technical development. and simple Use The layer above contains gadgets (widgets) which provide graphic ons to. adaply Mir interaction mechanisms like aggregation, transformation, filter, oF sort operation® , and manipulate the content of the underlying resources. «and configure suc Finally, the end-users from the business units are empowered 19 COMPIO TT The vi visual widgets according to their individual needs, which results ayer is called wig composition of input and output parameters of the widgets 9 the masnh exprise mashup layer, Rating, recommending, tagging or shari ferent ng entre f0F tHe GT, problems. support the collaborative reuse of existing knowlede® f° solve a jn the browser and organi 2. Consumer Mashups pines Google Map and 2 Consumer mashups combine data fro it through a simple browser user interface ee 1 public, Consum 2 js aimed at the general pu ic. fered datahiewise srs ‘can use their web browser to combine and reformat the a to their needs. Example of a Consumer Mashup: a Google Maps is one of the most popular services that is used by a lot ‘of other - companies/services. Applications that provide information about road conditions, using maps t show social media friends’ location, providing ratings of different cities, etc. are some services that combine Google’s data and use it in their own application. ‘Amazon e-commerce is another popular service that mashup websites use. Their API can be used in applications such as viewing product availability over different websites, cost comparison ofa particular product over different websites, ec. ublic sources Itiple pI ane jon com (eg, Wikipedia vis er mashups, when offered with 3, Data Mashups i Data mashups combine similar types of media and information from multiple sources into single representation. The- combination of all these resources creates a new and distinct web service that was not originally provided by either source. é vee the BI industry, a small number of vendors have been creating applications that allow the combination or mashup of disparate data sources to be improvized without necessarily relying on a middie i jg i ; Rae middle step of ETL and data warehousing. This is now being referred to as “datayesh? pata mashup EMPOWEES power-user, rds or sifts through reports with un pear expected questions. They also he for vr self-service. Unlike consume 107 analyst, of + OF casual business ; user ormally derlying data, Thy 5 who normally u A cy help in r pin i i" P in repurposing and reusing P in introducing new data with mashups and . s i FOF media mashups that siny ieee one the SF obi swith no further Processing of that information, dt ie ries ne and manipulating data in ways that yw » Gata mashup involves actu: it Se att Ny ‘ere not i ally con ation (0 take analysis Co the next level, Previously possible, creating new source, afin outcome Dashboard canes ers 170) 26.3% 79 Cows teen no » |... Po | sme oF bo «+ aE 7 ee we — =— ae loncome 2 | se econ ne Noe i oe » ne ETT Ses ete a Scene fer oo SS] | es Bese | sccarone > ae also? STS Ree cM BGG Ccome Cope are i Fig. 3.9: Sample of a Data Mashup Benefits of Data Mashups Data mashup allows users to go far beyond the limitations of IT services, like waiting for change requests, work backlogs, and administrative overhead. End-user-driven daa mashup Provides features that bring about many benefits: . ° A higher success rate of deployment due to higher end-user satisfaction, usage rates, and adoption rates. © A greater number of actionable learning from enterprise data, generating higher business returns through greater sales or greater efficiency. © Faster decision-making; greater competitive market responses or strategies that stay ahead of the competition; better tactical or strategic moves in reaction to market conditions or customer performance, ® Reduced personnel needed to support BI solutions. * Reduced number of highly-skilled analysts or DBAs to satisfy end-user demands,ae Advanced Data Visuateay,, 108 BI Mashup Models Mashups are a useful, interactive tool in the hands of the business end-user. They Provide , way to integrate different kinds of data, and visualize the results in a coherent and Persuasiyg manner. Mashups enhanced by technologies like Flash and AJAX, and featuring drill-down ,, drill-through capability, become very powerful visually-powerful BI instruments for ecision, makers. BI mashups fall into different categories, such as overlay mashups, widgets an, dashboards, 1. Overlay Mashups 2. Widgets 3. Dashboards as Mashups Overlay Mashups ~~ Overlay mashups are the combination of two or more data sources for use in a single Web, based feature or application, Overlay mashups merge data from more than one source into ; single user-controlled feature; a typical example is combining business metrics (eg., sales by rep) with Google Maps. A common example of the overlay mashup is the placement of charts, graphs and othe business-relevant data onto a map. Google publishes an open API that allows software vendors and developers to mash up geographically-relevant information with Google Maps. A BI application may show pins on a Web-based map to represent customer locations. It may col those pins to reflect customer segments. When a user hovers over a pin, interactive charts er graphs may pop-up dynamically showing recent sales activity and customer service information for that location. Map mashups are the most common example of overlay mashups. It is extremely easy 0 overlay a drillable chart or graph onto a map. Productivity, quality metrics and other worke: performance management metrics could be overlaid on workflow and process maps. There a two main components within an overlay mashup. The first is open APIs, like the Google Maps API. In addition to the open APIs, the BI tools need to be modular in design and web-based XML and HTML are the programming/markup languages used to integrate these two components. Widgets Widgets are small HTML chunks from a third-party site that are embedded and executed 00 | a web page. An example of a widget is a YouTube video that the user can watch by clicking onit | without exiting the page in which the video is embedded. Widgets fall into the category of mashups because information from two different sources or servers—the one hosting the site, plus the third party source from which the widget comes is combined. The role of widgets is primarily to add variety and dynamism to a site. For exampk, a financial-market blogger may embed a NASDAQ real-time quote chart on his web page to adda useful and dynamic element to it. Or, a sports news website may have a streaming vide? embedded as a widget to show the week’s best plays.109 ak Users access to useful content from a third ee Tom a technical standpoint, widgets are easy to ‘Set are access to the server of the host Page and access eset py widgets have a similar function, to In BY ‘pout opening another Web browser. of Ste ired elements to embed a wid He. © producing the widget. ees wt ards aS Mashups ror corporate dashboards can also be viewed as mashups because they represent a means + ringing together different Visual representations into a unified picture for an end-user. al are becoming. increasingly common in business intelligence applications. They include interactive and drillable charts grouped together into a single window. The most flexible hoards are built with AJAX (Asynchronous JavaScript and XML) which allows individual jets (chats graphs, etc.) to be refreshed and repositioned without refreshing the entire srowser window. Usually, dashboards represent data from a single database or data warehouse. While the eshboard itself may contain several individual charts and graphs, the underlying data most often comes from a single data source. For example, a country sales manager might use his dashboard to show the current sales pipeline, quarterly projections, regional sales performance and sales by reduc. All of these individual report objects are populated by the company’s data warehouse. With the more sophisticated BI products, however, data can be also brought together on 2 shboard from different data sources, e.g., different databases, plus web services, flat files, RSS feeds, etc. Fig. 3.10: Sample Mashup Dashboard Architectural Aspects of Mashups Atchitecturally, there are two styles of mashups: web-based and server-based.Advanced Data Viseatzy, ise the user's web browse; , uo mashups typically © 1, Web-based mashups: These nd reformat the data on a rem, combine and reformat the data. 2. Server-based mashups: ‘These mashups a” final form. server and transmit the data to the user’s brows it is @ design pater Mashups can appear to be a variation of a Facade ae Teds with different 1 provides a simplified interface to a larger agate eS) ‘After several years of stand, Mashups can be used with sofware provide sere service-oriented Architectures 604, b services. These wa, development, mainstream businesses are starting to. ‘ ; : king this data available as discrete We’ ices oo vices and applications within and acres, to integrate disparate data by m eae services can be reused to provide completely new S° eae organizations, providing business flexibility. pint can help mashup gain acceleration zx: enterprises adopting it as a next steP towards self-Se of the Mashup ge of Web Parts and the Enterprise SOA: Mashups are gener pieces, and services of other web applications that already exist, cr sts but is scattered around in multiple applications. Addition! when data cannot be sourced from internal or exter] the parts. Enterprise can work “glue” between vasily reuse their existing applications alyze a er in its Key Aspects and Benefits © Effective Levera built out of the bits, data that already exis coding is only required suppliers or tO provide integration feature of mashup to its advantage as they can €: asa service for mashing it up. By focusing on the simplst ¢ Simple, Lightweight Software possible technigues and formals, web mashups appear to be successful and widespres! primarily because just about anyone can create them, Mashups are typically built use techniques like cutting and pasting snippets of JavaScript, using RSS feeds and XML ine JavaScript inclusions that c=2 to connect the various parts together, and even one-l pull in and integrate a powerful external component, such as Google Maps or: vouTube video player, that originally required a massive investment from its creat. Google makes its AdWords and Maps widgets incredibly easy to install and deploy id every video on its site. uiile YouTube even puts the hosting code next to each an Finally, mashups are 100% pure Software as a Service (SaaS) and require ® installation, updates, plug-ins, admin rights, or anything but a garden variety web browser and the mashup’s URL to run. ‘© A Focus on Self-service and DIY: As mentioned earlier, mashup development can be mi ea 1s much for advanced web users as it is for professional software developer 0 create rea Models and Services: just a eee have the potential to give end-users the ability 1 a = ves vee means that applications that could never have been justified © dodo Get as e som possible. These apps can now just be created bY Tul cnet appaln lxiny tink minis to ba they ed ue eaens ipplication backlogs that continue to bedevil IT department's everywhere, For example, just about anyone can now create thedashboards, and other software .d faster. The developers? wirequired business function and they en presented. accelerated Time to Market: . ne 7 pe titive strength; ee significant compet eee nth; Organizations th: . siferprise mashups will inevitably find themselves noncomeerigne in their use of |, Mashup versus Portals: More and more organizations ov sect, environment which is definitely a good sign, are moving towards a portal Z 5 a as portal; - sin bring into enterprise with its demonstrated strength swe aves, he va, tat it ware functional areas of source aggregation, content analytics and classification, individual and collaborative filtering, etc. What must be understood is that mash aggregation technologies. ups and portals both share content intelligence of an organization. They can also prove very effective and bring in value to Hepise with its unique aspects and benefits. They help move Web 2.0 into the enterprise and a reduce application development costs, improve time-to-market, increase productivity, SOA investments, and spur innovation. But enterprises are currently in a wait-and-watch as some of the hard pressing challenges like governance, security, manageability, ization are yet to be addressed.y pdvanced Date Vala, 112 Widget Assembly Layer Visual mashup Bam es source for Vieb clipping ata sources: Data sie, somrstructured fog. 0B Web service (e.g. Pinas web source ete) Fig. 3.11: Mashup Architecture iness end-users. Interactivity | as in the ability to use te Mashups can be made dynanis performance indicators, erst Benefits of ‘Mashups ‘Mashups that are interactive and dynamic are very useful to busi i drill-down oF drill-through capability, dedicated desktop application. ‘Flash technology, for example, by enabling key dynamically when the page is opened or refreshed. arly beneficial in the case of business intelligence, where they c= ilities. With mashups, reporting applications can draw directly .d deeper insight by combining through the use of or charts to update Mashups become particul extend reporting and analysis capabi ased information source, providing richer an prise systems. A Bl 5 the web with data that resides in ente! orporating elements of Google Maps or other web applications -users to dynamically present report output through those from any web-b intelligence gathered from acros dashboard can be enhanced by inc into the environment, allowing end: vehicles. ‘Mashups also enable embedding of BI into other applications. Through a mashup of varios ADs end-users can conduct more detailed and comprehensive reporting directly from witha a or ia application. Or, report output can become fully embedded within automat r business process management systems, where the get i 5 L , erati on dynamically trigger a subsequent event, based on its content. Soe ee ~~ eS greatly extend and enhance BI applications. With the o in place, companies can fully leverage mashups to ensure that intel most effective decision-making and process execution possible. correct policies a ligence drivesy ups WS os inonly Used Mashups ~ Sample Use Cases Howing section covers some real world the fo examples of sed ve 1: Purchase Funnel commonly used mashups. etl Y measuring the number of people se amining wha z peop! " ‘ach a an aT i eee wae of people move from one stage to the next. The five of s, + considerati ste pulling data from differ ton, preference and purchase) clearly ee ! ent data sources, such as web analytics and CRM. ‘The purchase funnel is a very effective tool to Measure how many prospective customers are converted to actual customers. Filing inthe numbers for eath stage sree e teae understanding of ustomer flow, and helps in understanding the strengths and weaknesses of an organization. a cores Puisooen eater on (8 Visits 64% 210,050 GA: Visitors This is expected, {lB Site Events 51% 43,500 GA: Events Look at tracking more events. Yam Leads 70% 15,075 SF: Leads This is good. Keep it up! WH Opportunities 429% — 3,750 SF: Contacts Robert is looking into this. Wins 435% 329 SF: Accounts RE: Marketing strategy meeting Fig. 3.12: Example of a Data Mashup — Purchase Funnel The above example contains data from two key sources: Google Analytics and Salesforce. The “Notes” column pulls data from a Google Drive spreadsheet to show how the team is addressing each stage of the funnel. Example 2: Social Interactions and Web Visits One of the most effective ways to measure the success of social campaigns is to correlate this information with web analytics. Social interactions may boost traffic indirectly and increase search and direct traffic to your website. The following example displays a social media dashboard that maps Google Analytics data with social interactions from key sites like Twitter and Facebook. The mashup combines three data sources in a single chart: Google Analytics, Facebook, and Twitter. This dashboard clearly demonstrates the correlation between viral posts and increased website traffic. Sci Everts kw vst [AUgUET2OTS i pope 17 : | i SPF EEE PEPE PAF Fig. 3.13: Social Media Data MashupAdvanced Data Vis mt alg, Example 3: Server Response Time and Help Desk Tickets A SaaS (Software as a Service) organization has a strong correlation between PPh performance and the number of help desk tickets submitted. If an organization can figure out the correlation between hourly application performance a when the most tickets are submitted, this information can be used to hire the correct Dumber a help desk staff during key times or increase the number of servers and adjust load balancing, | App Pertormance and Submitted Tekets as | © Server 1 BM Serer? Sere} @ Teckats q { ! Fig. 3.14: Mashup — Social Server Response and Help Desk Tickets The above mashup displays average response time for each hour for 3 servers supporting 2 ‘web application along with the average number of tickets submitted over a 24-hour period. This gard demonstrates the connection between application performance and submitted help dest ets, and helps in identifying that the busiest times of the day are between 9 AM and 5 PM. ince the data sources are refreshed in real time, problems can be identified and fixed in real time IML POWER BI Business intelligence is a technology-driven method which helps to analyze data and to _ provide actionable information which helps corporate executives, business managers, and other users to make informed business decisions. The following section contains a brief description of -Power BI and is variants, features and benefits, Today, businesses are swirled with an enormous amount of data in the size of terabytes, petabytes, or in some cases, exabytes from many different sources at a faster rate than ever before. This enormous data is not much of use to businesses unless they are able to make sense of it. Understanding and transforming this data into information with meaning and relevance is key to gaining incredible insight into your business and staying ahead of the game in reporting and operations. In the past, Spreadsheets (Excel) were the go-to option for manually organizing data and creating pivot tables, graphs, charts, etc. to gain actionable insight from your business data to make business decisions. But these spreadsheets offered limited functionality, and that is where business intelligence (BI) comes in. BI is a set of processes and techniques to analyze raw data and extract information that helps drive business decisions. And Power BI is one such tool thatost es the manual process of data ce ms analyzing the data, creating Wers to their most important auroral ‘ollection, or ors, visualization, and much more to help peg nen zing data rertons and make data-driven decisions, inesses find ans w power BI ver BI is a cloud-based Busi power BI is acl ‘ed Business Intelligen . rs with tools for visualizi - benee platform offered by Micros vase rough a. userfiendly meee AEstegtng, and analywing ne as sources Y pierface, Power BI is a’ Business Intligence and. Dat ization tool for converting dat; ‘i jgualization 8 data ftom variou: into i . es reports. Power BI offers cloud-based ac IS data sources into interactive dashboards and , ‘vices for interactive visualizations wit ata for end-sers to create their own pone faa ee visualizations with a simple Power BI primarily connects disparate data sets, : oe . » Processes them, and transforms them into actionable insights, pen using visually compelling. visualizations, Teports, and dashboards. Power BI empowers uusinesses with the capabilities to look not only into historical data but also to make informed predictions using the data all lowing users to themselves to meet future demand and other key metrics, niin aes Power BI is highly customizable, enables users to personalize dashboards and interfaces, create custom reports, éreate stunning visualizations based on well-known JavaScript libraries such as jQuery, D3, and even R-language scripts, and m¢ past ee g lore. Businesses can even share custom visuals created within an organization, clients, community and customers. For businesses that require more analytical and reporting power than that is offered by Excel, Power BI is the next level in business intelligence. Power BI revolutionizes how your businesses analyze, use and benefit from data. As businesses evolve, managing large datasets becomes ineffective and clumsy while using spreadsheets, As a result, companies face obstacles while analyzing their data for driving insights with informed business decisions, In contrast, Power BI is designed to handle a large number of datasets, and generate detailed reporting and analytical insights, ultimately boosting businesses with ability of making better decisions. Different Power BI versions like Desktop, Service-based (SaaS), and mobile Power BI apps are used for different platforms. It provides multiple software connectors and services for business intelligence. Whether it is using rich reports to connect datasets from workloads or dashboards to Consolidate key metrics, Power BI is a vital tool for consolidating data from a wide variety of different sources and analyzing it to make smarter data-driven decisions. It offers many benefits to businesses, and here are some reasons why companies use Power BI solutions. © Total integration: Power BI offers seamless integration with Microsoft’s and non- Microsoft’s tools for better data connectivity. ; © Trends and predictions: With AI and ML capabilities, users can analyze their data that can help businesses monitor important trends and even predict future trends. © Unlimited capacity: Power BI can handle vast data sets compared to other platforms that struggle to process large amounts of data.eta ale M Advanced Data Viswallzatlon ges can be Fest aSsureq rower BI i8 Fegularly 16 © Regular updates: With cloud computing capabilities, busines that they are using the Iatest features and functionality 4% es updated. ° Custom alerts: Businesses can set up important metrics are at the forefront of their tea © Data security: Power BI offers end-to-end data securit both cloud and on-premises data servers. © Interactive Visualization: With pre-built and C0. create interactive dashboards and reports for better” «© User-friendly: Power BI has an intuitive interfere ® levels making it fr more user-friendly than comPIe® ace alerts on specific KPIS t0 ensure that all the m's minds. ty offering granular controls for .d customizable templates, businesses can ts. iat is accessible to all tech skilj About Power BI The primary objective goal unorganized, complex data into extra exploring data, and reporting. Power BI is capable data models, analyzing datasets, building reports, from data for better decision-making. Power BI takes advantage of th machine learning models, recognizing patterns and images, and use these patterns to make pret estimates, users can generate forecasts and prepal in transform all of their BI is to help businesses n th oe insights The central focus of Power BI is data mining, of mining data from diverse SOUrCeS, building creating dashboards, and extracting insights in Mic data, build latest advances in Microsoft AI to prepare data sar in structured and unstructured data, including text scenarios. With these dictions and run “what if” re themselves for future trends and demands. Power BI - Features « Ease of learning: Since Microsoft Excel is widely used and globally accepted, earning curve is small from Excel to software makes Power BI easy to learn. The I Power BI, as itis developed on the founding platform of Excel. If you are familiar with Excel functions, pivot tables, and charts, you can easily see how simple it is to create Power BI data models and their visualizations. Data Model, Power Query Editor, and DAX Measures found in Power BI are similar to those in Excel, and thereby the users can quickly adopt to Power BI. © Data connectivity: Power BI offers seamless integration with both legacies as well as modem enterprise applications. With Power BI, businesses can import data from a wide variety of data sources, as most of the data sources offer connectors built and deployed in production. Power BI integrates easily with all Microsoft products, such as Microsoft Teams, Excel, Dynamics 365, Azure, and more. In addition, it offers third- bl ee a oun for data connectivity with services like Salesforce, GitHub, le Analytics, Hadoop, i ‘lable eerie nk Angee zat more for better leveraging all available data forwu parts ; e we rower Bl Seat eiteee: visualizations: Power BI enables businesses to create De ee SPen-source data visualization tools when it comes to visualizations. Power BI can work on a wid i be bull le range of browsers and devices as these visuals can be built on HTMLS and CS$3, and th ce D3 JavaScript library to bring data to life. Bre, they use the open-source D3.js sm visuals thoy mood to oxy usinesses can quickly find or create the cee d share thei ~” COVEr any common business scenario. Businesses can create and share their data visualizations with the Id ‘el Sateiil world for others to discover and use. user/developers can either start from scratch and i take aaneeciaae aed 4 t create stunning new visuals or take an existing visual and customize it as per their requirement. power BI can be embi eee ecdded into your custom apps: A positive aspect of Power BI is that it can be embedded with your existing business envi t stom apps bling businesses t bed eee eee es arts 's to embed Power BI reports, analytics, and features into web-based or mobile apps or other apps. Power BI allows Independent Software Vendors (ISVs) and embedment capabilities into their own apps on a pay-as-you-go, “white-label™ basis, and end-users won't be able to know that Power BI powers the app’s analytics. So businesses, rather than building their own reporting features right from scratch, can use Power BI in their products instead. e Power BI offers a mobile app: Stay connected to your data without any legacy to carry around wherever business takes you. Power BI has a mobile app for Windows, Android, and Apple devices, which makes accessing data stored in the cloud or on-premises SQL Server easy — provided there is internet connectivity. Mobile users can explore the capabilities of Power BI by creating and sharing reports, submitting natural language queries, and setting up push notifications to get alerts on their devices. Customized dashboards: Power BI allows businesses to build personalized dashboards that can be customized to fit their business requirements. These dashboards can consolidate disparate data from on-premise and cloud in a single view enabling them to monitor data. Businesses can seamlessly embed the dashboards, and BI reports in the applications to offer an elevated user experience. In addition, easy drag-and-drop functionality has empowered enterprises to create custom reports quickly. Power BI gets prompt updates: Power BI products get constant updates from Microsoft monthly to enhance data collection and business operations. Power BI user community has millions of users worldwide, where users can post their queries, suggestions and vote for ideas on how to improve Power BI. The idea or suggestion that gets the most rating has a good chance of getting included in the next update — meaning users have access to the latest and advanced functions, and features to help make accurate business decisions. Due to the inclusive and prompt upgrading system, Power BI has continuously improved its product since its release. © Power BI is affordable and cost-effective: Users can download the Power BI desktop version and start building reports locally. Power BI is relatively inexpensive and offers different versions so that you can pick the one that suits your business requirement.me Advanced Data Visual, Power BI Pro is available on « monthly basis at a Yow pee Per wt thai et fy ae and mid-sized companies. In contrast, Power BI Premium can be employed fy lnrge corporate organizations, © Dat exports can leverage its native R Integra computing andl graphies language for manipulat Data experts can leverage R to ere! import pre-built R-based visualization : BID With native support for R serps, users can run sD discclly aaa i a and import datasets directly into the Power BI Desktop oo ae mle, users can forceast statistical and machine earning capa! ‘ust for business USETS Why future predictions. - F tasks: Power BI is not j © Power BI is useful to manage IT tasks: Power BI ust Fo have large amounts of data that need to be analyzed for getting insights, but Bee Bl can visualize data for IT monitoring tools, It offers a solution template for Azure ‘Activity Logs and a range of other Tools, pre-built Power BI Desktop reports, an ‘customizable dashboards to visualize data effectively. tion: Wis a comprehensive saistiey ing, data and creating visualization, import or transform dita irq wate. visualizations directly into @ standard dashboard. Power BI- Intended Audience Busines intelligence tool has been primary used by business analysts and data scientists, But due to power BI’s user-friendly nature, it is being used by a number of people within the business. Power BI is helpful fr all sorts of executives ‘working across different departments and different industries. From midsize to large organizations software to banking/finance industry, heatheare to manufacturing industries, allcan benefit from Power BI, For any business that uses lege reams of dtaor smaller firms who simply need data visualization, Power BI is the ideal platform. In addition, Power BI can provide benefits across different departments such as providing valuable insights into all aspects ofthe sales department, for operations and inventory department | offers inventory and warehouse management. Inthe marketing department, Power BI can uncover | trends and guide the creation of marketing campaigns. Whereas in the case of the finance | department, Power BI manages all payable and receivable rebates. | Power BI Components/Tools The Power BI product is made up of many apps. These can be utilized independently or in a | blend with their own features and used to offer outstanding business intelligence solutions. Some "ofthe key components of Power BI are: Power Query: Power Query is the data connectivity and data preparation technology that enables you to import, transform, combine, and enhance data from multiple ee fom conan and cloud services, It offers a graphical interface for istion i it i ee ferent sources and a Power Query Editor for applyingyon power Pivot: Power pivot isa calculation engine used f uy models, setting up relationships between tables, and . modeling complicated data jin the value field area of pivot tables/pivot ties Me Performing complex calculations utilizes Data Analysis Expression (Dax) Paths In addition, Power Pivot segments. DAX includes an extensive inary was Pe manipulation to create powerful data models ta bn acity that enables deep data information for future decision-making, ing out some hidden yet valuable Power View: Power View is a data interactive charts, maps, tiles graphs, and other visualizations, filter evaluate your data from differer tables and matrices, and more. Visualization tool that empowers you to create ee ne other visuals, Power View can create charts 4 ighlight data. Slicers lets you compare and Mt perspectives, and Sorting to sort the columns in @ Power Map; Power Map visualizes geospatial data in a 3D mode that lets you look at the information in new ways. It enables you to plot geographic and temporal data on a addition, Power Map lets you view your data in geographic space and build cinematic guided video tours. It works with Bing maps to give you geographical latitude or longitude or a country, state, city, or street address data for better visualization. ¢ Power Q&A: Power Q&A lets users explore their data in their own words, encouraging them to ask questions about their data without using complex terminology. In addition, Power Q&A creates the best visualizations for the rendered query form of charts and graphs. Power BI Q&A: The Q&A feature in Power BI allows users to explore your data in their own words using natural language. Q&A is interactive, and allows users to travel through the data, refining or expanding your question, uncovering new information, zeroing in on details, or zooming out for a broader view. The experience is interactive and fast, powered by an in-memory storage. Power BI Q&A is free and available to all users. In Power BI Desktop, report designers can use Q&A to explore data and create visualizations. In the Power BI service, everyone can explore their data with Q&A. Data Management Gateway: Power BI Gateway is a software that is required to access data situated in an on-premises network. Gateway act as a gatekeeper for the on-premises data source. If anyone needs to access on-premises data from a cloud or web-based app, the request goes through the gateway. All the connection requests are attended by the gateway and access is granted based on their authentication and Tequirements. Power BI Data Gateway works as the bridge between the Power BI Service and on-premise data sources like DirectQuery, ImportQuery and LiveQuery. It is installed by BI Admin, © Data Catalog: Data catalog is a structured collection of data used by an organization. It is a kind of data library where data is indexed, well-organized, and securely stored. Most data catalog tools contain information about the source, data usage, relationshipsAdvanced Data Visuateag, » 120 ct ities as well as data lineage. This component allows the user to di at rouse qocrlea the Data Catto Metadata can be facilitated for seq, 7 and reuse queries using the 7a Scare, functionality. Types of Power BI Tools © Power BI Desktop: Power BI desktop is the primary authoring and publishing too »,, Power BI. Developers and power users use it to create brand new models and repe, from scratch. Power BI Desktop is free and intended for small- to mid-size busi You can create, share, and publish interactive reports with visual analytics anytim. anywhere, and on any device. One of the drawbacks of Power BI Desktop is that j offers only 10 GB of cloud storage and uploaded 1 GB at a time. © Power BI Service: Power BI service is a cloud-based business analytics and dat, visualization service that enables anyone to visualize and analyze data with greate, speed, efficiency, and understanding, It connects users to a broad range of data through easy-to-use dashboards, interactive reports, and compelling visualizations that bring data to life. The Power BI service website is built upon the Azure Cloud platform ang adhere to HTML 5 standard, It has robust support for HTMLS via what is calle PowerView. It allows you to share data views and workbooks, which you can refresh jodels, reports, dashboards ‘in the ‘cloud. Pro license: $10/users/month. Power BI Service Pro is paid version that charges on a per-user basis with all the features that power BI can offer, suitable for a mid-size business. In contrast, the Power BI Service Premium is licensed as per the organization’s capacity and intended for large businesses and enterprises. Power BI service is hosted in the Microsoft Azure cloud. You get access to advanced AI that accelerates access to insights, self-service data preparation capabilities and simplifies data management. © Power BI Report Server: It can host paginated reports, KPIs, mobile reports and Power BI Desktop reports. It is updated every 4 months and installed/managed by the IT team. The users can modify Power BI reports and other reports created by the development team. Power BI Report Server is the on-premise Power BI solution for enterprises that need to protect their sensitive data and have their own dedicated server for managing those data. In addition, the Power BI Report Server offers the flexibility to move to the cloud in the future. Finally, for businesses that need to follow data Protection and regulation laws, depending on their industry or their geographic location, Power BI Report Server is an ideal option. © Power BI Mobile Apps: Power BI Mobile App is available for iOS, Android, Windows. It can be managed using Microsoft Intune. You can use this tool to view Teports and dashboards on the Power BI Service Report Server. The native app forf , . ort power BI empowers businesses to mI Stay ove you Seourly acces, view, ra ee with the business data on the go. 7 ; : joar create FEPOFS, FeceivE PUSH notifications ty person; ras a nd reports, Allows you to more from any digital device, wit ‘al data alerts, make quires, and “Android. lobile BY apps for Windows, iOS, and power BI Embedded: Microsoft offer ° : dent software vendors to emb : independent " peers \eir own apps; rather than having to build an analytical toot right from Scratch, you can simply ind Power BI into their apps. It works on a pay-as-you-go model, and ali the reporting and data analysis Will be taken care of by Power BI, th native m, Sa ‘white label? version of Power BI for ‘d Power BI in th ower BI Des ‘A workbook can have data entered manually or data, which is queried and loaded] from external data sources, | Files are simple text files with rows of da impl ta. Every row can contain one or more values, which is separated by a comma. | ‘You can use Power BI Desktop to query and load data from external data sources. | Kt allows you to connect live to Azure SQL Database, Azure SQL Data Warehouse, etc. You can connect directly to SQL Server Analysis Services Relational model} databases. A Power BI Enterprise Gateway is required. Architecture of Power BI Power BI Architecture Hundreds of data sources Power B service ORD ws Power Bt Desktop a OE ‘voor omsasecarions Bate ig. 3.15: Architecture of Power BI2 “Advanced Data Visualzayy © Data Integration: An organization needs to work with data ue somites different sources which can be in various file formats. The data eed This a from a different source which can be from different servers or databases is integrated into one standard format in a common staging MCN rye visualization © Data Processing: In this stage, the integrated data ig still no ee example, redundan, as the data needs processing, This data is pre-processe values, missing values will be removed from the data set- data is cleane d. Youan load The business rule should be applied to the data when the that data back to Data Warehouse. * sea, it can be visualized mu © Data Presentation: Once the data is loaded and apres ‘o offer. Use of dashboard better with use of various visualization thet Ba 1 sual report helps busines re intuitively and report helps one represent data mo} Me ond-uses to take business decision based on the insig) Power BI Dashboard ‘The figure below demonstrates th e Power BL Dashboard: Store Sales Overview sec tanrreen Soto an avo Be RIS : Baws Fig. 3.16: Sample Power BI Dashboard Power BI Desktop is an enlargement tool used to generate dashboards and reports. Its plications will be accessed by other users through desktop and mobile devices.ws 123 He ip Powe? BI f function ae ph 1 Analysis Expressions) is a f x (Data 4 formula expressi F ik ' ion language whic be used with jos wsuzaion peril poner BL Itis also known as a Tago iomaeehee the full oe js kept inside | DAX programming formulas contain two data types: Numeric and thet: , . * js linked to the other by having 7 gach is h g common colum is 3 gunn Eso showing the relationships, ta: Here ia siple dagram OLY : Fig. 3.17: Power BI Dashboard with Relationships DAX can be used in Power BI for the following: © Calculated Columns © Measures © Tables Let’s examine how DAX functions works: 1. Caleulated Columns: Calculated column allows y' the given data. ou to create new columns based on124 “Advanced Data Visueltzatoy F . Da aimple, there is no ‘Final price’ column available in the Pe table: Here, t, function is used to calculate anew column when only total pFIC® an! HAY ar available, ; i = List_Items[MRP]*List_Items[Qty] ate ues ill ” eas ie Ss ‘ ith y sures: You can perform a calculation using ee “a ats displayed, withou data. This is very helpful for reports where the Tequiring an entirely new column to store it. For example, Total of the MRP column * Total of Qty column 3. Tables: DAX functions in tabular model return entire tables. For example, in order to generate a list of all the country the organizat use the function: cities touched = DISTINCT(Customers[City]) sanction ee 7 + Some important DAX functions are: ; 4, Power BI DAX Functions: Ip ve find the average from a given set of @ Average: This DAX function allows you ‘aloes as shown in the below Power BI example. Example: AvgComm = AVERAGE(List_Items[Price]) e Mazz Helps you to find the maximum from a given set of ValUSS Example: Find out the highest order. HighSale = MAX(List_Items[Price]) ¢ Min: Helps you to find the minimum set of values. Example: Allows you to find out the lowest order. LowestSale = MIN(List_Items[Price]) ‘e Count: Count any numerical data. Example: Count number of ticket issued. TicketVolume = COUNT(Invoices[Ticket]) © Concatenate: This function helps you to join values in calculated columns. You on has clients in, can use Concatenate if using in measures. Example: Concatenate the Item names, and MRPs will give a unique code for all the price points at which each product is sold. ProMrp = CONCATENATE(List_Items[Item], List_Items[MRP]) © TotalYTD: This function allows you to calculate the sum from the start of the current year to the specified date. It performs calculate base on a calendar year, not a financial year. : .ns gxamp cumisates = TOTALYTD(SUM(List_ltems{Price),Invoices{Date)) all — Returns everything. Ignores filters, fe: Calculate Sales totals for the price cotum n. eS power BI? ger BI 0018 ATE VEY useful for the following job categories: Por pMO- Project and Portfolio Manager : Business and Data Analyst ; e Dev eloper and Database Administrator e IT Team and IT Professional | consumer for End-user Report @ Data Scientist of Power BI advantages . vera the advantages of using Power Bl: '¢ Offers pre-built dashboards and reports for SaaS Solutions. ‘¢ Provide real-time dashboard updates. ‘e Secure and reliable connection to your data sources in the cloud or on-premises. « Offers quick deployment, hybrid configuration, and a secure environment. © Allows data exploration using natural language query. Offers feature for dashboard visualization. © New features frequently added that are great for Excel users. e Extensive database connectivity capabilities Q&A feature publish to the web. ¢ Integration with both Python and R coding to use visualizations. © Provides many options related to wrangling and clean the dats. © Post publishing the data into Power BI web service can schedule refresh without manual intervention. © Power BI backed by the superpower of artificial intelligence and machine learning. Disadvantages of Power BI Here are cons/drawbacks of using Power BI: © Dashboards and reports only shared with users having the same email domains. © Power BI will not mix imported data, which is accessed from real-time connections. © Power Bi will not accept the file larger than 250 MB and zip file which compressed by the data of X-velocity in-memory database. © Power BI cannot accept file size larger than 1 GB.Advanced Data Visuatigy, 7 126 ity parameters. ¢ Dashboards never accept or pass user, account, or other entity para ete Re ; rts an © Very few data sources permit real-time connections to Power BI rep' hboary Summary i actionable information which hg, ss decisions, ke informed busine: © BI helps you to analyze data and to provide tion tool which helps you ,, corporate executives and business managers (0 m2 © Power BI is a Business Intelligence and Data Viswa convert data from a various data source. © Some important Power BI tools are: (1) Power BI (3) Power BI Data Gateway, (4) Power BI Report -esv), Power BI Desktop (phi) Apps. ( Value © Excel (xlsx, xixm), Comma Separated Val’ ) ) ee in the Cloud and Databases on-premises are important data SOUrces used in rts and Tiles are important terms used in Desktop, (2) Power BI servic, Server and (5) Power BI Motit, Power BI. Visualization, Power BI. Power BI was conceptualized by Ruler services team at Microsoft. Data Integration, Data Processing and Data Presentation are important components of Power BI architecture. ¢ Power Query, Power Pivot, Q&A, Data Management Gateway a1 BI Dashboard. DAX (Data Analysis Expressions) is a formula expression language which can be used Data sets, Dashboard, Rep and Dhers Netz of the SQL server coverage Power View, Power Map, Power BI Service, Power BL nd Data Catalog are important elements of Power og with various visualization tools like Power BI. Power BI offers Complete Dashboards for analysis, while Tableau only allows Ad Hoe Analysis. Important professional who uses Power BI is PMO (Project and Portfolio Manager), Business and Data Analyst, IT Team, IT Professional, etc. ~ @ The biggest drawback for Power BI will not accept the file larger than 250 MB and zip file which compressed by the data of X-velocity in-memory database. Power BI has revolutionized how businesses use data. It is a powerful tool to transform and . es intelligence data. It empowers enterprises to analyze their data and provide real, ee me and actionable insights with interactive dashboards and rich reporting from business Fn levees poner ot fave started employing Power BI after realizing its benefits. Businesses 4 T fo’ improve thei insi; izati - ven bins deckgnts iprove their data insights across the organization, and make a datane px BI DASHBOARD ~ HOW TO CREATE any USE THEM? nd Braphs to tell a story is called a Power ad is also known as a Canvas, The Power I Service, Since a Power BI dashboard is story. You cannot create a dashboard on A ee cclteatt ‘ re ste pase visualization with multiple charts ay yg. This one-page visualization in a dashbo: My ig a feature only available in Power B fg ne pages it only contains the highlights of ite" a pesktop- pn ah J pont Sie Tet Fig. 3.18: Sample — Sales Dashboard “Ina dashboard, visualizations are generated from reports, and each report is based on one daa set. The visualizations present on the dashboard are called tiles, and report designers pin hese tiles to the dashboard. Advantages of a Power BI Dashboard ¢ A Power BI dashboard enables users to analyze reports and view all important metrics ata glance. © Using a Power BI dashboard, users can create visualizations from muttiple datasets or multiple reports. _ © You can customize dashboards to meet the requirements of any enterprise. © Power BI dashboards can be embedded into applications to provide a unified user experience. © You can instantly share a dashboard with other colleagues in your organization. Sample Dashboards For example, the ’ following image displays a sample sales report which provides lot of ion to the sales team.ian Advanced Data Viualtay, a n TP Ka 367,086 € eure : Per etary 780,508 ¢ ul As Sel ms ey ASTI] s} 104 CUSTOMUTLI LIME VALUE customer AcauION cost clesasr oni hiiiiiaia Fig. 3.19: Sample ~ Sales Performance Dashboard The sales performance dashboard shown above gives a complete overview of sales targets and insights on whether the team is completing their individual objectives, Financial analytics can be kept under control with its numerous features that can remove complexities and establish a healthy and holistic overview of all the fin ancial information an organization manages.St sei wane tsa ooo BE 2mm a 3.41 Care nets i; 49700) wma Be a TTL end Acoma neamse € ona wea OO ie ‘bvertory € 312262 ena Pepto atet Fe tat ynye Long-Term Assets € e000 ~ =e HEB GEER wns ¢ ams no BOO Cmentunme ¢ ones Healla la 0.41 AcomnPaatie ¢ sane Reelin Oe Omrertiabiees —g an ae Sevesereaty € 44880 we TO Conmonsock € soaso BUREE RE Pt gts Pati Pe . SxetEemngs € aneo we Fig. 3.20: Sample — Financial Dashboard On the financial dashboard template above, the financial overview is concentrated on how ecient is the overall spending of the capital. The balance sheet gives an overview of the main ics which can easily define trends and the way crucial assets are being managed. __ Itdoesn’t stop here, Another business intelligence Teport sample can be applied to logistics, one of the sectors that can make the most out of business intelligence and analytics; therefore, sily ‘rack shipments, returns, sizes, or weights, just to name a few. __Tohancing the recruitment process with HR analytics tools can bring dynamic data under he umbrella of BI, making feedback, interviews, applicants’ experience, and staffing analytics {© process and derive solutions. This is just one business intelligence report sample that can ‘loped in more detail by establishing the right KPIs, and developing a strategy and goals. ting a Dashboard in Power BI Building reports on Power BI Desktop and publishing them to Power BI Service is one of “Ways to understand how a Power BI dashboard works." Advanced Data Visualsany Vovvor IL Dastonrd ‘Power Bako sa single pe often called a cnvas that uses visualizations to a ‘story Devnuse tis Hnited ene pag, a welltesigned! dashboard contains only the ms, inpovtant elements of that toy. he visualizations visible on th dashboard are know ates. ‘These tiles are pinned oth shboat hom reports, The visualizations ona dashboard come fram reports and each report thd on one dataset, fet, ene way to Fook a dashboard is to consider it as an entry poin, into the underlying reports ancl data sets, Selecting a visualization takes you to the report (anq «lata set) which was used (0 create i, Tr ae aes Fig. 3.21: Creating a Dashboard Dashboards are a wonderful way to monitor your business, to look for answers, and {0 se your most-important metrics at a glance. The visualizations on a dashboard may come from one underlying data set or many, and from one underlying report or many. A dashboard combines ot premises and cloud-born data, providing a ‘consolidated view regardless of where the data lis.