21/05/2021

Db2 Web Query for i

Doug Mack
mackd@us.ibm.com
Hernando Bedoya
hbedoya@us.ibm.com

Db2 Web Query Team email:

QU2@us.ibm.com

© 2018 IBM Corporation

Important Links

• Db2 Web Query for i Website

– Ibm.biz/db2webqueryi
• Db2 Web Query for i Wiki
– Ibm.co/db2wqwiki
• Db2 Web Query Getting Started Enablement:
– https://ibm.biz/db2wqconsulting
• Db2 Web Query Version 2.2.1
– http://ibm.biz/db2wqv221blog
– Videos: http://ibm.biz/db2wq-221-videos
• EZ-Install
– http://Ibm.biz/db2wqezinstall-info
• Recent Webinars
Follow Db2 Web Query guy Doug Mack on
– New Version 2.2.1 Visualizations: http://ibm.biz/db2wq-221-videos twitter at @mckdrmoly or blog
– COMMON Advanced Reporting Tips Virtual Conference http://db2webqueryi.blogspot.com/
http://www.common.org/virtualcommunity

© 2018 IBM Corporation

1
 21/05/2021

Current Pain Points?

• Only the database expert understands the data enough to build reports?

• Out of control ad-hoc queries (impact on performance)

• Data Security – can anyone see the data within a table/file?

• Building customized extracts to fulfill requests to I/T for data

• Maintaining business rules about the data

– What happens if our rule about sales commission changes?

• Maintaining historical perspective

• INFORMATION vs. DATA – easily spot trends NOW vs. wait until its too late

• Query/400 is OLD OLD OLD…OLD technology

© 2018 IBM Corporation

Benefits of Db2 Web Query

• Create “Controlled” Ad-hoc reporting environment
– No “free form SQL” allowed!

• Create “Self Service” reporting environment

– Get I/T out of the business of building extracts or
– Let end users run reports with selection criteria they want, when they want it, in the form they want
it in (dashboard, spreadsheet, visualization)

• Keep the data, but also the reports and visibility to files secure

• Enable interactive data analysis – not just static reports

• Automate building of an infrastructure to support the analytics

• MANAGE the whole BI stack just like you do any other IBM i Licensed Program Product
– HA/DR, Backups, Security, PTF Management, Start/Stop, Work Management

© 2018 IBM Corporation

2
 21/05/2021

IBM i Business Intelligence/Analytics Supporting Attributes

Data Usage Analysis Visualizations

Db2 Web Query for i Connections to
Warehousing
Metadata Watson/IBM BI/DW Tooling
DataMigrator ETL Cloud
Database Show
Statements Visual Explain SQL Plan Cache Index Advice DBE Tooling
Monitors

Encoded Vector Maintained Aggregations SQL Query EVI Index Only

Indexing Temp Indexes in EVIs Engine (SQE) Access
Star Schema Query Accelerators
Adaptive Query Materialized Result Set
Symmetric Multi- Optimization
Processing Query Tables Caching
Processing

SQL Aggregate SQL OLAP

SQL Set at a Time Functions Extensions SQL Language
Rank, Dense SQL Stats
Grouping Sets,
Rank Functions
Rollup

Single Level Store Large Memory Model IBM i Architecture

Delivered at various points over time

© 2018 IBM Corporation

IBM i Modernization Focus Areas

• Independently approached, interconnected topics
– Pursuing any one of these provides long term ROI
• Strategic and Tactical elements in each focus area
– Develop long term plan while delivering value to the business in the short term
Application Database Reporting/Analytics
Modernization Modernization Modernization
Leverage available skills Faster response to requests Improved data analysis
Benefits React with agility to changes Better data governance Empowered end users
Position for the future Improved data access Build “system of insight”

Modularize for flexibility Un-couple data access from RPG Modernize Query/400
Approaches Improve User Interface Shift to SQL foundation Leverage integrated BI solution
Enable for any HLL usage Enforce business rules in database Deliver interactive reports, dashboards
Phase in - for minimal disruption Visualize data for better analysis

“Our modernization of reporting tools with Db2 Web ©Query SAVED the AS/400 in our company” – Gary West Huhtamaki
2018 IBM Corporation

3
 21/05/2021

What DATA can you use in a report?

DATA SOURCES
Query/400 SQL Scripts Aggregated Data Programs (defined Excel
SQL Views (EVIs, MQTs) as Stored Procs) Non Db2 for i
Files/Tables Data Sources

User Defined Single

Date Attributes Version
Functions
Calculated Fields SQL Functions of the
TRUTH
SYNONYMS

ETL BI

Data
Warehouse
Reports/Dashboards/Visualizations or even a Data Warehouse

© 2018 IBM Corporation

Benefits of the Metadata Layer

• Define ONCE, Use Everywhere
– Data Integrity “Single Version of the Truth”
– Easier to change a business rule ONE TIME vs. in each and every report
• Ex: “LATE DELIVERY” is now defined as 1 day late vs. 2 days late

• Report Author Productivity

– All joins, virtual fields, field formatting, date conversions can be handled in metadata, not at the
report authoring level
– Allows many different data sources to be leveraged
• Not just your files, but uploaded spreadsheets, SQL Views or User Defined Functions, Stored
Procedures
– Ex: Define a function that converts U.S. Dollars to Canadian dollars on the report

• Documents data relationships from RPG logic

– File level relationships
– Data dimension (heierarchies) relationships
– Pull out the RPG (only the code knows) data relationships
• Ex: If field/column 1 = A, the field/column 2 = “REGION”, else, = “DISTRICT”
© 2018 IBM Corporation

4
 21/05/2021

We will hear more about this later!

© 2018 IBM Corporation

Db2 Web Query for i – Intuitive, Insightful, Extensible

Fast, easy access to business information assets for query, reporting and analysis

Authoring Tool Reports & Graphs

“Intuitive”
“Flexible delivery”

Dashboards OLAP
“Insightful” “Extensible”

INTEGRATED Business Intelligence and Data Warehousing for IBM i

MODERNIZE Query/400 or RPG Reporting Processes
LOW COST IBM i Analytics Infrastructure
ALIGNED with Db2 for i Advanced Query Processing
© 2018 IBM Corporation 10

10

5
 21/05/2021

Db2 Web Query Product Suite 5733-WQx

Chances are HIGH
– Express Edition you already own
 Entry level edition license rights to
 Report development and all report output options supported Express Edition
 Great for small, getting started environments
 User licensing is NAMED USER concept

– Standard Edition
 Adds report distribution, integration APIs
 Adds non Db2 Database Adapters
 Virtually unlimited runtime users support
 Optionally add JDEdwards Adapter

– DataMigrator ETL Edition

 Automate Data Extraction, Transformation, Replication and LOAD
 Consolidate data, isolate query/BI workloads
 Requires Express (Db2) or Standard (Non Db2 data sources)
 And at least 2 license of Developer Workbench

 Add 1 or 2 licenses of Developer Workbench to either Edition

© 2018 IBM Corporation

11

New Db2 Web Query Editions

Db2 Web Query Scheduler Edition 5733-WQB

– Express + Job Scheduling
– Unlimited report “consumers”
• Distributed via email, FTP,
or placed on network drive
• Users can work with data offline
• Support Mobile or At Home Workers

Db2 Web Query RunTime User Edition 5733-WQR

– Express + RunTime Licensing OVER 50% Savings for 20 user system
– Users can run interactive reports with live data OVER 75% Savings for 50 user system

Savings compares adding user licenses to Express vs. Upgrading to new Edition
Calculated using U.S. List Prices and includes SW Maintenance
ibm.biz/db2wq-blog-neweditions Assumes you already own Express Edition + 2 user licenses

12

6
 21/05/2021

Delivering INFORMATION from Data

• Interactive Dashboards

• Mobile Reports and Dashboards

• Excel Integration

• Schedule Reports and push to consumers

• Integrate reports/dashboards with your own (portal)

applications

• Slice and Dice for the Business Analyst

© 2018 IBM Corporation

13

Install with the EZ-Install Package (Request by sending

email to QU2@us.ibm.com w/ your system s/n)
– After completion, go to ibm.biz/db2wqezinstall
• Test Drive what you now have
• Learn how to build reports/dashboards
• Run Query/400 Discovery Tool
• Learn about Developer Workbench
• Learn about DataMigrator ETL
• Learn about Date Dimension Table

© 2018 IBM Corporation

14

7
 21/05/2021

User Interfaces

• Browser: “BI Portal”

• Mobile: “Mobile Faves” App for Apple and Android

• Excel: Plug in

• Roll Your Own: WQRAX URL Construction API

© 2018 IBM Corporation

15

Build/Edit Reports and Dashboards with InfoAssist

• Web based Report Authoring component (built into Express or Standard)

• Invoked from BI Portal (web) or Developer Workbench (Windows)
• Intuitive office-like interface
– Tool bars (Ribbon) change based on selected tab
• WYSIWYG layout area
• Build reports, charts, graphs, maps, dashboards
• Output types
– Excel including pivot tables
– “Active” technologies (good for mobile devices)
– PDF, File, Print, Database
– Auto drill down (OLAP) report
– HTML, HTML5, Flash
– Powerpoint
– User can select at Run Time

© 2018 IBM Corporation

16

8
 21/05/2021

IBM Db2 Web Query for i – InfoAssist

• Sort, Rank, Make it a top xx report

• Add headers, footers
• Add breakpoints, sub totals, grand totals
• Add column or row totals
• Change output types
• Change style sheet
• Add prompts/parameters
• Add traffic lights, Data Bars
• Change to one of the over 100 charts and graphs
• Add table of contents or pages on demand
• Add features like accordion collapse and expand sections
• Stack Measures
• Create link to a drill down report or URL

© 2018 IBM Corporation

17

IBM Db2 Web Query for i – InfoAssist

18
© 2018 IBM Corporation

18

9
 21/05/2021

Dynamic Data Analysis Report Types

• OLAP Reports
– Dynamic end use interaction with data against data source real time
• Drill up/down, Filter, Add/Remove Columns
• NOTE: Not enabled by default

• Auto Drill Reports

– Simple enablement of drill down and up within a report

• Auto LINKED Reports

– Simple enablement of linking a value in a parent report to another report
expecting that value as a parameter

© 2018 IBM Corporation

19

Report Types and Considerations

• Hyperlinked Reports
– More customized links from a report to
• Another report
• A website
• A BI Portal Refresh

• “Active” Reports and dashboards (Documents)

– Run once to get data – data set if fixed, views can be more dynamic
– Work with data offline
• Sort, filter, data bars, derived fields, roll up, pivot, extract to Excel
– Great for mobile devices
– Great for scheduling and routing to users
• Users run disconnected from server (controlled ad-hoc!!!)

© 2018 IBM Corporation

20

10
 21/05/2021

Report Types and Considerations

• Visualizations
– Interactive Dashboards
• Live data
• Drill downs and ups are automatically
built for you
• Filter controls automatically built
• “Lasso” filtering within view

– Chart/Map oriented dashboards

© 2018 IBM Corporation

21

Now that you have a report let’s turn it into an “OLAP” Report

 A single drill down (it’s really “drill ANYWHERE”)

report is a starting point for interactive, real time
multi-dimensional data analysis
–Drill down
–Add or remove fields/columns by dragging and
dropping
–Pivot rows and columns
–Navigate across any dimension
–Create charts from navigation point
–Save report based on navigation point

–Watch video at ibm.biz/db2webqueryi

– (take “View Db2 Web Query End User Demos” link)
© 2018 IBM Corporation

22

11
 21/05/2021

Dump the Data Into an Excel Pivot Table

• Sales Report defined with Excel Pivot Table Output

© 2018 IBM Corporation

23

Create/Edit/Run Reports Directly from within Excel

• Using the Excel add-in

• Create ad-hoc form to simplify parameter selection for end users

© 2018 IBM Corporation

24

12
 21/05/2021

Schedule the Report to Run in the Background

‒ Db2 Web Query Specific Job Scheduler
 GUI Interface
 Standard or customized schedules
 Can exclude dates (e.g., holidays)
 Email, FTP, or save to repository
 Burst the report based on key field
 Logging and notification functions

‒ CL Command to add to your IBM i job scheduler

 Run job as part of a job stream
 After end of day processing run reports and distribute
 Can pass parameters from CL program
 Email, FTP, Save to IFS or network driver
 Include report as email attachment or link to file in in IFS or network drive
© 2018 IBM Corporation

25

Dashboards
• Personal Dashboards – My Favorite
Reports on a single window pane

• Active Dashboards – build once, alter the

views of data
– A nice fit for Mobile or scheduled
Dashboards

• Highly Customized Dashboards/Guided Ad

Hoc Forms

• Interactive Dashboards: Visualizations

© 2018 IBM Corporation

26

13
 21/05/2021

Spotting Trends in Data with Visual Analysis

• Db2 Web Query Version 2.2.1 adds a new category of reports/dashboards: VISUALIZATIONS
– More quickly recognize trends or correlations across large data sets with Interactive Dashboards
– Iterate through the visualizations with drill downs, filtering, hyperlinks to other reports
• Video Demonstrations at http://ibm.biz/db2wq-221-videos or accessible within the product

© 2018 IBM Corporation 27

27

Robust Set of Visualizations to Choose From

© 2018 IBM Corporation 28

28

14
 21/05/2021

Demo Time!

© 2018 IBM Corporation

29

Push the Work to Db2 for i

• See “To SQL or not to SQL” Blog Post at
db2webqueryi.blogspot.com

• There are many ways to integrate database aspects into Db2 Web
Query
– Can often improve performance
– Can often simplify difficult reporting challenges
– Can leverage what you do in Db2 in other applications

• Remember, you build reports over a REPRESENTATION of the

underlying data source. That data source can come from:
– Db2 Views
– Db2 Stored Procedures that return a result set
– Date Dimension table (more on this later)
– Db2 Functions (including USER DEFINED Functions)

© 2018 IBM Corporation

30

15
 21/05/2021

Salary Analysis

• Your HR department wants to ensure salaries are equitable across the company and
across departments and there aren’t outliers or other discrepancies

– What is the employee’s salary compared to the average WITHIN their department?

– What is an employee’s ratio of salary within their department and overall company?

– For each employee, compare their salary to the two closest behind this person’s salary,
and the two closest ahead of them in salary

© 2018 IBM Corporation

31

Db2 for i OLAP Specifications to the Rescue !

With IBM i 7.3, Db2 for i supports the following additional OLAP specifications for
summarizing and referencing rows within windows of result sets:

 Aggregates (SUM, COUNT, etc.)

 LAG and LEAD
 NTILE
 FIRST_VALUE, NTH_VALUE, and LAST_VALUE
 RATIO_TO REPORT
 CUME_DIST

What is the value of this?

 You can address more complex analysis in a much simpler way (than the alternatives)
 BI tools like Db2 Web Query can “push the work” to Db2 for better performance

© 2018 IBM Corporation

32

16
 21/05/2021

The Process

• Create a SQL Views (or script) using new 7.3 SQL Extensions

View for Salary compared to AVERAGE View for 2 ahead, 2 behind

© 2018 IBM Corporation

33

The Process
• Within Db2 Web Query, create a “synonym” (term for meta data) over each of the SQL Views

These are fields returned

from the view

© 2018 IBM Corporation

34

17
 21/05/2021

The Process

• Build Your Reports/Charts/Dashboards with Db2 Web Query InfoAssist

– Add additional filters or virtual fields
– Format header/footer/stylesheet
– Choose output
• Excel, HTML, mobile
– Add to dashboard
– Embed in your app
– Feed into your data warehouse

These are fields returned

from the view and
available in your report

© 2018 IBM Corporation

35

Examples

© 2018 IBM Corporation

36

18
 21/05/2021

QUESTIONS?

© 2018 IBM Corporation

37

DataMigrator ETL (Extract, Transformation, Load)

 Product ID 5733-WQM
‒ Requires either DB2 Web Query Express or Standard Edition V2.1 with
Group PTF Level 11
‒ Core based. Must have same number of cores as DB2 Web Query
core licenses
‒ Standard Edition required for non DB2 relational databases as a
data source
‒ One license of DB2 Web Query Developer Workbench is required to set
up data and process flows

 ETL for IBM i

‒ Like DB2 Web Query, all server code and ETL objects run in and/or are
stored in IBM i

 INTEGRATED with DB2 Web Query

‒ Can leverage meta data created with DB2 Web Query or vice versa
‒ Same look and feel
‒ Shared services and administration © 2018 IBM Corporation

38

19
 21/05/2021

DataMigrator Use Cases

 Data Replication
‒ Ongoing Log (Journal Receiver based) in near real time
 Data Migration
‒ One time or on going data synchronization across different databases
 Data Preparation
‒ Consolidate and/or transform data in preparation for feeding into
predictive analytics or Watson
 Data Warehousing or Data Mart Automation
‒ Timed ETL
 Spreadsheet Consolidation
‒ Automate upload of spreadsheet of flat file data to offer reporting
combining this data with DB2 data
© 2018 IBM Corporation

39

Prototype Example: Augmenting Operational Data with Weather Data

Augment Data with

Weather info
Visualize/deploy insights with
Db2 Web Query

IBM Bluemix

Db2 Web Query

Db2
for i for i
Order, Product, DataMigrator
Inventory and Datamart
measurements
(revenue, COGS)
Information

Db2 for i

Power Systems(IBM i)

© 2018 IBM Corporation

40

20