AVS Automation and Reporting Requirements

Tamar Piehler
Spring 2022
AVS Automation and Reporting Requirements
April 2, 2022

ASSUMPTIONS: ................................................................................................................................................ 3
USERS, GOALS, AND SECURITY .......................................................................................................................... 4
REQUIREMENTS FOR ALL REPORTS ................................................................................................................... 5
DATA VALIDATION REPORT REQUIREMENTS ..................................................................................................... 5
DATA MANIPULATION REQUIREMENT .............................................................................................................. 8
AUTOMATION REPORT REQUIREMENTS ........................................................................................................... 8
6 REPORTS CORRESPONDING TO VCP AUTOMATION:............................................................................................................ 8
v1 RI Consumption Report [For contracts 13398274 (US) and 24942417 (RoW)] ............................................... 8
v1 OnDemand Consumption Report [For contracts 13229759 (US) and 24855606 (RoW)] ................................ 8
v2 RI Consumption Report [For contracts…] ......................................................................................................... 8
v2 OnDemand Consumption Report [For contracts…].......................................................................................... 8
v1 New RI Bookings Report [For contracts…] ....................................................................................................... 8
v2 New RI Bookings Report [For contracts…] ....................................................................................................... 8
DASHBOARD REQUIREMENTS........................................................................................................................... 8
HIGHEST PRIORITY USER REPORT REQUIREMENTS ............................................................................................ 8
NEW RI BOOKINGS REPORT (V1 AND V2) ........................................................................................................................... 9
CUSTOMER COUNTS REPORT ............................................................................................................................................ 9
HYPERSCALER DASHBOARD INFOSHARE .............................................................................................................................. 9
RI HIGH-LEVEL NODE BILLING TREND REPORT ..................................................................................................................... 9
ONDEMAND HIGH-LEVEL HOURS CONSUMPTION TREND REPORT ......................................................................................... 10
RI’S BY EXPIRATION ...................................................................................................................................................... 10
FY23 NET NEW NODES REPORT (NEW DASHBOARD ITEM) .................................................................................................. 11
CUSTOMER REVIEW REPORT........................................................................................................................................... 12
PHASE 2 REPORTING & DASHBOARD REQUIREMENTS ..................................................................................... 12
CHURN RATE................................................................................................................................................................ 12
TOP OD CUSTOMERS BASED ON ROYALTY PAID .................................................................................................................. 12
TOP RI BOOKING CUSTOMERS ........................................................................................................................................ 12
PARTNER REPORTING .................................................................................................................................................... 13
MORE TBD ................................................................................................................................................................. 13
SUMMARY ..................................................................................................................................................... 13

VMware Company Confidential 2

AVS Automation and Reporting Requirements
April 2, 2022

Assumptions:
1. It’s assumed that all the data in the database will have backups and disaster recovery
maintained by VMware IT.
2. It’s assume that all data is secure, and available only for those people who are given
access, or whose groups to which they belong, are given access.
3. It is assumed access to Customer Names and Geos can be limited by user and/or group.
4. It’s assumed that there will be a way to associate all Countries to sub-regions and Geos
[“DC Country” or “Area” (a.k.a. “HQ Country”) → Subregion → Geo]. A table can be
provided. Maintenance of the table must be possible by IT Ticket or by AVS access.
5. It’s assumed that the customer names that we receive in a mapping table can be
combined with the consumption data, to provide customer names to a limited number
of people, hopefully at the last moment the access is requested. If it is co-mingled and
stored with the data in the database, then we will need to assure that #2 & #3 above,
restricts who can see any data with customer names in it.
6. Demonstrations on current billing and booking procedures and the use of VCP are
sufficient for the automation team to calculate the billing and booking info and build
systems that input this into VCP, while at the same time, providing a report for human
review and validation. Because the automation of this part of the system is nearly
complete, I assume documenting this has already taken place somewhere else by the
Automation team.
7. It is assumed that the database that will store the AVS data, with the proper setup for
access, can accessed with Mode, Tableau, and Microsoft Power BI without
export/import.

VMware Company Confidential 3

AVS Automation and Reporting Requirements
April 2, 2022

Users, goals, and security

As an/a… I need access to see Security of data
Corp Sales  Access to all WW Data except customer names WW data
Executive  Primarily interested in high-level views, trending, and No Customer Names
measurable KPIs Watermarking: Yes
 Drill in capabilities by Geo and sub-region
AVS Specialist  Access to data for their customers or subregion Subregion data only
or SA No Customer names
(for now – hope to
track their
customers)
Watermarking: Yes
Sales Geo VPs  Geo Data Access – for Customer s with HQ Country or Geo data only
or AVS Mgrs Data Center Country in their geo Customer Names:
 Access to all types of reporting Yes
Watermarking: Yes
AVS HQ VP,  Access to all data, and able to download the data to CSV All data
Mgr, BI or Excel Customer Names:
Yes
Watermarking: Yes

Sales Ops HQ  Access to all data, and able to download the data to CSV All data
or Excel Customer Names:
Yes
Watermarking: Yes

Sales Ops Geo  Lookup for specific customers as sales core inquires Geo data only
about circumstances Customer Names:
Yes, but only one at a
time. Reports with
more than one
customer name will
provide only
customer IDs
Watermarking: Yes
Marketing  Will want a duplicate of their dashboard at WW data and limited
https://app.mode.com/vmware_inc/reports/fa8a25353407 to only certain
reports.
No customer names.
No export ability.
Watermarking: Yes
BU  Reports will focus on DC countries and technology info
related to usage. Need to contact the BU for a list of their
needs.

VMware Company Confidential 4

AVS Automation and Reporting Requirements
April 2, 2022

Requirements for all reports

- Every column should be sortable
- Every column should filterable
- Every report should be capable of Geo and sub-region sorting (and some users can only
see their Geo or sub-region data)
- If a screen capture or export is taken, then it should be watermarked with their login

Data Validation Report requirements

Use case: As a receiver of information of Microsoft (MSFT) data. I need to find, as quickly as
possible, any potential errors. The faster we can start looking at potential problems, the more
likely we’ll be able to correct it and book revenue within the month it was intended. It’s
envisioned that the report produced, would be reviewed by personnel to investigate unusual
issues and resolve any data errors – prior to the VCP automation. (This report has been
specified, below, to include errors we have seen in the past. This report may evolve over time
to include new errors, once seen.)

Header of the report:

1. Report title “AVS Data Validation Report”
2. Date of the report
3. “Name of the file processed:” <filename>
4. the number of records reviewed of each type (OnDemand, RI, Adj, etc.) for assurance
that the whole report was reviewed (e.g. 246 rows reviewed)
5. number of errors found (e.g. 6 rows failed)
6. passing percentage of the total rows (e.g. 97.6% pass rate)
7. % of stability (e.g. 82.6% Stability rate) (see below)
8. the body of the report is a list of all the failures as described in the table below
(Store the results of this header info so it can be used month over month to measure
improvements in Microsoft data integrity.)
9. Trending of the 5 measures above compared to previous months

% of stability measurement (OnDemand Only)

Stability is measured as a set of nodes with no changes. This shows in the data as a row
with an even number of nodes used that month. To measure, determine what % of the
hours per row are evenly divisible by the number of hours in the consumption month.
(e.g. for a Feb report about Jan data, Jan has 31 days x 24 hours = 744 hours. 744 is the
denominator. 744 hrs, 2232 hrs, and 20088 hrs are all evenly divisible with no digits
after the decimal point. By contrast 8866/744=~11.92. This would not be even, so it
shows there was some change. The row was not stable all month. Keep a percentage of
the evenly divisible vs all the rows. If the % is very low, it can be an early indicator that
VMware Company Confidential 5
AVS Automation and Reporting Requirements
April 2, 2022

not all the hours of usage were reported in the data. [Store this percentage or calculate
again later for a trending report.]

Body of the report. Here are the criteria for passing/failing records. If the record has a failure,
print the whole row of data in the report and the reason why it’s being listed (shown in the
“Failure Message” column).

Requirement Failure Message

A All Cells from all spreadsheets must be completed. None Data cells not complete or in
can be blank unless a comment column. All UUIDs must need of investigation
conform to the UUID format (google search for format).
All instances of “N/A” should be investiaged.
B No numbers should ever be negative EXCEPT for a Negative dollar amount
Royalty $ amount, and this is only acceptable if it’s in an
Adjustment section.
C A Royalty $ amount can only be $0 if it is a Trial. $0 charge for production
services
D The start date of an RI must always be 1 year or 3 years Questionable RI end date
earlier than the end date. No End date can ever be prior
to a start date. The 1 yr and 3 yr duration should
correlate precisely to the Term length (1 or 3). Any other
time lengths for RIs, or mismatches to the Term, should
be reported.
E All RIs with Microsoft ID numbers new on THIS month’s Booking alert! UUID and RI
report, must have an RI start date that is in the month start mismatch. Investigate.
being reported (i.e. if the report for March 2022 is
reporting on Feb 2022, then the start dates for these
first-ever-reported UUIDs must be between Feb 1 and
Feb 28, 2022.) Otherwise booking may not have
occurred when it should have. Report these for
investigation.

F All RI Cancelations have the potential for many errors Cancelations and RI History
today in the Microsoft systems. For each Cancelation, list
entries in the data with the same start date and time
stamp. Initially this can be done with the current file
being input. As soon as possible, please provide all the
rows from previous months too with this start date and
time stamp, ordered by usage month and UUID.
G HCX Customer ID numbers must have a corresponding HCX Customer ID is not found
matching UUIDs in either the Paygo (a.k.a. OnDemand) in OD or RI data
or RI tabs. The HCX product is an add-on to nodes
running as Paygo or RI nodes.
VMware Company Confidential 6
AVS Automation and Reporting Requirements
April 2, 2022

H Consistency of Customer ID and Area assignments. Identification Error! UUID,

 Each UUID (a.k.a. Subscription Number; a.k.a. Customer ID, and Area are
Microsoft ID number; etc.) should be dedicated not consistent with past or
to only one customer (Customer ID) and one HQ present data.
Country for that customer (“Area”).
 Each month the UUID appears, it should always
have the same Customer ID and Area assigned to
it.
 This is true for all tabs each month.
 In the new month, compare each UUID to the
others in this month’s data. There are often more
than one instance of a UUID appearing in the
may Excel tabs of the data. If found, the
Customer IDs and Areas should be identical. If
not, it’s an error.
 Then compare each UUID to those in the
database. When found, ensure that the same
Customer ID and Area were given to that UUID in
the past. If there are any discrepancies, list those
rows in the error report.
List the current row and the mismatches found (and
their locations)
I Minimum number of nodes met. From this month’s data, Usage is under 3 nodes. Max
ensure that at least 3 nodes were used. Total nodes per OD nodes = <x> and RI nodes
Customer ID = (Max OD daily columns/24) + RI nodes. = <y>
This should be equal to or greater than 3 because the
product has a three node minimum for functionality.
Anything less may be a data reporting error. EXCEPT
where the Customer ID appeared in the Trial data during
the same month. The transition from Trial for production
will often trigger customer action and very few hours
before spin down.
J Compare Trial UUIDs to historical data. If any customer Customer has been on Trial
remains in trial three months in a row, report it as an since <usage mm/yy>
error on the fourth month. (e.g. Customer ID 3452643
appears in the Jan, Feb, and March data, and then also
appears in April. Record this as an error in the April Rpt.

VMware Company Confidential 7

AVS Automation and Reporting Requirements
April 2, 2022

Data Manipulation requirement

Use case: As a Business Intelligence specialist, I need to be able to mine the data for insights.
▪ All data must be accessible to me for integrated reporting tools (Mode, Tableau, and
Microsoft Power BI).
▪ I must be able to export all the data for import into another tool, when needed.
▪ I must be able to correct data that has errors in the data. Because errors are sometimes
not found for several months, Microsoft does not always provide new updated data files
(at least not currently). So for the time being, I need to be able to update or refresh in
some way the data that is used for all reporting.

Automation Report Requirements

6 Reports corresponding to VCP Automation:

These reports are in existence with manual reporting and are well-known. Thus, not
documented here.
v1 RI Consumption Report [For contracts 13398274 (US) and 24942417 (RoW)]
v1 OnDemand Consumption Report [For contracts 13229759 (US) and 24855606 (RoW)]
v2 RI Consumption Report [For contracts…]
v2 OnDemand Consumption Report [For contracts…]
v1 New RI Bookings Report [For contracts…]
▪ All RI UUIDs for which this month is the first appearance of these nodes as evidence by
the historical data in the database and by the FCStartDate.
▪ V1 will typically not have any new RI nodes added, and it is normal for this report to
capture no data meeting this criteria.
v2 New RI Bookings Report [For contracts…]
▪ All RI UUIDs for which this month is the first appearance of these nodes as evidence by
the historical data in the database and by the FCStartDate.
▪ Prior to processing the new RIs, a thorough review needs to be done to validate the data
and then there needs to be a way to add or subtract what’s included in the VCP booking.
We should discuss.
▪ The New RI Bookings Report, below, and the Data Validation Report, above, will be
instrumental in reviewing this automation and checking it’s correct.

Dashboard Requirements
Highest Priority User report requirements

VMware Company Confidential 8

AVS Automation and Reporting Requirements
April 2, 2022

New RI Bookings Report (v1 and v2)

As a Sales Ops Lead or AVS HQ BI Lead, I need validate the data related to the new
Reserved Instances. Each month I need to know which subscriptions for RIs have shown
up for the first time (the first time the RI’s UUID has appeared). This is what should be
booked. I need to know all of the following fields, plus Geo and Sub-region when
available.
Microsoft ID Customer ID Customer Name Type US/NonUS Area DC Country FcDateTime
Cores Nodes perTerm
RI Min of Reporting Month Max of Reporting Month

For other users who cannot see the Customer Name, this report should be available
with the customer name omitted, and filtered for only their Geo or sub-region
(according to Personas, above).

Customer Counts Report

▪ This report may be the most sought after data from the farthest reaches. It’s used to
judge the growth rate of the business. It’s reported at QBRs and monthly reviews. It’s
discussed between Microsoft and VMware. This report will be used by all users.
▪ There is a fair amount of detail that will need to be written about this. For now, let it be
noted that the total unique customers (the yellow stripe below) are unique across all
the AVS products, v1 and v2, OnDemand, RI, and HCX.
▪ All the data in the following chart includes both v1 a v2 combined.
▪ It would also be helpful to add a graph to visually show the changes in population.
▪ Need a count of customer to provide this:

Hyperscaler Dashboard Infoshare

▪ As Marketing, we provide this current dashboard
https://app.mode.com/vmware_inc/reports/fa8a25353407 to interested viewers.
▪ Propose: Initially provide the marketing team with the info they need to update
manually created tables behind these diagrams.
o Then later down the road we evaluate the value of converting the marketing
dashboard to be serviced by this new database. It may or may not be worth the
effort.

RI High-level Node Billing Trend Report

As a Geo or WW Sales Leader, I need to see how customers are growing across all their usage
deployed globally. Create a report that shows:

VMware Company Confidential 9

AVS Automation and Reporting Requirements
April 2, 2022

▪ By Geo
o By Sub-region
▪ By HQ country
• One row per customer in their HQ Country
• Columns:
o Customer ID
o Customer Name (inserted only when the user is allowed to
have that access)
o Ri Expiration date
o A column per month (user can select the number of
months)

This example shows a similar report from a pivot table. But instead of sectioning off by Country,
just make columns for Geo, Sub-region, and HQ Country. Then the 4th column is the Customer
ID as shown here. The 5th column is the customer name (only if the user has access to that info.)
The 6th column should be the expiration date of the RI. Then the usage months continue from
there. There should be no totals rows nor columns.

OnDemand High-Level Hours Consumption Trend Report

Same as the RI High-Level Node Trend Report with the following changes:
▪ The number in each month’s cell will the total number of OnDemand hours per
customer

RI’s by Expiration
GEO leads need to be able to run a report of RI’s upcoming for expiration. From the RI
Consumption data that was received this past month, produce this report:
▪ If the RI is expiring this month or the following 2 months, List:
o Geo
o Sub-region
o HQ Country
o Customer ID
o Expiration date
VMware Company Confidential 10
AVS Automation and Reporting Requirements
April 2, 2022

o Customer Name (if access is allowed)

o Term of the RI (1yr or 3yr)
o # nodes for the RI
o DC location
▪ Sort by Geo, Sub-region, HQ Country, Customer ID

FY23 Net New Nodes report (New Dashboard item)

As a member of the AVS team, I want to understand how we’re doing towards our collective goal of
adding 772 new nodes in FY23. These new nodes are nodes that are generating bookings. Bookings
consist of all OD Consumption plus all RI Bookings approved to go through VCP calculations.

Limitations: Below is how the business intends to reach 772 additional nodes, but as this writing (April
2022), we have no way to distinguish which VMware organization made the sale. We can, however,
track by Customer HQ Geo, so that’s what we’ll do.

Counting Rules:
▪ All new RIs nodes count once they have been approved for booking
▪ Increases in OD nodes count (except where the increase is due to an RI expiration)
▪ Migrations from v1 to v2 do not count

Building the report

1. You will be provided with the total number of nodes in use per Geo. This will be used as the
comparison for each month of FY23
A. FYI – this is how the base number of nodes will be determined:
• Count the number of v1 & v2 RIs being consumed in Jan RI Consumption
(Dec usage)
• Add to that, the total of each customers’ average OD node consumption for
the Jan booking (Dec usage)
• Record this sum, separating by Geo of the HQ Country. These will be the
base numbers to which all we will compare each month of FY23

B. Determine the new month’s “Net New Nodes” compared to the Baseline (without
counting RIs that revert to OD), by:
i. Each month (BY GEO)
a. Count the number of RIs being newly BOOKED in FY23. These
have a start date of January 1, 2022 or newer
b. Add the total of this month’s active customers’ node count
▪ Take the total number of hours billed by MSFT that month
and divide by the number of hours in the month (e.g. a 31
day month ha 744 hours because 31x24=744.
c. Add together a & b from both v1 and v2.
d. Compare to the base. Each node more than the base is 1 Net
New Node.
e. In a table put the Net New Nodes each month
b. Create a (geo) stacked bar chart for the 12 months of FY23. Place AMER on the
bottom, EMEA in the middle, and APAC on the top of the stacked bars.

VMware Company Confidential 11

AVS Automation and Reporting Requirements
April 2, 2022

c. Place a chart at the top, then table of data below the chart, and this description
of how this was measured below the table. Place

Customer Review Report

As an AVS Specialist, I need to understand what is happening with my customers. If usage is
falling off, I want to call and see if there’s anything wrong and how I can help them. I want to
confirm he implementation of plans and see when the usage increases. As such, it would be
helpful to have a customer-centric report, with all their usage in one place.
▪ At the top of the report, put the Customer ID. (One day we may add the customer name
to this chart, but only once all security is in place and tested).
▪ List information in the following sections
a. Trial usage info
b. OnDemand usage info
c. RI booking info
d. RI Cancelations and adjustments
e. HCX info
We can specify how this looks when ready

Phase 2 Reporting & Dashboard Requirements

Churn rate
- Churn stats and trending; showing new customers and departing customers each
month, along with a churn rate %.
- List, for those who can see them, the customer names of departing and returning
customers

Top OD Customers based on Royalty paid

▪ Create by Geo and WW views
▪ Allow user to modify the date range included in the report
▪ Limit to top X customers (configurable – currently default to 10)
▪ This report for OnDemand only, at present
▪ Columns: Geo, Sub-region, HQ Country, Customer ID, Customer name (only if allowed
and watermarked), Royalty paid, % of revenue (simple calc of total $ Royalty for this
customer, divided by the total amount of $ Royalty for the month)

Top RI Booking Customers

▪ Same, but convert monthly consumption to ACV
▪ Map across a date range for FCDateTime

VMware Company Confidential 12

AVS Automation and Reporting Requirements
April 2, 2022

Partner Reporting
▪ Requirements TBD

More TBD

Summary
The development of the AVS automation, dashboard, and reports will be iterative. It’s the
outcome of a partnership between teams. If you have questions, please let me know. I am
eager to answer them. We are #OneVMware!

VMware Company Confidential 13