Essbase Calculation Optimization Techniques

By Richard Hoeschele Principle Solutions Architect AdvancedEPM

AdvancedEPM at a Glance
An Enterprise Performance Management (EPM) firm laser focused solely on Oracle / Hyperion solutions. Oracle Platinum Partner Recognized by Oracle for our in-depth expertise and excellence across the entire suite of Oracle Hyperion EPM solutions. Serve leading global organizations (including 11 of the top 100 companies and 28 of the top 500 according to the latest Fortune rankings). Expert consultants that have been working in the Hyperion space for more than a decade, across all products and platforms.

Deep Hyperion Product Knowledge

Essbase/Planning
WorkForce Planning CapEx Planning FX Impact Analysis Complex Allocations Statistical predictive modeling functions leveraging Java API Intercompany Elimination logic on one or more dimensions Web Form customization using JavaScript

Hyperion Financial Management (HFM)

Financial Consolidation SEC Reporting Sarbanes-Oxley Compliance International Financial Reporting Standards (IFRS)

Financial Data Quality Management (FDM)

Drill-through Reporting Complex Mapping and integration with DRM Complex VB Scripting for RDBMS to FDM automation Extraction of data directly via FDMs backend relational tables

Data Relationship Management (DRM)

Emphasis on Information Governance Mappings integrated in the hierarchies ERP Master Data Management Consolidation, Planning, Analytical / Reporting metadata management Data Warehouse Dimension Management

Automation of entire EPM suite of products. AdvancedEPM knows how to leverage tools and technologies like Perl, Java, VBScript, JavaScript, MaxL, etc. to truly make the life of an administrator much easier

EPM Practice Highlights

We have the marquee list of EPM clients, including clients such as GE Healthcare, SimplexGrinnell, AT&T, Allstate, Del Monte, Hertz, Wal-Mart, Motorola, and many more. We have recruited and developed the top EPM talent in the marketplace Oracle Hyperion certified Seasoned professionals We have developed creative cost effective implementation concepts that will ensure project success in a cost effective manner with reduced project timelines Centers of Excellence Rapid design / build toolkits Offshore Our experience and focus is exclusively on the EPM market leader: Oracle Hyperion

We are proud to serve market leaders

Auto Financial Comms Complex Consumer Energy /Retail Mfg Services & Media High Tech Insurance Life Public & Health Sciences Sector Travel & Trans

Topics Covered
Fixes Nested fixes, Conditional fixes, Exceptions Minimizing Xrefs calls Macros in Business Rules Dense Fixes / Calculation Blocks Launching Business Rules remotely Handling Duplicate Ranks in loops Loop Counters and breaking out of loops Creating Blocks Data export using Calculation scripts MaxL and Calculation scripts Allocations considerations in Essbase. Dynamic Currency Conversions Ranking across two different measures

Nested Fixes
Nested fixes are used for a number of tasks. Fixes are used to limit the portion of the database on which calculations are performed. They can also be useful in optimizing calculations. These are the tasks I will be covering.
Fixes used to limit passes through the database Fixes used to handle exceptions in the calculation process Fixes used to control conditional execution of code segments

Fixes to limit passes through the database

Here we need to calculate each Entity based on its associated currency UDA. If we used a single fix, we would loop through the entire entity structure once for each currency. Here we loop through the database only once. fix([varGETSPlan_Years],[varGETSPlan_Scenario], [varSourceVersion]) /* Fix on each set of legal entities based on its currency UDA */ Fix (@UDA("Legal Entity", C_AUD)) fix( "Region","Org","Cost Center","Currency","Legal Entity" ) Jan( MC_Cur_jan = @XREF("FX_Conv", "MOR", "C_AUD") ; VC_Cur_jan = @XREF("FX_Conv", "VAL", "C_AUD"); )

Nested fixes and exceptions

Another use of nested fixes is to temporarily expand the scope of the fix. Nested fixes should be read as an OR. By this I mean that when you use the nested fix, the inner fix is ORed with the outer fix. All members of the inner and outer fix are processed. This is useful where all of the data to be processed shares most dimensions and one section only needs to process one or a few additional members in a dimension. In my example the exception is to process all data for all months and some data for beginning balance as well. Rather than two passes through the database, we only loop through once and process the exception in its own nested fix.

Nested fixes and exceptions example

FIX( "Functional Currency", "USD", [varGETSPlan_Years] ,@idesc("800",0), @RELATIVE ("FX_TOT", 0 ), @RELATIVE ("P00010", 0 )) Fix(@RELATIVE ("BT0010", 0 ), BegBalance) datacopy Actual->Final to F2->Working; Endfix Fix(Jan:Dec) datacopy IOP->Final to F2->Working; Endfix Endfix

Fix statement as a conditional

By using a combination of Fix, Set, Substitution variables, and Calculation functions, you can make a fix statement that is in fact a conditional statement. This is useful in calculation scripts that need to be time sensitive. Forecasting applications are a good example. We will use the following structures to build this conditional fix statement. EMPTYMEMBERSETS: This set statement tells Essbase to ignore any fix statements that have no members. @REMOVE: This Calculation command allows you to remove members from a list of members. FIX: This statement contains the members on which we wish to fix. SUBSTITUTION VARIABLE: A global variable used to store a member name.

Conditional Fix statement example

If the Last_ACT_Month is set to BegBalance, the code in the fix will be run, since removing Beg_Balance from Jan has no effect. If the Last_ACT_Month is set to Feb, the remove statement will include BegBlance, Jan, Feb. Since Jan is in this list, it is removed from the fix. Now the fix is empty and its code is skipped. SET EMPTYMEMBERSETS ON; Fix (@Remove(Jan , Begbalance: &Last_ACT_Month)) %GETSPlan_CTA_all(); endfix

Minimizing Xrefs calls

Xref is an excellent way of pulling small amounts of data from one Essbase database into another. While very useful, each call consumes a significant amount of time. If you plan to make repetitive calls to the same intersection, it is beneficial to limit the number of calls. This example will be used again when discussing Macros and Macro usage. The example I have chosen deals with currency conversions. I have one set of currency conversion data for each month for each currency. These are stored in a separate Essbase database. I need to query this database for each entity and month and convert the data. I chose to get the rates once for each currency then process all entities having that currency UDA. Since the source database has fewer dimensions, I chose to use top level members in the non relevant dimensions. This limits the xrefs to one call per month, year, scenario, version, and currency.

Minimizing Xrefs calls Example

Var MC_Cur_jan, VC_Cur_jan; fix([varGETSPlan_Years],[varGETSPlan_Scenario], [varSourceVersion]) fix( "Region","Org","Cost Center","Currency","Legal Entity" ) Jan( MC_Cur_jan = VC_Cur_jan = . Dec(MC_Cur_Dec = VC_Cur_Dec = endfix @XREF("FX_Conv", "MOR", "C_AUD") ; @XREF("FX_Conv", "VAL", "C_AUD");) @XREF("FX_Conv", "MOR", "C_AUD") ; @XREF("FX_Conv", "VAL", "C_AUD");)

Macros in business rules

The successful use of Macros requires thought and planning to be effective. Macros are not available in calculation scripts. Questions you need to answer are: Can the candidate process stand alone? It needs to validate as a calculation script. It can be embedded in fixes and if statements later. Will the code in the Macro work without modification in all the business rules that call it? Will I be able to use this macro in at least three or more places? Less than this limits the Macros usefulness. Can I use variables and fixes variables to parameterize the Macro? Can I build a Business rule to call the Macro code, including setting variables and using fixes?

Macros and Variables

The final step in the currency conversion business rule, detailed in the Xref topic, is to call the macro that does the currency conversion. We use a fix to limit the conversion to the entities having the correct UDA. We use the variables to which we previously loaded the currency conversion values. Since the variable names are not currency specific, the macro can run against all currencies. We can call the same macro for each currency using the same set of variables set to different conversion values. Dec( MC_Cur_Dec = @XREF("FX_Conv", "MOR", "C_AUD") ; VC_Cur_Dec = @XREF("FX_Conv", "VAL", "C_AUD");) Endfix Fix (@UDA("Legal Entity", C_AUD)) /* once the rates have been set call the macro to run the conversions. */ %GTSPlan_FX_Convl_2(); endfix

Dense Fixes/ Calculation Blocks

When I first started working with Essbase, Calculation Blocks were known as dense fixes. In some ways this is a better description of their function. Calculation blocks should be based on a dense member whenever possible. Otherwise many blocks may be called, dramatically slowing calculations. Using a sparse member seems to act more like a nested fix and calls all blocks with that sparse member, resulting in much slower execution. Actual validation notes: Changed the calc block from Local to FTE_W. This seemed to make a significant improvement in performance: 1841.3 seconds with local: 882.29 with FTE_W as the calc block member. A Calculation block member does not necessarily limit the calculation to a single member. You may assign other members in the block. Calculation blocks are required when using calculation script variables. You may select any dense member from the appropriate dimension to set your calc block. It does not need to be one you are actually calculating.

Calculation Block Example

Examples of setting members and using variables in a Calculation block. Var MC_Cur_jan, VC_Cur_jan; Jan( MC_Cur_jan = @XREF("FX_Conv", "MOR", "C_AUD") ; VC_Cur_jan = @XREF("FX_Conv", "VAL", "C_AUD"); ) A_59761( A_59761 = MC_Cur_jan * A_11234; A_59873 = MC_Cur_jan * A_11834; )

Launching Business rules remotely

During Automation of planning applications, it is often desirable to automate execution of certain Business rules. Planning provides a launcher batch file, but it resides in the Hyperion\Products\Essbase\eas\console\bin directory. (CmdLnLauncher.bat) Most IT groups would rather not put script files in a bin directory. By replacing the inferred references with hard coded directories, we can move the batch file to any other directory in the Hyperion directory tree. You will also need to Automate Launch Variables by right clicking the rule to be automated. This generates the XML file used to launch the business rule. It contains parameters that may be modified. This is a sample call to launch business rule. CmdLnLauncher1.bat -SHYP_Server" -U"Hyp_User" PPassword" r"rollup_deip" -f"roll_deip.xml"

Modified Business Rule Launcher

set CPATH=D:\Hyperion\AdminServices\console\lib\CmdLnLauncher.jar Set CPATH=%CPATH%;D:\Hyperion\AdminServices\console\lib\ easclientplugin.jar set CPATH=%CPATH%;D:\Hyperion\AdminServices\console\lib\log4j-1.2.8.jar Set CPATH=%CPATH%;D:\Hyperion\AdminServices\console\lib\ framework_common.jar set CPATH=%CPATH%;D:\Hyperion\AdminServices\console\lib\eas_common.jar set CPATH=%CPATH%;D:\Hyperion\AdminServices\console\lib\jaxp-api.jar set CPATH=%CPATH%;D:\Hyperion\AdminServices\console\lib\jdom.jar set CPATH=%CPATH%;D:\Hyperion\AdminServices\console\lib\dom.jar set CPATH=%CPATH%;D:\Hyperion\AdminServices\console\lib\xercesImpl.jar D:\Hyperion\common\JDK\Sun\1.4.2\jre\bin\java DHBR_HOME=D:\Hyperion\AdminServices\ -cp %CPATH% com.hyperion.hbr.cmdlnlauncher.CmdLineLauncher %1 %2 %3 %4 %5 %6

Handling duplicate ranks in loops

I have had to loop through groups of members in order to perform calculations that had to be done sequentially based on rank. The case is that Essbase can create duplicate ranks. This makes a conditional like Mbr_rank = last_rank +1 fail when it hits a duplicate rank. We solved this case using a member in the outline that records the members calculation status. The "Calc_Completion member stores a flag indicating whether the member has been calculated. This allows us to use a <= condition that will work with duplicate ranks. The following code also demonstrates how to break out of a loop when you finish processing your code.

Rank Sorting example

This is an abbreviated section of the code used. There may be additional assignments and a fix as well. Var breakpt = #Missing LOOP (30,breakpt) "Title Slate Rank"( IF( "Title Slate Rank" <= RankCount and "Calc_Completion" <> 1) "Running LF" = @ANCESTVAL("Studio",-2,"Running LF") "Calc_Completion" = 1; RankCount = RankCount +1; if(@ANCESTVAL("Studio",-2,"Title Slate Rank") < RankCount) breakpt = 1; ENDIF; ENDLOOP

Loop Counters and breaking out of loops

Loop counters in calculation scripts must be set to an integer. Because of this the developer needs to set the loop limit to a value greater or equal to the highest count that can ever occur. This is often much higher than needed. Essbase provides a tool to handle this issue. It is the breakpoint variable. When creating a loop, if you add a variable to the loop statement, as in the previous slide, you can break out of the loop as soon as you have met your requirement. You do this by setting the variable to 1. In the example if the ,"Title Slate Rank" < RankCount we set the breakpt = 1. Once our RankCount (actually a member processed count) exceeds the highest rank stored, we break because we have calculated all the required members.

Loop example
This is an abbreviated section of the code used. There may be additional assignments and a fix as well. Var breakpt = #Missing LOOP (30,breakpt) "Title Slate Rank"( IF( "Title Slate Rank" <= RankCount and "Calc_Completion" <> 1) "Running LF" = @ANCESTVAL("Studio",-2,"Running LF") "Calc_Completion" = 1; RankCount = RankCount +1; if(@ANCESTVAL("Studio",-2,"Title Slate Rank") < RankCount) breakpt = 1; ENDIF; ENDLOOP

Creating Blocks
Essbase is very picky in creating blocks. Often you will write assignments that wont seem to work. This is generally a block creation issue. There are a number of ways to create blocks. DataCopy: This method is best used when you really need to copy data from a sparse to a sparse dimension example Version to Version. If all you need do is create blocks, consider writing a 0 to a single dense member intersection. CREATEBLOCKONEQ: Creates blocks if a sparse member has its value set by something other than a constant. Default is off. I would set this to on either in the database or most scripts. CREATENONMISSINGBLK This command is much more sensitive. It processes all potential blocks and creates any where the formula returns a nonmissing value. It is very slow! Use only within Fix commands on small portions of the database. Use SET CREATENONMISSINGBLK off to stop processing blocks when not needed.

Creating Blocks examples

SET CREATENONMISSINGBLK ON; FIX ("HSP_InputValue,"Alloc_Source", "No Department", @RELATIVE("YearTotal",0), "Current Working","Final", "Budget", "Actual", &PriorActYear, &BudYear, @RELATIVE("Income",0), Local ) " FTE_W"( FTE_W = @XREF("_PnlCube_","Apollo Degree Programs"); ) ENDFIX SET CREATENONMISSINGBLK Off; CREATEBLOCKONEQ: Working = Actual * 1.05;

Data export using Calculation scripts

In the most recent releases of Essbase a new export functionality has been added. It allows you to export data from BSO databases, from within a calculation script. There are two associated commands that you need to use. DATAEXPORTOPTIONS This statement allows you to control how the data will be exported. DATAEXPORT This statement actually does the export and defines information about the file based on the options you selected. The export command is best used within Fix statements. The export command executes MUCH faster than Essbase reports. It offers much more flexibility then the existing Export option.

Data export using Calculation scripts 2

Additional options commands are DATAEXPORTENABLEBATCHINSERT allows batch insert into tables if the ODBC driver supports it. DEXPSQLROWSIZE allows you to control how many rows are inserted at a time. This command does not affect batch size. EXPORTFILESIZELIMIT sets a limit to the export files size. Default 2 GB. It automatically creates additional files as needed. DATAEXPORTCOND allows you to further filter the data, based on data values rather than members.

DATAEXPORTOPTIONS Command
DataExportLevel - All, Level0, Input. DataExportDynamicCalc Do you want dynamically calculated members exported? This will slow exports if on since the values need to be calculated. DataExportNonExistingBlocks Exports all potential blocks, used with export dynamic calcs to get all data. DataExportColFormat If on it exports in column format. I would recommend this, especially for backups and exports to other systems. DataExportColHeader For text outputs, it sets the dense dimension to be used as column headers. Pick a small dense dimension or the number of columns may be excessive. DataExportRelationalFile - Formats the file for a relational export. DataExportOverwriteFile Only for text files. It will overwrite existing files if on.

DATAEXPORTOPTIONS Example
SET DATAEXPORTOPTIONS { DataExportLevel LEVEL0 ; DataExportDynamicCalc OFF ; DataExportNonExistingBlocks OFF ; DataExportPrecision 9 ; DataExportColFormat ON ; DataExportColHeader "Period" ; DataExportRelationalFile OFF ; DataExportOverwriteFile ON ; };

DATAEXPORT command
File - exports a text file. You specify file/path, Delimiter, and MissingChar. Essbase interprets the path in context to the server. Export files cannot be written to a client. I didnt find a way to have tab as a delimiter. MissingChar defaults to #MI. You can use NONE to get an empty field. Relational output. dsnName, table name, user, and password. The table must exactly match the output columns. The table is not cleared by this command. Substitution variables can be used. BIN output. Requires only file name. It is used for static backups Both the outline and file must have the same date time stamp .

DATAEXPORT Examples
Relational Export FIX("Final", &GETSPlan_FcstYear, @LEVMBRS("Legal Entity", 0), &GETSPlan_FcstScn, "Jan":"Dec) DATAEXPORT "DSN" "Oracle Wire Protocol" "GET_FR_MOD_GETSPLAN_FINAL_T" User" Password" ; ENDFIX File Export FIX("Final", &GETSPlan_FcstYear, @LEVMBRS("Legal Entity", 0), &GETSPlan_FcstScn, "Jan":"Dec) DATAEXPORT FILE" ," E:\hyperion\Essbase\app\test1\test1\PlanFin.txt #MI ; ENDFIX

MaxL and Calculation scripts.

MaxL is the primary tool to execute Essbase commands from a batch file. When executing multiple MaxL scripts from a batch file, make sure you use Call to ensure the scripts run in sequence. In the Tech Ref under MaxL Invocation there are instructions to encrypt passwords in MaxL scripts. The following MaxL Shell invocation generates a public-private key pair that you can use to encrypt a MaxL script. essmsh -gk The following MaxL Shell invocation encrypts the input MaxL script, obscuring user name and password, and changing the file extension to .mxls. essmsh -E scriptname.mxl PUBLIC-KEY The following MaxL Shell invocation decrypts and executes the MaxL script. essmsh -D scriptname.mxls PRIVATE-KEY MaxL scripts can have any number of parameters passed into them as arguments.

MaxL and Batch Files.

A typical Batch file calling MaxL scripts and EIS cbs scripts. There could easily be calls to the Business rule CmdLNLauncher as well. Call Call Call Call Call Call Call E:\bin\essmsh.exe "E:\Hyperion\batch\OrderHouseKp1.txt E:\bin\essmsh.exe "E:\Hyperion\batch\OrderSNAPclrHist.txt E:\bin\olapicmd -f"E:\Hyperion\batch\Ord_Mbr_Build.cbs E:\bin\essmsh.exe "E:\Hyperion\batch\OrderSNAPclrSnap.txt E:\bin\olapicmd -f"E:\Hyperion\batch\Ord_Data_load.cbs E:\\bin\essmsh.exe "E:\Hyperion\batch\LoadAffiliates.txt E:\bin\essmsh.exe "E:\Hyperion\batch\OrderHouseKp2.txt"

Time Stamps in output files

These windows commands, when used as variables in a batch file, allow you to include a time/date stamp in your file name. This only works in Windows Batch files and MaxL files. set datestamp=%date:/=_% set datestamp=%datestamp: =_% Set Timestamp=%time:.=_% set timestamp=%timestamp::=_% export database NVL_Plan.NVL_Main using server report_file 'ext_plan' to data_file "'D:\\Hyperion\\Datafiles\\dailyplan$datestamp$timestamp.txt'";

Allocations considerations in Essbase

The keys to allocations in Essbase are fairly simple. The Allocation function and MDAllocate are the best ways to run allocations. But there are some pitfalls to allocations. Never allocate a dynamically calculated value to its children. The value allocated will change as you allocate each member. Where possible, allocate to a different version, account, or organization. This also allows you to validate the allocation.

Allocations considerations in Essbase 2

Use variables to store the summed value that you are allocating if it is stored in a dynamically calculated member. This prevents recalculating the sum for each target member. Try to use another measure/measures as the allocation percentage. Again watch for dynamic members as the denominator. Sales/Total_Sales. Use multiple passes across dimensions, if you start having problems with cache sizes or CALCLOCKBLOCK limits.

Dynamic Currency Conversions

We can create a member that dynamically calculates local currency for parent members. It assumes that you have already decided to do currency conversions in Essbase and have currency conversion values loaded at some location in Essbase. Entities at all levels where you wish to have conversions must have a UDA identifying their local currency. You must have a fixed reporting currency. All level 0 members must have their currency values in at least the reporting currency. The currency conversion values from Local to reporting currency must be stored in the database. We can use reciprocal values to convert back to local. Create a dynamically calculated member in the currency dimension. This member will use the UDA associated with the entity to change reporting currency to Local currency. Sample member name Entity local currency.

Sample currency Member formula

This code assumes that the currency conversion percentages are stored in a single location for each Month, Year, Scenario, and Version if applicable. Having a member/substitution variable whose value is the cross dim of members used in the path to the conversion values can simplify this process. Curr_path = No_Company->No_Product->any other dimensions if (@isUDA(Entity",INR)) Entity_Currency= USD*1/USD_INR->Curr_Path; endif; (Elseif can be used) if (@isUDA(Entity,EUR)) Entity_Currency= USD*1/EUR_INR->Curr_Path; endif;

Ranking across two different measures

The key to this is knowing the maximum rank of the second ranking. I knew the rank could never exceed 99. Multiplying the first rank by 100 and adding the second rank gives me an integer that will always be in the correct order. Duplicate ranks, in the combined rank, are still possible but unlikely.

Ranking example
Fix(FY08) "Title DFR Rank"( if ("DFR Actual" > 0) "Title DFR Rank" = @RANK(SKIPBOTH,"DFR Actual", @RANGE( "DFR Actual",@RELATIVE([Entity_Prompt],0))); "Avail_Rank" = @RANK(SKIPBOTH,Avail date", @RANGE( Avail date",@RELATIVE([Entity_Prompt],0))); "Combined_Rank"= -1*((100*"Title DFR Rank")+"Avail_Rank"); endif; ) endfix;

Contacts
Eric Cuka
Business Development Director
8803 Brecksville Rd., Suite 7-198 / Brecksville, Ohio 44141 Cell: (952) 239-5324/ eFax: (440) 201-7486 ecuka@advancedepm.com www.advancedepm.com

Todd Rebner
Managing Partner
8803 Brecksville Rd., Suite 7-198 / Brecksville, Ohio 44141 Cell: (330) 212-3097 / eFax: (440) 201-7486 trebner@advancedepm.com www.advancedepm.com

Ash Jain
Managing Partner
2538 Acadiana Lane / Seabrook, Texas 77586 Cell: (832) 428-3339 / eFax: (440) 201-7486 ajain@advancedepm.com www.advancedepm.com