Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Standard view
Full view
of .
Look up keyword
Like this
0 of .
Results for:
No results containing your search query
P. 1
Choosing BI Appliance - InfoMgmt-2

Choosing BI Appliance - InfoMgmt-2

Ratings: (0)|Views: 10|Likes:
BI appliances come in many variations, so it takes some amount of groundwork and research to make sure that you choose the right appliance that meets your needs.
BI appliances come in many variations, so it takes some amount of groundwork and research to make sure that you choose the right appliance that meets your needs.

More info:

Published by: Saama Technologies Inc. on Dec 28, 2011
Copyright:Attribution Non-commercial


Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less





Saama White Paper
January 2010
Choosing a BusinessIntelligence Appliance
As published in: 
The term
business intelligence appliance 
is defined differently by manyfolks, however, a broadly accepted definition is that it is server hardwareand database software bundled to specifically meet data warehousingneeds.Just like the name suggests, BI appliances are turning out to have somesimilarities with other familiar appliances, i.e., their kitchen counterparts.They come in many variations, so it takes some amount of groundworkand research to make sure that you choose the right appliance thatmeets your needs.I was involved in two such bake-offs over the last year, and through theexercise, we worked out a pretty good process that helped us assessour options and make our choice. I will note some of the key highlightsin our approach.First, the client was a large financial institution, and this initiative was ata departmental level. Our key objective was to consolidate four datamarts into a single infrastructure because we were getting manyrequests to combine information across these and eliminate datalatency between them. We were primarily facing challenges on the dataload side, which in some cases was greater than 30 hours for a run.Performance pain on the query side but wasn’t very significant. Also asa secondary benefit, we were attempting to consolidate databaselicenses and servers across these four environments, includingdevelopment and test boxes for each environment. Like many of thelarge enterprises, we had the server support outsourced to one of thelarge infrastructure support players and were being charged a hefty sumper server on a monthly basis.Because the combined total data size was expected to be around 2TB,we were reluctant to even begin the process as we had heard about the5TB plus starting point for the appliance solutions to prove valuable.
“BI appliancescome in manyvariations, so it takes someamount of groundwork and research to makesure that youchoose the right appliance that meets your needs.”
Page 2
Saama Article
April 2010
However, we really needed something to help the load situation anddecided to move ahead with this initiative.We started with the usual “product evaluation” approach and broke itout into four key steps:1. Long list based on Internet research.2. Short list – based on discussions with analyst firms andminimal interactions with the vendors.3. Proof of concept bake-off for the short list contenders.4. Final assessment and decision.
Step 1: Long List
For the long list of candidates we got most of the players fromGartner’s Magic Quadrant. We found that we could broadlyclassify these into a few categories:1. Hardware and software solution,2. Software solution or3. Hardware solution.
Step 2: Short List
We trimmed down the initial list based on our size and performanceneeds. During this process we used input from Gartner and Forresterand had minimal interaction with the actual vendor sales reps. We didconsider customer references as part of the decision process.So, that led us to two players, as our main contenders for our POCbake-off. Interestingly, it turned out to be a mix of a “proprietaryhardware plus software” player and a “commodity hardware plussoftware” player. Although one vendor appeared to be a startup player,they were given high marks by the analysts and, more importantly theyseemed to be working very closely with Sun and even had commonboard members, making their viability question a little less risky.
Step 3: POC Bake-Off
We put together a set clear and transparent of guidelines for thisprocess to ensure that we had an even playing field. Some of the thingswe laid out were:
We insisted that the POC be done on site.
We would have one of our team members shadow the vendorengineer during the entire process to understand and reportback on what it took.
We also time-bound this to be a one week on-site activity.We knew that the on-site requirement would mean getting a bunch ofapprovals internally to allow for the vendor hardware to be set up in ourdata center, so we initiated that process during step one. By the time we
“We put together a set clear and transparent of guidelines for this process toensure that wehad an even playing field.”
Page 3
Saama Article
April 2010
had the final contenders, we had things in place from a legal/securityperspective to not hold.For the POC task, we identified one load process that was takingapproximately 33 hours as a prime candidate. This process consisted ofa set of Informatica jobs and involved picking up data from flat files andmoving it to stage, to final schema and finally to a set of aggregatetables. Since we had existing investments in reporting and analyticalapplications, we had to ensure that the existing schema remainuntouched so as to avoid changes to these BI applications.The load process consisted of all the three types of operations, i.e.,inserts, deletes and updates. We were going to measure the loadperformance of this entire task, and then we would have one of our BIenvironments point to the vendor appliance and benchmark runningsome of the long-running reports and queries.Both the vendors shipped over their boxes to our data center, and forlogistic reasons, it turned out that we had the vendors come in and workon their tasks on staggered weeks. So we had vendor one on week 1and vendor 2 on week three. We would have preferred this to be thesame week, but that would have required some additional setup on ourside.In both runs, we ran into some technical snags in moving the raw dataover to the appliance, but we used one of the big USB devices to moveit over. The runs were fairly smooth and the engineers were very good.They knew what they were doing and were able to carry out the taskswith minimal issues. The performance runs of the whole process in bothcases yielded mind-blowing results. The 33-hour process took less than40 minutes in both the cases. We also tested both using mixed loads,i.e., loading data and running reports simultaneously, and we did notsee much degradation. These systems have been architected to allowfor loads without impacting the end usage. Of course, this can also leadto some read consistency issues if the overall process is not designedproperly, but that’s a separate discussion.Both of these appliances had proven themselves with goodperformance, with very little difference (less than four minutes) betweenthem. So, the decision process now switched from performance toprice/performance, price here being total cost of ownership over athree-year period, including accounting for the projected data growth.This particular exercise we did in late 2008, and the economy was suchthat it was already a buyers market. Both vendors were willing to bendover backward to close the deal. So, even the price/performance wasbecoming a difficult metric to base the decision on.
“The performanceruns of the whole process in bothcases yielded mind-blowingresults. The 33-hour process took less than 40minutes in boththe cases.”

You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->