May 27, 2011

Its The Dawning Of The Age Of BI DBMS

by Boris Evelson for Business Process Professionals

Making Leaders Successful Every Day

For Business Process Professionals

Its The Dawning Of The Age Of BI DBMS

May 27, 2011

complement your rDBMS-Based BI With More Agile DBMS Technologies

by Boris Evelson with rob Karel, Brian Hopkins, James G. Kobielus, leslie Owens, and Allison caine

ExEcuT I v E S u M MA ry
We know how to address the key business intelligence (BI) challenges of the past 20 years, such as stability, robustness, and rich functionality. Agility and flexibility challenges now represent BIs next big opportunity. Business process professionals realize that earlier-generation BI technologies and architecture, while still useful for more stable BI applications, fall short in the ever-faster race of changing business requirements. Forrester recommends embracing Agile BI methodology, best practices, and technologies to tackle agility and flexibility opportunities. Alternative database management system (DBMS) engines architected specifically for Agile BI will emerge as one of the compelling Agile BI technologies business process pros should closely evaluate and consider for specific use cases.

TABl E O F cO n TE nTS
2 Fitting BI Into A RDBMS Is Like Putting A Square Peg Into A Round Hole Specialized BI Databases Will come To The rescue 4 Learn To Navigate The Crowded Landscape Of Specialized BI DBMS use columnar rDBMS To leverage And Turbocharge your Existing BI Applications use In-Memory Index When nothing Else Is Fast Or Agile Enough use Inverted Index For complex Data Structures And unstructured content use Associative When you cant Predict The Future But Have To Prepare For Anything
rEcOMMEnDATIOnS

n OT E S & rE S O u rcE S
Forrester interviewed 18 vendors and 10 user companies for this research.

Related Research Documents Forresters Business Intelligence DBMS Effort Estimation Model May 27, 2011
Trends 2011 And Beyond: Business Intelligence March 31, 2011 The Forrester Wave: Enterprise Data Warehousing Platforms, Q1 2011 February 10, 2011 Stay Alert To Database Technology Innovation november 19, 2010 Empower BI HErOes With Self-Service Tools October 26, 2010

14 Choose Wisely Between Mainstream And Specialized DBMSes For BI

WHAT IT MEAnS

14 BI-Specific DBMS Will Overshadow Traditional RDBMS In 10 Years

2011 Forrester Research, Inc. All rights reserved. Forrester, Forrester Wave, RoleView, Technographics, TechRankings, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective owners. Reproduction or sharing of this content in any form without prior written permission is strictly prohibited. To purchase reprints of this document, please email clientsupport@ forrester.com. For additional reproduction and usage information, see Forresters Citation Policy located at www.forrester.com. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change.

Its The Dawning Of The Age Of BI DBMS

For Business Process Professionals

FITTINg BI INTO A RDBMS IS LIkE PuTTINg A SquARE PEg INTO A ROuND HOLE Mainstream relational databases are an awkward fit for BI. You can use them, but it requires lots of tuning and customization and constant optimization which is difficult, time-consuming, and costly. Unfortunately, row-based relational database management systems (RDBMSes) like IBM DB2, Microsoft SQL Server, Oracle, and Sybase ASE were originally designed and architected for transaction processing, not reporting and analysis.1 In order to tune such a RDBMS for BI usage, specifically data warehousing, architects usually:2

Denormalize data models to optimize reporting and analysis. Data modelers usually model

transactional data in whats referred to as third normal form (3NF). 3NF best practice calls for storing each data entity and attribute in one place and then creating multiple lookup or decoding tables that map and translate keys into full item descriptions. For example, a customer record would only have a pointer, or a key, to a full address record stored in a different table. This works great when one needs to insert, update, or delete rows of data, as the amount of data for each transaction is limited. BI abhors such a structure, as it often creates very complex multiway joins of data that negatively affect query performance. To solve the problem, developers flatten or denormalize data models so that every row contains all of the attributes needed for a report. In the example above, each customer record would contain the customer address and become longer and flatter.

Build indexes to optimize queries. Transactional processes, on the other hand, loathe many

database indexes, such as pointers and data shortcuts, because inserting, updating, or deleting records will then require updating all relevant indexes, slowing down performance. The opposite holds true for BI the more indexes the merrier! Additional indexes actually improve query performance.

Build aggregate tables to optimize summary queries. Some SQL queries contain a group by

clause to summarize information by time period, geographic region, line of business, or general ledger chart of accounts, for example. But calculating such summaries or aggregates at query time requires many processing cycles, especially if youre managing millions or even billions of rows of detailed transactions. To solve this problem, data integration pros usually design a BI database schema to pre-aggregate data by the attributes they think will be used most often. ETL processes load these aggregated tables during off-peak times, so that subsequent reporting queries can instantaneously access the aggregated data without waiting for the database to pull it all together.

Build OLAP cubes to further optimize analytic queries. Multidimensional online analytical

processing (OLAP) databases, also known as cubes, represent another specialized approach to aggregate and index transactions by business-defined hierarchies like product level, region, and business unit. BI pros often use products like IBM Cognos PowerCubes, Microsoft Analysis Services, Oracle Essbase, SAS OLAP, and open-source Mondrian cubes on top of a RDBMS to further tune and optimize analytical queries.3

May 27, 2011

2011, Forrester research, Inc. reproduction Prohibited

Its The Dawning Of The Age Of BI DBMS

For Business Process Professionals

Unfortunately, theres one basic problem with these approaches: Its impossible to build denormalized data models, indexes, and aggregates for every possible query that users will execute during the lifetime of a database. So BI pros must pick their battles and optimize the RDBMS based on current or near-term expected usage. But in todays fast-paced business environment, near-term may mean days or even hours, requiring BI and data warehouse (DW) pros to spend a significant amount of their time doing nothing more than constantly optimizing and reoptimizing these databases.4 Specialized BI Databases Will Come To The Rescue Mainstream row-oriented RDBMS will not be the lead database technology for BI going forward. First, theres the need to constantly redesign entity and attribute relationships in the logical data model and reoptimize indexes and aggregates in the physical data model. And these RDBMSes offer minimal support for:

Unstructured content. Databases crave structure. Regardless of what many RDBMS vendors

say, theres no easy way to organize unstructured content from emails, documents, and web traffic unless you first extract and parse structured entities, like customers and products, and attributes like customer name and address. Data management pros must first organize unstructured content into well-defined data structures before adding it to a RDBMS even one that has XML extensions to handle hierarchical and semistructured data.

Diverse data structures. RDBMS vendors and users have significant experience supporting

financial information, with its relatively constant and easily defined data structures. For example, when organizing data for income statements and balance sheets, one can typically structure everything by account, time period, geographic region, and business or legal unit. But its much more difficult to organize data structures representing lower levels of detail for operational and product analysis say, by product type into a neat RDBMS model. Any manufacturer, distributor, or retailer who deals with thousands of product types where each product has a unique set of attributes can articulate these limitations. For example, a car manufacturer may use attributes like diameter, width, and seasonality to describe tires and attributes like manual, automatic, 4-speed, and 5-speed to describe transmissions. Attempting to fit diverse descriptions into the same attribute list results in ragged, sparse, and unbalanced product hierarchies a nightmare for data architects to model and users to query.5

To address these challenges, both mainstream BI vendors and startups introduced databases designed specifically for BI reporting and analysis use cases. Some of these specialized technologies have been available for years, but Forrester still sees relatively low levels of adoption and buyer interest. 2009 was the first year in which we witnessed early adoption, but adoption levels actually dipped slightly in 2010. However, the number of IT execs and decision-makers saying that they plan to implement these new technologies soon grew slightly (see Figure 1).

2011, Forrester research, Inc. reproduction Prohibited

May 27, 2011

Its The Dawning Of The Age Of BI DBMS

For Business Process Professionals

Figure 1 Adoption Of And Interest In BI-Specific DBMS

What are your firms plans to adopt specialized database engines like OLAP, columnar databases, or warehouse appliances? Expanding/upgrading implementation Implemented, not expanding Planning to implement in the next 12 months Planning to implement in a year or more Interested but no plans Not interested Dont know 6% 7% 10% 9% 8% 11% 18% 14% 15% 16% 18% 2010 2009*

21% 22% 25%

Base: 784 IT executives and decision-makers in North America and Europe *Base: 853 IT executives and decision-makers in North America and Europe Source: Forrsights Software Survey, Q4 2010 *Source: Enterprise And SMB Software Survey, North America And Europe, Q4 2009
58853 Source: Forrester Research, Inc.

LEARN TO NAvIgATE THE CROWDED LANDSCAPE OF SPECIALIzED BI DBMS Relatively low adoption of BI-specific DBMSes reflects the fact that these databases are very specialized and not meant to support a wide variety of use cases unlike their bigger, older, more versatile, jack-of-all-trades RDBMS cousins. Additionally, not all BI-specialized DBMSes are created equal, and inconsistent or conflicting vendor marketing messages only add to customer confusion. Business process pros should consider four types of BI-specific DBMS: columnar RDBMS and inmemory, inverted, and associative index DBMS. The applicability of and use cases for each specialized DBMS vary greatly depending on multiple factors, including but not limited to database size, data structure complexity, and a requirement to organize and report on unstructured content (see Figure 2). Four other non-relational DBMSes are worth mentioning, but theyre too unique and differentiated for comparative review. The first two are NoSQL key value stores and RDF, semantic, or triplet stores although Forrester still questions whether theyre really applicable to BI. The other two are graph DBMS and big data technologies like Hadoop (HDFS and HBase) and MapReduce.6

May 27, 2011

2011, Forrester research, Inc. reproduction Prohibited

Its The Dawning Of The Age Of BI DBMS

For Business Process Professionals

Figure 2 Four Types Of Specialized BI DBMS

Quickly changing requirements In-memory Agile BI

Inverted/associative index Columnar RDBMS

Relative scalability High Medium Low

Slowly changing requirements

Traditional BI Similar data Disparate data

Best for: Large volumes Reusing existing BI infrastructure Nonrelational Disparate data Polystructured multiple star schemas and/or sparse, ragged, unbalanced hierarchies Unstructured content Data-driven BI (versus schema-driven) Full functionality Nearly full functionality

Columnar In-memory RDBMS index*

Inverted index* Associative

Limited/partial functionality

Very limited functionality

No capabilities in this area

*These technologies often overlap signi cantly. For example, all of the in-memory vendors use columnar, but not relational, technology. Some of the inverted index vendors use a combination of in-memory (for caching) and columnar (for persistence) technologies.
58853 Source: Forrester Research, Inc.

2011, Forrester research, Inc. reproduction Prohibited

May 27, 2011

Its The Dawning Of The Age Of BI DBMS

For Business Process Professionals

use Columnar RDBMS To Leverage And Turbocharge Your Existing BI Applications Columnar DBMS is the closest relative of traditional row-based RDBMS; its also relational and thus has most of the benefits of row-based RDBMS, such as the ability to work with all mainstream BI tools. Unlike other RDBMSes, however, it stores data in columns rather than rows (see Figure 3). Querying requires selecting, sorting, filtering and aggregating data by columns, so consider this specialized database type as innately tuned for BI. But just like all other specialized BI DBMSes, this comes at the expense of being less optimized for transaction processing. A major healthcare company accelerated report generation from five days to a matter of seconds and increased the number of users from 200 to 2,000 by replacing its row-based RDBMS database with Sybase IQ columnar RDBMS. Row-based RDBMSes can also be tuned to improve performance, but this retuning needs to occur for every new application and new query not a very agile approach. In a similar example, a leading business-to-business (B2B) and retail office products company uses ParAccel to augment its existing row-based DW, where it needs the extra BI performance and agility. In addition to its ability to leverage existing BI tools and apps, Forrester recommends using columnar RDBMS technology when:

Data volumes are exceptionally large. Data stored in columns compresses better than data

stored in rows because data values in a row, such as combinations of numbers, strings, and dates, often differ whereas data values in most columns, like all dates, are similar. Additionally, loading raw data into a row-based RDBMS and tuning it for BI can increase the size of the raw data by a factor of three to seven after building indexes and aggregates. In a columnar RDBMS, a column already represents its own index; aggregates are calculated on the fly, which can keep the database size roughly equal to that of the raw data set or sometimes even cut it in half.

Business requirements call for agility and flexibility. Columnar DBMSes greatly reduce the

need for physical modeling and query tuning, as there are fewer or no indexes and aggregates, and often eliminate the need to build multidimensional data sets like cubes. This means that BI developers can implement new business requirements much more quickly and with less effort.

In recent years, columnar RDBMS has greatly improved upon its inherent weakness: the ability to insert, change, and delete detailed transactions, usually in so-called microbatches. But even with these improvements, it still cant match the innate transaction-processing strength of its row-based cousins. Hence, Forrester does not recommend basing a BI implementation on columnar RDBMS for very low-latency, real-time operational reporting like real-time algorithmic trading analytics. Pure-play columnar RDBMS vendors on Forresters watchlist include Calpont InfiniDB, HP Vertica, Infobright, Kx Systems kdb+, ParAccel, PivotLink, Quiterian, SAND Technology, SAP Sybase IQ, SAP BW Accelerator and the new HANA appliances, SenSage, SiSense, and 1010data. Mainstream DBMS vendors and vendors with hybrid row-based and columnar DMBSes include EMC Greenplum, IBM Netezza, Ingres VectorWise, Oracle, and Teradata Aster Data. Microsofts PowerPivot product is also based on a columnar VertiPaq engine.7

May 27, 2011

2011, Forrester research, Inc. reproduction Prohibited

Its The Dawning Of The Age Of BI DBMS

For Business Process Professionals

Figure 3 Major Differences Between row-Based And columnar rDBMSes

Need a special index to speed up query by column Aggregation: scan all rows; extract column values or build and maintain an index; aggregate

Customer No. Rowbased RDBMS 1 2 3 Values in columns are similar and easier to compress

Name John Andre David Each column is its own index

Address Newton Washington San Diego Easy-to-manipulate columns

Balance $100 $200 $300 Values in row are di erent and harder to compress Easy-to-manipulate rows

Difcult-to-manipulate columns

Aggregation: just add all values in a column

Benet Challenge

Columnar RDBMS

Customer No. 1 2 3

Name John Andre David

Address Newton Washington San Diego

Balance $100 $200 $300

Difcult-to-manipulate rows

58853

Source: Forrester Research, Inc.

use In-Memory Index When Nothing Else Is Fast Or Agile Enough An in-memory index DBMS can be described as a spreadsheet on steroids. It does precisely what spreadsheets were designed to do: flexible reporting and analysis.8 Spreadsheets do not require complex data models just load rows and columns of raw data into them and analyze it by sorting, filtering, and pivoting. BI applications based on in-memory indexes can impose fixed models, but the data model, data, and resulting report are one and the same. So creating a report is the same process as modeling data and is known as inline modeling. For example, a large global transportation company uses Microsoft PowerPivot to address the ongoing challenges the logistics industry faces, as unstable geopolitics and unpredictable atmospheric and geological events result in constantly changing and often unpredictable requirements. And
2011, Forrester research, Inc. reproduction Prohibited May 27, 2011

Its The Dawning Of The Age Of BI DBMS

For Business Process Professionals

several major financial services institutions use Tibco Spotfire to make quick modifications to any dashboards that are not already preconfigured so they can see changes as they happen without requesting IT support. Finally, a global manufacturing company likes its QlikView product because of the agility with which it can access varied views of information agility difficult to replicate in RDBMS-based BI applications. Additionally, when evaluating in-memory technologies, consider that:

The best use cases support advanced agility and flexibility requirements. In-memory BI offers

the most flexibility and agility of the specialized DBMS technologies reviewed here. Consider inmemory when RDBMS-based BI is too inflexible and spreadsheets dont offer enough depth or breadth of BI functionality, like formatting, advanced analytics, and interactive data visualization.

The functionality in-memory vendors offer varies greatly. The leading in-memory BI vendors

differ significantly for example, whether the product can manage data models too large to fit into a single memory space and whether it can update in-memory models row by row for low-latency, operational BI or whether it needs to reload or rebuild the entire model (see Figure 4). Business process pros have to ask questions like: Are the in-memory DBMS and the vendor GUI inseparable, or can other BI tools access the in-memory DBMS? Does the vendor offer integrated advanced analytics like statistical analysis and predictive modeling? Is the in-memory DBMS a true index that does not require a fixed OLAP model, or is it more of a traditional OLAP cube with a star schema, facts, and dimensions that just happens to be all loaded into memory, like with IBM Cognos TM1 and MicroStrategy? Finally, if business users are to use these tools, how much IT involvement is required to set up and maintain the applications?

You may not be able to fit your entire data model into a single memory space. All in-

memory vendors offer pretty good data compression, so most small to medium-size business and departmental BI applications should fit into a single memory space without issue. Even when deploying these applications on 64-bit servers, Forrester is not aware of any production applications that are larger than 200 GB to 300 GB. When sizing these applications, consider the size of the raw data set, compression ratios look for at least 10:1 and the number of concurrent users, as each active user may take up anywhere from a few dozen to a few hundred megabytes of RAM. There are also other vendor-specific factors, like data heterogeneity and sparseness, to take into account. If the total number is larger than a few hundred gigabytes, consider Tibco Spotfire, which can dynamically swap chunks of your model in and out of RAM, or in-memory/disk hybrid offerings from Endeca, SiSense, and Quiterian, because youre probably not a good candidate for other in-memory vendor technologies.9

Operational risk may increase. All of the power, flexibility, and agility that in-memory

technology offers is not free of risk: If you dont know what youre doing, you may come up with the wrong answers. While existing RDBMS data models are indeed rigid, in most cases they do protect business users from creating queries and reports that may produce wrong results and lead to incorrect decisions. In-memory-based BI could be a free-for-all. Yes, IT can restrict how you

May 27, 2011

2011, Forrester research, Inc. reproduction Prohibited

Its The Dawning Of The Age Of BI DBMS

For Business Process Professionals

use or change in-memory models, but that would defeat the purpose and main advantages of inmemory BI. So dole out these applications carefully and create a set of guidelines and policies that spells out when to encourage the use of such agile but risky applications and when not to. Additionally, follow Forresters recommendations on Agile BI; reduce the risk by making sure that no matter how business users use in-memory BI applications, IT is still in firm control of preparing data for those apps, including data integration, data cleansing, and data security.

2011, Forrester research, Inc. reproduction Prohibited

May 27, 2011

10

Its The Dawning Of The Age Of BI DBMS

For Business Process Professionals

Figure 4 Features Of Selected In-Memory Index vendors

Ac t O uat bj e ec BI ts RT Da Ad ta vi zo rS ol ut IB io M ns Co gn os TM M ic 1 ro so ft Po w M er ic ro Pi vo St ra t te gy Q lik Vi ew

HA NA

JM

Ta bl ea

Feature

Tibco Spot re

SA P

In-memory index type OLAP functionality (MDX queries) Associative index (analysis with inline modeling) In-memory architecture Memory swapping/ paging (for models that are larger than RAM)* Row updates and inserts (for low latency, operational BI) Write-back (generating new transactions for what-if scenarios) BI features Other BItools can access it Advanced analytics (statistical analysis, data mining, predictive) Faceted navigation/ search via Eclipse ODA OEM KXEN

via MDX

via SPSS Limited

Limited

SA S

Limited

Search

Search

Search

*While this is a nice-to-have option, using models that do not t into a single memory space will signi cantly slow down analysis.
58853 Source: Forrester Research, Inc.

May 27, 2011

2011, Forrester research, Inc. reproduction Prohibited

Its The Dawning Of The Age Of BI DBMS

For Business Process Professionals

11

Figure 4 Features Of Selected In-Memory Index vendors (cont.)

Ac t O uat bj e ec BI ts RT Da Ad ta vi zo rS ol ut IB io M ns Co gn os TM M ic 1 ro so ft Po w M er ic ro Pi vo St ra t te gy Q lik Vi ew

HA NA

JM

Ta bl ea

Feature Other Requires IT involvement Requires specialized hardware SDK/APIs O ered as SaaS
58853

Tibco Spot re

SA P

use Inverted Index For Complex Data Structures And unstructured Content Common RDBMS agility challenges involve the management of unstructured content. The inverted index BI DBMS approach questions the assumption that you must start with a database and then worry about tuning it by building indexes on top of it. This approach builds one big index, but instead of just pointing to data sources as traditional search engines like Google or Yahoo do it embeds data in the index itself. Typical use cases for inverted index DBMS include:

Managing a variety of unharmonized data types and data sources. A Fortune 100 consumer
packaged goods (CPG) company recently implemented Endeca to analyze heterogeneous data from multiple sources within a single view in order to determine the root causes of observed responsiveness to sales campaigns. The relevant data for this application came from different sources and spanned structured data, semistructured data, and unstructured content. An inverted index DBMS gave the CPG company the ability to see this complex span of data and content in a single interactive analytical view. This would have been much harder to achieve using row-oriented RDBMS, particularly because the firm included more challenging formats like unstructured content and unharmonized structured data and allowed the interactive discovery of relationships between these disparate collections.

Combining SQL queries and keyword search. A global financial services firm recently

implemented Attivio because the inflexibility of its traditional RDBMS would have required very costly and time-consuming data normalization processes to integrate numerous new data sources. The firms earlier-generation, RDBMS-based BI application could not handle the variety

2011, Forrester research, Inc. reproduction Prohibited

SA S

Source: Forrester Research, Inc.

May 27, 2011

12

Its The Dawning Of The Age Of BI DBMS

For Business Process Professionals

of data types, including unstructured and semistructured content, that the firm needed to access, and did not support the desired range of query styles: SQL, keyword search, and the use of financial services-specific ontologies. Forrester also recommends considering inverted index DBMS when you (see Figure 5):

Build too many data marts or cubes. Earlier, we described examples from manufacturing and

retail industries with complex data structures that often hold unbalanced, ragged hierarchies. To address the challenges that such complex data structures and hierarchies pose for traditional relational or even multidimensional DBMSes, IT often builds specialized data marts to flatten out hierarchies or programmatically fill in missing nodes. In some extreme cases, as many as one-third of all new reports require building specialized data marts or cubes. Such complex data structures are often more hierarchical than relational in nature. Theyre easy to represent as XML and index in a single inverted index DBMS.

Really need data-driven, not schema-driven, BI. Entirely schema-driven BI applications based
on a relational data model hold the key to what one can analyze and report on. BI applications must be created specifically for the query at hand to navigate the data effectively. In contrast to the schema-bound data exploration and query refinement BI provides, an inverted index BI DBMS provides data-driven data exploration and query refinement. With a data-driven approach from ingest-through-the-index to presentation, a new source system column flows straight through to the application. In some vendor apps, such as Endecas, data-driven UI components even allow users to rearrange the data without requesting custom reports.

Dont know what you dont know. All RDBMS- or OLAP-based BI applications assume

that you need to know what you are looking for before you ask a question and construct a query. But BI end users often dont! Faceted navigation a key feature of all inverted DBMS BI applications allows you to subtract what you know you dont want. A good example is the faceted navigation on any eCommerce website, where you can eliminate all products but electronics, then all electronics but computers, and then all computers other than laptops in the $500 to $1,000 price range. Perhaps you didnt know what you were looking for at the beginning of your search, but categories and prompts let you easily eliminate what you knew you didnt want. In many business scenarios, its more obvious what you dont want than what you do. In a similar example, one of Endecas retail customers makes its inside sales reps more productive by helping them eliminate all customers except for those who purchased products in the past six months, then all recent buyers except for those who bought part of a bundle, then all bundle buyers except for the ones for whom the rest of the bundle is now on sale.

May 27, 2011

2011, Forrester research, Inc. reproduction Prohibited

Its The Dawning Of The Age Of BI DBMS

For Business Process Professionals

13

Figure 5 Features Of Selected Inverted Index DBMS vendors

Features* GUI Access by third-party tools *Exalead and Sinequa also claim similar capabilties.
58853 Source: Forrester Research, Inc.

Attivio Custom-built SQL-based

Endeca Part of the package XQuery-based

use Associative When You Cant Predict The Future But Have To Prepare For Anything The future is truly unpredictable. Forrester often hears from DW planners that their business users want everything stored and tracked in their DW, just in case. Thats precisely what associative DBMS attempts to do. Imagine a DW that can store all-to-all relationships associations or vectors between every entity and every attribute in your domain, with counters, aggregates, and indexes for every intersection! A Global 100 manufacturing company uses Saffron Technology in a number of operational areas to leverage past engineering decisions and business experiences, such as component and part order optimization, failure root-cause analysis, and predictive maintenance. Other associative DBMS products include Ingres VectorWise, Illuminate Solutions iLuminate, Relavance Associate, and Lazysoft Sentences. A variation of an associative DBMS an associative index, like Splunk provides similar functionality by just pointing to data sources rather than physically pulling data into a DBMS. The price of associative DBMS is its infamous X-factor: the multiplication factor used to calculate the size of a DBMS as a multiple of the raw data set. In many of the associative databases used in academia, this factor gets as high as 100, although Saffron claims to get it down to 30. Additionally, because associative DBMS structures do not resemble anything close to relational or multidimensional structures, they require a purpose-built GUI, and SQL- and MDX-based queries cant easily access them. Associative DBMS does not think in terms of joins or where clauses, but rather in connections, analogies, classifications, and trends. Forrester is aware of one leading BI vendor releasing a version of its popular product to connect to Saffron via such REST APIs in the near future. More may follow.

2011, Forrester research, Inc. reproduction Prohibited

May 27, 2011

14

Its The Dawning Of The Age Of BI DBMS

For Business Process Professionals

r E c O M M E n D AT I O n S

CHOOSE WISELY BETWEEN MAINSTREAM AND SPECIALIzED DBMSES FOR BI

Forrester does not recommend deploying BI-specific DBMSes and specialized BI applications to replace traditional, broadly functional rDBMS and enterprise BI platforms. not just yet. rather, use them as complementary platforms and supplementary applications for all of the use cases highlighted in this report. In order to ensure the effectiveness of these complementary technologies and efforts, Forrester recommends that business process pros:

Prioritize and emphasize data governance efforts. Data governance efforts are extremely
important for BI. When and if you implement these additional complementary specialized BI applications, the data governance will be an order of magnitude more critical, as you will have to deal with multiple BI applications, multiple analytical databases, and less structured data usage.10

Prioritize and emphasize common metadata efforts. until large-stack vendors like
IBM, Microsoft, Oracle, and SAP acquire, build, and integrate all of the technologies weve described here, you are more than likely to end up using more than one BI tool. Therefore, business process pros should work with their enterprise architecture (EA) colleagues to create a single semantic layer using data federation technology using infrastructureas-a service (IaaS) products, for example that can then serve as virtual data source to all, including specialized BI applications. Or create and maintain a common metadata repository.11 use these centralized metadata repositories to sync metadata between all BI platforms and applications. To ensure these metadata repositories integrate across competing BI applications, look for the vendors to OEM standard metadata adapters from Meta Integration Technology or adhere to metadata exchange standards like xMI.12

Build a business case with tangible ROI. Forrester often finds that the most effective BI
business cases which are often also the most difficult to build and support focus on top-line benefits.13 However, more pragmatic business cases usually involve calculating the cost of ownership, which mostly targets support and not just initial license costs. leverage Forresters BI DBMS Effort Estimation Model to help you understand the potential long-term cost-of-ownership savings from specialized BI DBMSes and tools.14
W H AT I T M E A n S

BI-SPECIFIC DBMS WILL OvERSHADOW TRADITIONAL RDBMS IN 10 YEARS

unlike claims by some of the vendors described in this report, we cant predict the future and offer a crystal-ball vision into what percentage of BI applications will be based on these specialized DBMS technologies. But the tried and true 80/20 rule works pretty well most of the time. Therefore, we predict that within three years, 20% of all BI applications are going to be based on these specialized technologies. That fraction will then slowly but surely evolve to 80% over the next 10 years.

May 27, 2011

2011, Forrester research, Inc. reproduction Prohibited

Its The Dawning Of The Age Of BI DBMS

For Business Process Professionals

15

SuPPLEMENTAL MATERIAL Companies Interviewed For This Document Actuate Advizor Solutions Attivio Endeca IBM Kanda Software Microsoft MicroStrategy ParAccel Pivotstream LLC ENDNOTES
1

QlikView Quiterian Saffron Technology SAP SAS SiSense Splunk Tableau Software Tibco Software You Get

In Forresters 153-criteria evaluation of enterprise open source and closed source database management systems (DBMSes), we rated the highest performers to determine which vendors offer the most mature, high-performance, scalable, secure, and flexible solutions. See the June 30, 2009, The Forrester Wave: Enterprise Database Management Systems, Q2 2009 report. In Forresters 56-criteria evaluation of enterprise data warehousing (EDW) platform vendors, we found the EDW market increasingly competitive, as illustrated by tighter clustering of top vendors. See the February 10, 2011, The Forrester Wave: Enterprise Data Warehousing Platforms, Q1 2011 report. Online analytical processing (OLAP) is a core component of a complex business intelligence (BI) architectural stack. Even as vendors begin to explore alternative technologies for slicing and dicing large data sets, traditional OLAP approaches will remain an essential component in most BI installations for at least the next few years. See the November 7, 2008, OLAP: In Fashion Or Old-Fashioned? report. Forrester continues to see ever-increasing levels of interest in and adoption of business intelligence (BI) platforms, applications, and processes. There is one key ber-trend and best practice: agility. BP pros should adopt Agile BI processes, technologies, and architectures to improve their chances of delivering successful BI initiatives. See the March 31, 2011, Trends 2011 And Beyond: Business Intelligence report. In ragged or unbalanced hierarchies, the parent of at least one member of a dimension is not in the level immediately above the member and the branches of the hierarchies can descend to different levels. Additionally, in a ragged hierarchy, null values can appear in any of the level columns. Null column values between member names are skipped, so a parent can have a child member multiple levels below the parent level.

2011, Forrester research, Inc. reproduction Prohibited

May 27, 2011

16

Its The Dawning Of The Age Of BI DBMS

For Business Process Professionals

Growth in the volume and pace of business is driving new application and data requirements that are often well beyond the design limits of existing relational database technologies. Relational databases will ultimately innovate to deliver higher degrees of automation, improved performance, and scale as well as to support larger volumes of unstructured data. See the November 19, 2010, Stay Alert To Database Technology Innovation report. Many of the in-memory index vendors mentioned in the in-memory index section are based on a variation of in-memory columnar DBMS/index. Microsoft recently announced that the Denali version of SQL Server will include VertiPaq as a new storage/index type. In Forresters book Empowered, Forrester Research analysts describe ways that new technologies can empower business people and make them true HEROes through individual contributions to their respective companys top and bottom lines. The book also points out how business intelligence (BI) is a key technology for HEROes, not only helping them to make sense of the mountains of data that they have to deal with, but actually allowing them to make better and faster decisions. See the October 26, 2010, Empower BI HEROes With Self Service Tools report. In the near future, in-memory technology will span multiple memory spaces and break through current limitation barriers. This is something that SAP claims its HANA in-memory BI appliance can already accomplish. Or perhaps other in-memory BI vendors will integrate their products with elastic cache technologies like Tibco ActiveSpaces to make random access of data across nodes feasible. Business process and data management professionals recognize the business need and potential value effective data governance could deliver, yet data governance remains an immature and poorly understood competency within most organizations. Identify the most critical business processes and decisions that trusted data can optimize, define the policies, business rules and standards to ensure trusted data, and deliver business-value oriented metrics that will justify the resource investments. See the May 4, 2011, Data Governance Must Bridge Business Process, Policy, Architecture, And Value report. Some options include: standalone metadata repositories like ASG-Rochade; integrated data management metadata repositories from Ab Initio, IBM, and Informatica; and specialized BI metadata vendors like Kalido, WhereScape, BIReady, and BI-Insight. Source: Meta Integration Technology (http://www.metaintegration.net). As the demand for pervasive and comprehensive BI applications increases, the complexity and cost of large enterprise BI implementations are not easily reduced. Therefore it is now more critical than ever to demonstrate tangible value from BI by building bulletproof BI business cases. See the August 25, 2009, The Business Case For BI: Now More Critical Than Ever report. See the May 27, 2011, Forresters Business Intelligence DBMS Effort Estimation Model report.

10

11

12 13

14

May 27, 2011

2011, Forrester research, Inc. reproduction Prohibited

Making leaders Successful Every Day

Headquarters Forrester Research, Inc. 400 Technology Square Cambridge, MA 02139 USA Tel: +1 617.613.6000 Fax: +1 617.613.5000 Email: forrester@forrester.com Nasdaq symbol: FORR www.forrester.com For a complete list of worldwide locations visit www.forrester.com/about. Research and Sales Offices Forrester has research centers and sales offices in more than 27 cities internationally, including Amsterdam; Cambridge, Mass.; Dallas; Dubai; Foster City, Calif.; Frankfurt; London; Madrid; Sydney; Tel Aviv; and Toronto.

For information on hard-copy or electronic reprints, please contact Client Support at +1 866.367.7378, +1 617.613.5730, or clientsupport@forrester.com. We offer quantity discounts and special pricing for academic and nonprofit institutions.

Forrester Research, Inc. (Nasdaq: FORR) is an independent research company that provides pragmatic and forwardthinking advice to global leaders in business and technology. Forrester works with professionals in 19 key roles at major companies providing proprietary research, customer insight, consulting, events, and peer-to-peer executive programs. For more than 27 years, Forrester has been making IT, marketing, and technology industry leaders successful every day. For more information, visit www.forrester.com.

58853