What is a data warehouse?
The standard answer to this question is it is a database designed to support themanagement decision-making process. A more accurate answer might be that it is where business managers can find the information they need to run the business correctly. Notice that the first answer is product-oriented, the second one emphasises functionality.The guru of data warehousing, Bill Inmon, characterises a data warehouse has being:
•
linked to, but different from, the many production databases that run in anorganisation;
•
subject-oriented, rather than application-oriented, to provide a consistent view of the business;
•
integrated, because data is consolidated from different application systems;
•
time variant, because information has a time dimension, whereas operational datais valid only at a particular moment;
•
non-volatile, since data is added to the data warehouse, rather than replaced.
Where did the concept come from?
While many people say that the concept of an ‘information’ warehouse came from IBM,a number of companies were building data warehouses during the 1980s but giving themdifferent names. In South Africa, pioneering organisations like Eskom and the former United Building Society set up very large databases for their management and executiveinformation systems well before the warehouse concept was established. IBM can becredited with realising the true potential of a database for management-oriented rather than operational needs, and of course for promoting the concept world-wide.
Why do I need a data warehouse?
Any of the following reasons can apply.1.Organisations have spent many years improving ways to put data into their operational systems; now they are beginning to appreciate the value of getting thatdata out again. However, in many cases the databases that support thoseoperational systems are not suitable for quick and easy access to information.2.Application databases are designed to handle data for specific, quite narrow purposes, not integrated organisational views.3.The IS department often does not want to allow users on the operational systemfor fear of degrading the system with resource-intensive queries, or breaching datasecurity and integrity.4.The data management differences between databases for application systems andfor decision-making cannot be re-conciled on one system.5.The software tools that users need should be on platforms that make it easy toaccess and use data - the operational platform may not be suitable.
6.The growth in recent years of decision support and data mining applicationsnecessitates the move to a different database architecture.7.‘Get those xxxx users off our back’!
How is data warehousing developing?
If you were building a data warehouse five years ago, chances are it would be on your company’s mainframe computer. But with increased use of the data warehouse came aheavier load on mainframe computer resources and degradation of overall performance. Nowadays, a common solution is to position the warehouse on a mid-range platform thatcan scale up easily and at a suitably low cost - conditions not yet available onmainframes. The lower implementation costs of a client-server warehouse make it easier to motivate the development of a warehouse. And data warehouses are not mission-critical systems, so the common client-server criticism about lack of system managementtools, is less significant.
What are the implications of data warehousing?
Because a data warehouse crosses organisational boundaries, there are serious non-technical issues that you must bear in mind at an early stage; not to do so can be a career-limiting move. Consider how organisations start data warehousing projects, from twodifferent approaches.One approach begins with corporate data modelling, and builds the warehouse by takinga broad view of business requirements; you can call this a top-down methodology. Theother approach is to focus on a specific information application and use that as the corearound which the warehouse will grow; this is the bottom-up methodology. Bothapproaches are valid but have their own problems. The top-down method requires longhard work to motivate and justify, political savvy, and approval by someone to pay up-front before seeing the final product. And that’s before you even start on development.The bottom-up approach avoids many of those problems, but runs the risk that theinformation requirement you start with becomes quickly out-dated, or is not accepted by business users.Before you embark on a data warehouse project, you need to look at your currentapplications and databases. If the warehouse is being developed because no one apartfrom programmers can access corporate data in its current state, then have a clear understanding about the data standards you will apply, and how the data warehouse will be designed. Too many organisations discover that they have multiple standards or definitions for their data. For data warehouses built using relational databases, it is acommon mistake to design the warehouse database around the same normalised form asthe application system. While the ‘normal form’ concept is fine for application efficiency,it creates havoc for users of a data warehouse. Next, examine the processes that will be implemented to extract and convert data fromyour existing application systems to the warehouse. Among other things, these processes
must ensure cleanliness and integrity - for example, identifying invalid data - as well asextracting the correct data every time - getting daily data for the wrong day is not anuncommon problem. While these housekeeping functions seem obvious, they can becomea major headache as the warehouse grows, unless they are properly thought through at theoutset.As the data warehouse grows, two other warehouse admin. functions will become moreimportant. One function is packaging the data for commonly requested queries so that thequeries run quicker. Doing this is easier said than done. Not many database systemsanalyse the contents of queries. More significantly, common queries at one time becomerare at another time due to business changes. This is an area that requires vigilance. Theother admin. function is what to do as the data ages. As the data gets older, theimportance of detail decreases. So to conserve disk space, this data should be summarisedto higher and higher levels of summary, and eventually archived. Deciding how and whatto summarise requires knowledge of past query patterns, and some insight into future business information requirements.To make the contents of a data warehouse easily understood, users have begun to demanddecent directories of corporate data, otherwise known as data dictionaries. This is a badnews and good news story. The bad news is that many organisations have spend a lot of time and money trying to implement corporate dictionaries for a data warehouse withlittle obvious success. While products do exist, the average user has great difficultyunderstanding them since they are still aimed mainly at programmers. The good news isthat some of the end-user query tools that are beginning to emerge have quite adequatedata dictionary functions. Watch this space for more developments.When starting a data warehouse, the best advice is to start with a small and focused need,and work on getting get it right first time. This makes it easier to project benefits, and toget data issues sorted out on a small system before you tackle a large one.
What benefits do data warehouses provide?
As with many strategic systems, determining bottom-line benefits is not straightforward.The major reason for a data warehouse is to improve access to corporate information. If this is accomplished successfully, users should find increased productivity because theydo not waste time looking or waiting for information. With better access to thisinformation, management can make better decisions. By separating the operational andwarehouse data, you allow both databases to be optimised for their own specific purposes, and reduce possible disruptions to the operational systems. A truly usefulwarehouse will also create spin-off by suggesting further applications.
Search History:
Result 00 of 00
00 results for result for
p.
More From This User
Notes
Load more
