Star Schema and Data warehousing Cardinality is used for aggregation purposes and helps Frame Work manager

determine where the facts reside

Query subjects with only one to many cardinalities attached are identified as Fact tables Cardinality is applied in the context of a query.
OLTP

OLTP stand for Online Transaction Processing. This is a standard, normalized database structure. OLTP is designed for transactions, which means that inserts, updates, and deletes must be fast. Imagine a call center that takes orders. Call takers are continually taking calls and entering orders that may contain numerous items. Each order and each item must be inserted into a database. Since the performance of the database is critical, we want to maximize the speed of inserts (and updates and deletes). To maximize performance, we typically try to hold as few records in the database as possible.
OLAP and Star Schema

OLAP stands for Online Analytical Processing. Here, we will assume that any system of read-only, historical, aggregated data is an OLAP system. In addition, we will assume an OLAP/Star Schema can be the same thing as a data warehouse. It can be, although often data warehouses have cube structures built on top of them to speed queries. A data warehouse (or mart) is way of storing data for later retrieval. This retrieval is almost always used to support decision-making in the organization. That is why many data warehouses are considered to be DSS (Decision-Support Systems). Both a data warehouse and a data mart are storage mechanisms for read-only, historical, aggregated data. Date 4/24/99 Order Number 1 Dog Food 5 Cat Food 2

2 3 4 5

3 2 2 3

0 6 2 3

4/25/99

1 2 3

3 2 4

7 1 0

Table 1

Now, as you can see, there are several transactions. This is the data we would find in a standard OLTP system. However, our data warehouse would usually not record this level of detail. Instead, we summarize, or aggregate, the data to daily totals. Our records in the data warehouse might look something like this: Quantity Sold Date 4/24/99 4/25/99
Table 2

Dog Food 15 9

Cat Food 13 8

GRAIN: - Almost every star schema is aggregated to some base level, called the grain Like the time dimension the grain might be daily / weekly/ monthly are the grains

OLTP Systems
OLTP, or Online Transaction Processing, systems are standard, normalized databases. OLTP systems are optimized for inserts, updates, and deletes; in other words, transactions. Transactions in this context can be thought of as the entry, update, or deletion of a record or set of records

OLTP Disadvantages
There are some disadvantages to an OLTP structure, especially when we go to retrieve the data for analysis. For one, we now must utilize joins and query multiple tables to get

all the data we want. Joins tend to be slower than reading from a single table, so we want to minimize the number of tables in any single query. With a normalized structure, we have no choice but to query from multiple tables to get the detail we want on the report. One of the advantages of OLTP is also a disadvantage: fewer indexes per table. Fewer indexes per table are great for speeding up inserts, updates, and deletes. In general terms, the fewer indexes we have, the faster inserts, updates, and deletes will be. However, again in general terms, the fewer indexes we have, the slower select queries will run. For the purposes of data retrieval, we want a number of indexes available to help speed that retrieval. Since one of our design goals to speed transactions is to minimize the number of indexes, we are limiting ourselves when it comes to doing data retrieval. That is why we look at creating two separate database structures: an OLTP system for transactions, and an OLAP system for data retrieval.

Reasons to Denormalize
Speed

Weve already discussed some disadvantages to the OLTP structure; it is built for data inserts, updates, and deletes, but not data retrieval. Therefore, we can often squeeze some speed out of it by denormalizing some of the tables and having queries go against fewer tables. These queries are faster because they perform fewer joins to retrieve the same record set.
Joins are slow

Joins are slow, as we have already mentioned. Joins are also confusing to many end users. By denormalizing, we can present the user with a view of the data that is far easier for them to understand.

Facts and Dimensions

When we talk about the way we want to look at data, we usually want to see some sort of aggregated data. These data are called measures. These measures are numeric values that are measurable and additive Just tracking measures isnt enough, however. We need to look at our measures using those by conditions. These by conditions are called dimensions. When we say we want to know our sales dollars, we almost always mean by day, or by quarter, or by year.

In designing a star schema, our first order of business is usually to determine what we want to see (our measures) and how we want to see it (our dimensions). Mapping Dimensions into Tables When we start building dimension tables, there are a few rules to keep in mind. 1) all dimension tables should have a single-field primary key. This key is often just an identity column, consisting of an automatically incrementing number. The value of the primary key is meaningless; our information is stored in the other fields. These other fields contain the full descriptions of what we are after. For example, if we have a Product dimension (which is common) we have fields in it that contain the description, the category name, the sub-category name, etc. These fields do not contain codes that link us to other tables. Because the fields are the full descriptions, the dimension tables are often fat; they contain many large fields Dimension tables are often short, however. Comparison of Dimension and fact size We may have many products, but even so, the dimension table cannot compare in size to a normal fact table. For example, even if we have 30,000 products in our product table, we may track sales for these products each day for several years. Assuming we actually only sell 3,000 products in any given day, if we track these sales each day for ten years, we end up with this equation: 3,000 products sold X 365 day/year * 10 years equals almost 11,000,000 records! Therefore, in relative terms, a dimension table with 30,000 records will be short compared to the fact table. Given that a dimension table is fat, it may be tempting to denormalize the dimension table. Resist the urge to do so; we will see why in a little while when we talk about the snowflake schema. Dimensional Hierarchies Hierarchical structures in an OLAP system are different because the hierarchy for the dimension is actually all stored in the dimension table.

Notice that both Category and Subcategory are stored in the table and not linked in through joined tables that store the hierarchy information. This hierarchy allows us to perform drill-down functions on the data. We can perform a query that performs sums by category. We can then drill-down into that category by calculating sums for the subcategories for that category. We can calculate the sums for the individual products in a particular subcategory. The actual sums we are calculating are based on numbers stored in the fact table. We will examine the fact table in more detail later.

Consolidated Dimensional Hierarchies (Star Schemas)

The above example (Figure 7) shows a hierarchy in a dimension table. This is how the dimension tables are built in a star schema; the hierarchies are contained in the individual dimension tables. No additional tables are needed to hold hierarchical information. Storing the hierarchy in a dimension table allows for the easiest browsing of our dimensional data. In the above example, we could easily choose a category and then list all of that categorys subcategories. We would drill-down into the data by choosing an individual subcategory from within the same table. There is no need to join to an external table for any of the hierarchical informaion. In this overly-simplified example, we have two dimension tables joined to the fact table. We will examine the fact table later. For now, we will assume the fact table has only one number: SalesDollars.

Figure 8

In order to see the total sales for a particular month for a particular category, our SQL would look something like this:
SELECT Sum(SalesFact.SalesDollars) AS SumOfSalesDollars FROM TimeDimension INNER JOIN (ProductDimension INNER JOIN SalesFact ON ProductDimension.ProductID = SalesFact.ProductID) ON TimeDimension.TimeID = SalesFact.TimeID WHERE ProductDimension.Category=Brass Goods AND TimeDimension.Month=3 AND TimeDimension.Year=1999

To drill down to a subcategory, we would merely change the statement to look like this:
SELECT Sum(SalesFact.SalesDollars) AS SumOfSalesDollars

FROM TimeDimension INNER JOIN (ProductDimension INNER JOIN SalesFact ON ProductDimension.ProductID = SalesFact.ProductID) ON TimeDimension.TimeID = SalesFact.TimeID WHERE ProductDimension.SubCategory=Widgets AND TimeDimension.Month=3 AND TimeDimension.Year=1999

Snowflake Schemas
Sometimes, the dimension tables have the hierarchies broken out into separate tables. This is a more normalized structure, but leads to more difficult queries and slower response times. Figure 9 represents the beginning of the snowflake process. The category hierarchy is being broken out of the ProductDimension table. You can see that this structure increases the number of joins and can slow queries. Since the purpose of our OLAP system is to speed queries, snowflaking is usually not something we want to do. Some people try to normalize the dimension tables to save space. However, in the overall scheme of the data warehouse, the dimension tables usually only hold about 1% of the records. Therefore, any space savings from normalizing, or snowflaking, are negligible.

Figure 9

Building the Fact Table

The Fact Table holds our measures, or facts. The measures are numeric and additive across some or all of the dimensions. For example, sales are numeric and we can look at total sales for a product, or category, and we can look at total sales by any time period. The sales figures are valid no matter how we slice the data. While the dimension tables are short and fat, the fact tables are generally long and skinny. They are long because they can hold the number of records represented by the product of the counts in all the dimension tables. For example, take the following simplified star schema:

Figure 10

In this schema, we have product, time and store dimensions. If we assume we have ten years of daily data, 200 stores, and we sell 500 products, we have a potential of 365,000,000 records (3650 days * 200 stores * 500 products). As you can see, this makes the fact table long. The fact table is skinny because of the fields it holds. The primary key is made up of foreign keys that have migrated from the dimension tables. These fields are just some sort of numeric value. In addition, our measures are also numeric. Therefore, the size of each record is generally much smaller than those in our dimension tables. However, we have many, many more records in our fact table.
Fact Granularity

One of the most important decisions in building a star schema is the granularity of the fact table. The granularity, or frequency, of the data is usually determined by the time dimension. For example, you may want to only store weekly or monthly totals. The lower the granularity, the more records you will have in the fact table. The granularity also determines how far you can drill down without returning to the base, transactionlevel data. Many OLAP systems have a daily grain to them. The lower the grain, the more records that we have in the fact table. However, we must also make sure that the grain is low enough to support our decision support needs.

----------------------------------------------------------------------------------------------------

One of the major benefits of the star schema is that the low-level transactions are summarized to the fact table grain. This greatly speeds the queries we perform as part of our decision support. This aggregation is the heart of our OLAP system. ----------------------------------------------------------------------------------------Fact Table Size
We have already seen how 500 products sold in 200 stores and tracked for 10 years could produce 365,000,000 records in a fact table with a daily grain. This, however, is the maximum size for the table. Most of the time, we do not have this many records in the table. One of the things we do not want to do is store zero values. So, if a product did not sell at a particular store for a particular day, we would not store a zero value. We only store the records that have a value. Therefore, our fact table is often sparsely populated. Even though the fact table is sparsely populated, it still holds the vast majority of the records in our database and is responsible for almost all of our disk space used. The lower our granularity, the larger the fact table. You can see from the previous example that moving from a daily to weekly grain would reduce our potential number of records to only slightly more than 52,000,000 records. The data types for the fields in the fact table do help keep it as small as possible. In most fact tables, all of the fields are numeric, which can require less storage space than the long descriptions we find in the dimension tables. Finally, be aware that each added dimension can greatly increase the size of our fact table. If we added one dimension to the previous example that included 20 possible values, our potential number of records would reach 7.3 billion.

Changing Attributes
One of the greatest challenges in a star schema is the problem of changing attributes. As an example, we will use the simplified star schema in Figure 10. In the StoreDimension table, we have each store being in a particular region, territory, and zone. Some companies realign their sales regions, territories, and zones occasionally to reflect changing business conditions. However, if we simply go in and update the table, and then try to look at historical sales for a region, the numbers will not be accurate. By simply updating the region for a store, our total sales for that region will not be historically accurate. In some cases, we do not care. In fact, we want to see what the sales would have been had this store been in that other region in prior years. More often, however, we do not want to change the historical data. In this case, we may need to create a new record for the store. This new record contains the new region, but leaves the old store record, and therefore the old regional sales data, intact. This approach, however, prevents us from comparing this stores current sales to its historical sales unless we keep track of its previous StoreID. This can require an extra field called PreviousStoreID or something similar.

There are no right and wrong answers. Each case will require a different solution to handle changing attributes.

Aggregations
Finally, we need to discuss how to handle aggregations. The data in the fact table is already aggregated to the fact tables grain.\ However, we often want to aggregate to a higher level. For example, we may want to sum sales to a monthly or quarterly number. In addition, we may be looking for total just for a product or a category. These numbers must be calculated on the fly using a standard SQL statement. This calculation takes time, and therefore some people will want to decrease the time required to retrieve higher-level aggregations. Some people store higher-level aggregations in the database by pre-calculating them and storing them in the database. This requires that the lowest-level records have special values put in them. For example, a TimeDimension record that actually holds weekly totals might have a 9 in the DayOfWeek field to indicate that this particular record holds the total for the week. This approach has been used in the past, but better alternatives exist. These alternatives usually consist of building a cube structure to hold pre-calculated values. We will examine Microsofts OLAP Services, a tool designed to build cube structures to speed our access to warehouse data.