Data Lineage and Impact Analysis

As both a BI Consumer and Author, I have always felt the need to know the origin of the data getting displayed on the
reports. Having worked on quite a few BI Apps implementations in the last few years, the joy I felt on hearing about
the new Data Lineage and Impact Analysis functionality provided by Oracle as part of OBIA 11.1.1.8.1 knew no
bounds!
Traditionally BI implementers have had to in a way fight for scraps of information pieces contained within multiple
applications or documents. This would then be loosely coupled together to cater to a quick fit-gap requirements
exercise or in other cases a more detailed impact analysis job.
With the introduction of these new features, Oracle has ensured a highly integrated and transparent environment
providing metadata to put together metadata from the data warehouse, reporting and configuration layers and
creating a holistic and end-to-end view of the BI Apps dataset.
How Data Lineage Works in BI Apps
In BI Apps data traverses through multiple layers before actually reaching the dashboards and reports. From a
reporting perspective the physical model of the data warehouse is mapped to a business model which forms the
logical or semantic layer. This in turn is mapped to the OBIEE presentation tables and columns which are used for
the dashboard and report creation. The warehouse itself is loaded in two stages, first using source specific data
mappings (SDEs) to extract data from source to staging tables and then using source-independent mappings (SILs)
which are used to populate the final target data warehouse tables from these staging tables.
With ODI used to extract metadata from the BI Apps Configuration Manager and its own tables, Oracle has integrated
this quite seamlessly and combined the OBIEE metadata through use of RPD and catalog metadata documentation.
An ODI load plan then is executed to combine the data contained in these metadata repositories and stored in
warehouse tables which are then used for the OOTB Data Lineage dashboards and can also be used to create one's
own data lineage and impact analysis reports.
Benefits and Improvement Areas
Once the initial setups and configurations are done and the data lineage tables have been loaded, the OOTB model
itself provides enough metadata to cover all the lineage and impact analysis reports that might be required. As is
evident, this reduces analysis effort drastically and also reduces dependence of any kind on the ETL team to trace
back columns. Anyone and everyone is now enabled to look at and understand where their data is coming from! The
metadata provided covers all the main stages in the data load process and also extends to OTBI if that is being used
along with BI Apps.
Even though we are still in the early days of exploration, there are a few areas which we are yet to discover. There
doesn't seem to be any support for formulas applied in analytics and temp interfaces do not seem to be covered.
Data lineage doesn't seem to work if views/procedures are being used in the ODI interfaces. Additionally any
customization done in interfaces doesn't seem to be captured. We are in the midst of taking a deep dive on these and
will post an update ASAP!
The Final Verdict
The Data Lineage functionality provided is definitely a good start and the belief is it would continually evolve to
provide an even more integrated, robust and automated solution to information discovery within BI Apps.