You are on page 1of 97

OBIEE11g Architecture, Components & Internals

Mark Rittman, Rittman Mead BI Forum 2011


Brighton & Atlanta, May 2011

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Agenda

• Overview of OBIEE 11gR1 (and what’s new in 11.1.1.5)


• High-Level Product Architecture, and Role within Oracle Fusion Middleware 11g
• The Oracle BI Domain, WebLogic Server, and the Oracle BI Instance
• Use of Oracle Fusion Middleware and Oracle Database
• Filesystem and Environment Changes
• What do WebLogic and Enterprise Manager Do?
• How do Diagnostics and Systems Metrics work with 11g?
• How does the Systems API work?
• Clustering & Multi-Tenancy

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Oracle Business Intelligence 11gR1

• The latest release of Oracle’s business intelligence platform


• Derived from Siebel Analytics; second major Oracle release
• Now at version 11.1.1.3 (hopefully 11.1.1.5 by the time of the BI Forum)
• 11gR1 new features focused on manageability and middleware integration

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Key New Features introduced with OBIEE 11gR1

• New Fusion Applications / ADF look and feel for front-end components
• WebLogic Server and Oracle Enterprise Manager
• Some enhancements to the Oracle BI Repository
• Action Framework
• KPIs and Scorecards

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


OBIEE 11g Basic Architecture

• Overall system called an


“Oracle BI Domain”
• Made up of Java and non-Java
components
• Java components hosted in
WebLogic Server 11g
• Managed using OEM and
WebLogic Admin Server
• Supporting database schema
for repositories, metadata
• New in 11.1.1.5 - Managed
Server and Node Manager
are optional

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Based on Fusion Middleware 11g Platform

• Fully-integrated with Oracle Fusion Middleware 11g


• Common security (Oracle Identity Management, Oracle Access Manager)
• Built on WebLogic Server
• Managed through Fusion Middleware
Control (Enterprise Manager)
• Built using multiple technologies
including JDeveloper (ADF)

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Leveraging Oracle Fusion Middleware Technologies

• OBIEE 11g leverages many Fusion Middleware 11g technologies


‣ WLS (WebLogic Server)
‣ OPMN (Oracle Process Manager and Notification Server)
‣ OPSS (Oracle Platform Security Services)
‣ DMS (Dynamic Monitoring System)
‣ ODL (Oracle Diagnostic Logging)
‣ JPS (Java Platform Security)
‣ CSF (Credential Store Framework)
‣ OPATCH / OPACK
‣ UA (Upgrade Assistant)
‣ RCU (Repository Creation Assistant)

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


OBIEE 11g System Components, Java Components and OPMN

• OBIEE components divided into System and Java components


• System components are still C/C++ executables, are controlled by OPMN,
and are managed by Fusion Middleware Control
• Java Components are JEE applications, are usually installed in the
managed server, and are controlled
by Fusion Middleware Control

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Oracle WebLogic Server 11g

• J2EE Application Server used for hosting OBIEE 11g Java components
‣ Plays role performed by OC4J/OAS in 10g version
• Managed using the WebLogic Administration Server
‣ Weblogic Server Administration Console
‣ Fusion Middleware Control
‣ JMX MBeans (programmatic
access to BI Domain)
• OBIEE Java components held in
(n) Managed Servers
• System Components Managed by
Fusion Middleware Control,
but exist outside of WLS domain

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Node Manager

• Used for stopping and starting distributed Managed Servers


• Java utility that runs as a separate process from WebLogic Server
• Can also restart managed servers that have crashed
• The equivalent of OPMN, but for Java components
• Usually must be running as a service for full start of OBIEE 11gR1
• New in 11.1.1.5 - only installed when managed server installed

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Oracle Process Manager and Notification Server

• Used for distributed process


start/stop/restart
• Process recovery (optional),
process “ping”
• Managed via EM, or from
command line
• Can also be used for
performance collection (DMS)
• Required on every
OBIEE server C:\Middleware\Oracle_BI1\opmn\bin>opmnctl status
• Standard FMW process Processes in Instance: instance1

control tool ---------------------------------+--------------------+---------+---------


ias-component | process-type | pid | status
---------------------------------+--------------------+---------+---------
coreapplication_obiccs1 | OracleBIClusterCo~ | 3364 | Alive
coreapplication_obisch1 | OracleBIScheduler~ | 1012 | Alive
coreapplication_obijh1 | OracleBIJavaHostC~ | 4400 | Alive
coreapplication_obips1 | OracleBIPresentat~ | 2092 | Alive
coreapplication_obis1 | OracleBIServerCom~ | 5040 | Alive

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The WebLogic Domain

• Each OBIEE 11g Oracle BI Domain contains a single WebLogic Domain


• Contains one Admin Server, and at the start,
one Managed Server
‣ New in 11.1.1.5 - Managed
Server is optional
‣ Can add managed servers
through Horizontal Scaleout
• WebLogic Domain administered
through Admin Console,
and WLST (Weblogic scripting)
• Managed servers can be
started manually, or through
Node Manager

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The Oracle Instance

• A collection of System Components, spread over one or more physical servers


• Managed together using OPMN (instances are OPMN organisational units)
• Starts with one of each system
component, on the initial server
‣ Can be added to by use
of Vertical Clustering

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Oracle Enterprise Manager Fusion Middleware Control

• Manage System Components (BI Server, BI Presentation Server etc)


• Start, Stop and Restart all System Components and Managed Servers
• Configure Preferences and Defaults
• Scale out System Components
• Performance Monitoring
and Diagnostics

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


11g Architecture Compared to 10g Architecture

• Main difference is wrapping components in WLS platform and EM management


• Individual servers, components are still the same (BI Server, Presentation Server,
BI Publisher J2EE application etc)
• Some reworking has taken place in 11gR1 (unified logging, management of
config files using EM etc)
• Basic concepts, plus clustering (OBIEE), scheduling etc are the same as in 10g
Web Server SAW Bridge (J2EE/ISAPI) Web Browser
(IIS, Tomcat, Websphere, iPlanet) XML, HTML,
Javascript
XLS, PDF, TXT
for Usability & Interactivity
Oracle BI Publisher Oracle BI Presentation Services TCP/IP (SSL) over
HTTP/HTTPS
Delivery Server SOAP Web Services, XML and URL Interface Web
Catalog
Layout Interfaces Service
Oracle Interactive Oracle Answers
Dashboards
Data Logic HTML, SOAP
XSL XML
User Profiling, Security and Session Mngmt over
Framework
HTTP/HTTPS
Cache Services (Web) & Connection Mngmt External Applications and Portals

ODBC over TCP/IP (SSL)

Oracle Delivers Server Oracle BI Server


TCP/IP
Scheduling/Event Services (SSL) Logical SQL ODBC/JDBC (Logical Business Model)

vs
Agent Execution Logic Load Balancer
Metadata Interchange

Device Adaptive Content Session Management


Intelligent Request Generation
Oracle BI Administration
.
Logical Request Generation
Oracle BI Cache
Cluster Controller Navigator
Security Services

Services
Query Govern.

Metadata Management Services


Multi-Pass /
Sub-Request Logic Multi-User Development Services
System / Perf Monitoring

Fragmentation
Externalized Optimization Metadata Documentation Services
Authentication
Aggregate Server Management Services
LDAP
Navigator
DB Authentication
Optimized Query Rewrites
Custom Authenticator
Execution Engine

Data Source Adapters

ODBC, CLI, OCI, XML, MDX

Analytical and Operational Data Sources

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Demonstration #1
WebLogic Server, Fusion Middleware and Enterprise Manager

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Core OBIEE Components Still Play Same Role

• Once you “strip away” all of the middleware


components, OBIEE 11g still works in
fundamentally the same way as OBIEE 10g
‣ Users queries via the Presentation Server
‣ The Oracle BI Server converts these
queries to physical SQL/MDX, via the
Oracle BI Repository
‣ Queries are passed to the underlying
physical databases and OLAP cubes
‣ Data returned to users in the form of
dashboards and reports

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Oracle BI Server Still Works as the “Analytic Middleware”

• Parses incoming logical SQL queries


• Uses the navigator to generate optimized
physical queries
• Manages the query cache
• Handles row and subject-area security
• Provides load-balancing information
to the Cluster Controller
• But now delegates authorization/
authentication to Fusion Middleware

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


OBIEE 11g Security and Oracle Fusion Middleware 11g

• OBIEE 11g delegates security to Oracle Fusion Middleware 11g


‣ Leverages Oracle Platform Security Services
• Users and Groups in RPD now moved to
embedded WLS LDAP Server
‣ RPD and Webcat groups replaced by Applications

FMW11g Application Roles


• Comprehensive SSL and Credentials Management Middleware

• Encrypted RPD, plus optional report encryption


and watermarking Database
• Flexible authorization model through
WLS and OPSS Infrastructure &
Management
• Still backwards compatible with
LDAP model in OBIEE 10g

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


OBIEE 11g Security Administration Tools

• WebLogic Server Admin Server (LDAP Server, Security Providers)


• Fusion Middleware Control (Application Roles)
• BI Administration tool (subject-area, and row-level security)
• Catalog Manager, and Presentation Services Catalog View (object permissions)
• Presentation Services Administration Page (PS functional permissions)

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


WLS Embedded LDAP Server

• By default, OBIEE 11g users and groups are now held in the WLS LDAP Server
‣ More robust directory for storing user details
‣ Recommended for >1000 users
• WLS Admin Server Console now used for
creating and maintaining users
• BI Server “outsources” all authentication,
authorization to FMW11g
• BI Administration tool now used for
subject area and row-level security,
connection pool passwords only
• WLS LDAP Server can be “swapped out”
for alternative directories (MS AD etc)

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Application Roles and Application Policies

• Application roles introduce an indirection between LDAP groups and BI groups


‣ LDAP server provides users, groups
‣ FMW11g provides application roles
‣ Application roles are granted to
LDAP users, groups
‣ Permissions are assigned to

Application Roles / Policies


application roles
• Breaks direct link between groups
Oracle
and roles Business Roles Groups
• Application roles can be exported Intelligence
Users Users
between FMW11g environments
• All RPD, webcat permissions secured
against application roles

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


OBIEE Security Providers

• OBIEE 11g (through FMW11g) uses three Security Providers


‣ Authentication Provider (defaults to WLS LDAP Server)
‣ Policy Store Provider (defaults to WLS)
‣ Credential Store Provider (defaults to WLS)
• Flexible security framework that allows easy linking to external security
• All enabled through Oracle Platform Security Services

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Oracle Platform Security Services

• Standards-based, portable, integrated enterprise-grade security framework


• Underlying security platform that provides security to Fusion Middleware 11g
• Abstraction layer in the form of API that insulate applications from security
infrastructure

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Demonstration #2
Oracle Platform Security Services

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


So What’s New in OBIEE 11.1.1.5?

• Simple install option installs Java components in WLS administration server,


doing away with node manager and managed server
• 64-bit Linux install lays down the 64-bit JVM
• 64-bit Windows install lays down a working 64-bit BI Administration tool (sort of)
• Option to install BI Composer, a simplified analysis generation tool
• Support for Oracle OLAP and
SAP/BW as data sources
• In-place and out-of-place
upgrades from 11.1.1.3

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Upgrading from 11.1.1.3 to 11.1.1.5

• To options for upgrading; “out-of-place” and “in-place” upgrade


• Out of Place Upgrade involves installing 11.1.1.5 in a new MW Home
‣ Manually migrate RPD, catalog and other BI objects
‣ Manually migrate security, roles, users to new environment
‣ Advantage : conceptually simple, but resource-intensive
• In-Place Upgrade involves following the FMW11g patching process
‣ Detailed at http://www.rittmanmead.com/2011/05/in-place-upgrading-of-
obiee-11-1-1-3-to-11-1-1-5/
‣ Upgrade WLS from 10.3.3 to 10.3.5 via metalink-only Upgrade Installer
‣ Do software-only install of OBIEE 11.1.1.5 into existing MW Home
‣ Perform pre- and post-patching steps
‣ Advantage : upgrades in-place, but a non-trivial exercise

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Common OBIEE 11g Architecture & Internals Questions

• What is the new directory structure for an OBIEE installation (and why are there
so many duplicated directories?)
• What does Oracle WebLogic Server actually do?
• What does Oracle Enterprise Manager Do?
‣ And What Happens When I Press the “Activate” Button?
• How Does Logging Now Work in OBIEE 11g? What is ODL and DMS?
• Where Does EM and WLS Keep Its Settings? What “Secret” Files Are There?
• So How Does the Systems Management API Work?
• What’s the Difference Between WebLogic and OBIEE Clustering?
• Can You Run Multiple RPDs and Webcats on the Same OBIEE11g Box?
• How Has Change Management Changed?

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


OBIEE 11g Directory Structure

• Middleware Home is the top-level


installation directory
• Binaries and template
files in the BI Oracle Home
• Configuration files (RPD,
config files, diagnostics,
WebCat) in the
Instance Home
• WLS binaries in the WL Home
• WLS domain files
in BI Weblogic Domain Home
• Separation of binaries
from configuration/instance
files

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Why So Many Duplicated Directories?

• Throughout the /instances/instancen


sub-directory structure, there are lots of
duplicated directories for BI Server, PS etc
• These are for the System Components
‣ One per system component
‣ Divides config files, repositories etc
into subfolders, one for each component
• Found under the following /instancen folders
‣ /bifoundation : for RPD, webcat etc
‣ /config : for NQSConfig.INI etc
‣ /diagnostics : for logs etc
‣ /tmp : for logs etc

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Demonstration #3
Filesystem Locations for Key OBIEE11g Files

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


What Does Oracle WebLogic Server Do?

• Oracle WebLogic Server, along with Oracle Fusion Middleware 11g, provides the
following key areas of functionality
‣ Hosts the Oracle BI Analytics plugin (as per OC4J and IIS in OBIEE 10g)
‣ Hosts the other java components
‣ Provides the host platform for
Oracle Fusion Middleware 11g
‣ Provides the LDAP server, and other
security “stores” for the OPSS security
providers
‣ Hosts the WLS Admin Server and
EM Fusion Middleware Control

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


What Does Oracle Enterprise Manager Do?

• Enterprise Manager uses the OBIEE 11g Systems Management API to perform
administration tasks
‣ You can also use this API through WLST scripting, or through Java code
• Uses a set of JMX MBeans (Java Management Extensions) to perform tasks
‣ MBeans are Managed Beans, Java utilities
that have properties and methods
‣ Every EM task has a corresponding
MBean method
‣ Some MBeans have yet to be
exposed through EM

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The MBean Browser

• The list of OBI 11g administration MBeans can be viewed in the MBean Browser
• Using EM, right-click on the AdminServer, then select
farm_bifoundation_domain > Weblogic Domain
> bifoundation_domain > MBean Browser
• OBI11g entries under several headings:
‣ oracle.biee.admin
‣ oracle.biee.local
‣ oracle.as.management.mbeans.opmn

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The MBean Browser

• The list of OBI 11g administration MBeans can be viewed in the MBean Browser
• Using EM, right-click on the AdminServer, then select
farm_bifoundation_domain > Weblogic Domain
> bifoundation_domain > MBean Browser
• OBI11g entries under several headings:
‣ oracle.biee.admin
‣ oracle.biee.local
‣ oracle.as.management.mbeans.opmn

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The Oracle BI Systems Management API

• A collection of MBeans used for providing systems management functions for


OBIEE
• A subset of the wider range of MBeans in Fusion Middleware 11g
• Key Systems Management API MBeans include
‣ ServerConfigurationMBean : upload and register new repositories
‣ BIDomainMBean : lock, commit and rollback changes to OBIEE config
‣ BIInstanceMBean : Start, stop and restart components
‣ BILogConfigurationMBean, EmailConfigurationMBean etc

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


What Happens When You Press “Activate”?

• So what happens, for example, when you press Apply and Activate within EM
‣ For example : Uploading a new Oracle BI Repository (RPD file) online
• What MBeans are called? What files are updated?
• And can we then script this?

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


EM, Beans & RPD Uploads : Step 1

• User presses Lock and Edit Configuration button within EM


Deployments > Repository page
• This operation uses the
oracle.biee.admin:type=BIDomain,group=Service MBean

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


EM, Beans & RPD Uploads : Step 1

• User presses Lock and Edit Configuration button within EM


Deployments > Repository page
• This operation uses the
oracle.biee.admin:type=BIDomain,group=Service MBean

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The oracle.biee.admin:type=BIDomain MBean

• Main MBean type for managing Oracle BI Domain configuration state


• Methods for locking, rolling back, committing changes to the domain
• lock method is called to perform the Lock and Edit Configuration step

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The oracle.biee.admin:type=BIDomain MBean

• Main MBean type for managing Oracle BI Domain configuration state


• Methods for locking, rolling back, committing changes to the domain
• lock method is called to perform the Lock and Edit Configuration step

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


EM, Beans & RPD Uploads : Step 2

• Within EM, enter the name and path to the RPD file, and the password
• Then press the Apply button to save the changes
• This operation uses the
oracle.biee.admin:type=BIDomain.BIInstance.ServerConfiguration,biInstanc
e=coreapplication, group=Service MBean

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


EM, Beans & RPD Uploads : Step 2

• Within EM, enter the name and path to the RPD file, and the password
• Then press the Apply button to save the changes
• This operation uses the
oracle.biee.admin:type=BIDomain.BIInstance.ServerConfiguration,biInstanc
e=coreapplication, group=Service MBean

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The BIDomain.BIInstance.ServerConfiguration MBean Type

• Another MBean type within the oracle.biee.admin MBean


• Used for managing the server configuration (i.e. uploading an RPD)
• uploadRepository method is called to record name and path to RPD

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The BIDomain.BIInstance.ServerConfiguration MBean Type

• Another MBean type within the oracle.biee.admin MBean


• Used for managing the server configuration (i.e. uploading an RPD)
• uploadRepository method is called to record name and path to RPD

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Generating the RPD File Name and Sequence Number

• OBIEE 11g adds a sequence number to each RPD, to provide “versioning”


• Obtained from the DataZipFileVersion attribute from the
oracle.biee.local:Location=bi_server1,type=DomainConfigProxy,
group=Config MBean]
• RPD file properties are then available in the ServerConfiguration MBean

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Generating the RPD File Name and Sequence Number

• OBIEE 11g adds a sequence number to each RPD, to provide “versioning”


• Obtained from the DataZipFileVersion attribute from the
oracle.biee.local:Location=bi_server1,type=DomainConfigProxy,
group=Config MBean]
• RPD file properties are then available in the ServerConfiguration MBean

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


EM, Beans & RPD Uploads : Step 3

• In EM, you would then press the Activate button


• This leads to the configuration files being updated, and the RPD uploaded
• This operation uses the
oracle.biee.admin:type=BIDomain,group=Service MBean again

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Activating the Change

• Activate causes the commit method to be called with the


oracle.biee.admin:type=BIDomain,group=Service MBean
• A constant value (for example, ERROR) has to be passed to specify the degree to
which errors will be allowed

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Activating the Change

• Activate causes the commit method to be called with the


oracle.biee.admin:type=BIDomain,group=Service MBean
• A constant value (for example, ERROR) has to be passed to specify the degree to
which errors will be allowed

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Updated Configuration Files, and File Transfer

• Pressing Activate / calling the commit method will trigger several actions
‣ Updating the biee-domain.xml central configuration file
‣ Updating the NQSConfig.INI Oracle BI Server configuration file
‣ Transferring the RPD via the biee-data.zip archive file
‣ Copying the RPD password to the credential store

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


RPD Password Stored in the Credential Store

• All repository passwords (that have been taken online) are also stored in the
Credential Store through the Activate / commit method
• Required so that the BI Server can get access to the RPD contents

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


EM, Beans & RPD Uploads : Step 4

• After the changes have been made, you need to restart the Oracle BI Server
‣ System components need to be restarted to pick up configuration changes
• Actioned through Overview or Capacity Management > Availability tabs in EM

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The OPMN MBean

• The oracle.as.management.mbeans.opmn:Location=AdminServer,
name=instance1,type=opmn MBean has methods for OPMN
‣ Stop, Start, Restart; Create, delete component; Get status
‣ Can run for single component, component group or all components

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The OPMN MBean

• The oracle.as.management.mbeans.opmn:Location=AdminServer,
name=instance1,type=opmn MBean has methods for OPMN
‣ Stop, Start, Restart; Create, delete component; Get status
‣ Can run for single component, component group or all components

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


So Can This All Be Scripted?

• All calls to MBeans can be scripted through WLST


• Command-line tool using Jython as the scripting language

connect("weblogic","welcome1","localhost:7001")
domainCustom()
cd ('oracle.biee.admin')
cd ('oracle.biee.admin:type=BIDomain,group=Service')
biinstances = get('BIInstances')
biinstance = biinstances[0]

cd ('..')
cd ('oracle.biee.admin:type=BIDomain,group=Service')
objs = jarray.array([],java.lang.Object)
strs = jarray.array([],java.lang.String)
invoke('lock',objs,strs)
cd('..')

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


So Can This All Be Scripted?

• All calls to MBeans can be scripted through WLST


• Command-line tool using Jython as the scripting language

connect("weblogic","welcome1","localhost:7001")
domainCustom()
cd ('oracle.biee.admin')
cd ('oracle.biee.admin:type=BIDomain,group=Service')
biinstances = get('BIInstances')
biinstance = biinstances[0]

cd ('..') Lock method


cd ('oracle.biee.admin:type=BIDomain,group=Service')
objs = jarray.array([],java.lang.Object)
strs = jarray.array([],java.lang.String) • Create lock on Oracle BI
invoke('lock',objs,strs) Domain, prior to configuration
cd('..') change

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


WLST Script Continued

cd (biinstance.toString())
biserver = get('ServerConfiguration')
cd('..')
cd(biserver.toString())
ls()
argtypes = jarray.array
(['java.lang.String','java.lang.String'],java.lang.String)
argvalues = jarray.array(['C:/SampleAppLite.rpd','Admin123'],java.lang.Object)
invoke('uploadRepository',argvalues,argtypes)

cd('..')
cd('oracle.biee.admin:type=BIDomain,group=Service')
objs = jarray.array([],java.lang.Object)
strs = jarray.array([],java.lang.String)
invoke('commit',objs,strs)

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


WLST Script Continued

cd (biinstance.toString())
biserver = get('ServerConfiguration') uploadRepository Method
cd('..')
cd(biserver.toString())
ls() • Obtain next RPD sequence no.
argtypes = jarray.array • Copy the RPD name and
(['java.lang.String','java.lang.String'],java.lang.String)
password into the MBean
argvalues = jarray.array(['C:/SampleAppLite.rpd','Admin123'],java.lang.Object)
properties
invoke('uploadRepository',argvalues,argtypes)

cd('..')
cd('oracle.biee.admin:type=BIDomain,group=Service')
objs = jarray.array([],java.lang.Object)
strs = jarray.array([],java.lang.String)
invoke('commit',objs,strs)

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


WLST Script Continued

cd (biinstance.toString())
biserver = get('ServerConfiguration')
cd('..')
cd(biserver.toString())
ls()
argtypes = jarray.array
(['java.lang.String','java.lang.String'],java.lang.String)
argvalues = jarray.array(['C:/SampleAppLite.rpd','Admin123'],java.lang.Object)
invoke('uploadRepository',argvalues,argtypes)

cd('..')
cd('oracle.biee.admin:type=BIDomain,group=Service')
objs = jarray.array([],java.lang.Object)
strs = jarray.array([],java.lang.String)
invoke('commit',objs,strs)

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


WLST Script Continued

cd (biinstance.toString())
biserver = get('ServerConfiguration')
cd('..')
cd(biserver.toString())
ls()
argtypes = jarray.array
(['java.lang.String','java.lang.String'],java.lang.String)
argvalues = jarray.array(['C:/SampleAppLite.rpd','Admin123'],java.lang.Object)
invoke('uploadRepository',argvalues,argtypes)
commit Method
cd('..')
cd('oracle.biee.admin:type=BIDomain,group=Service')
objs = jarray.array([],java.lang.Object) • Activate configuration changes
strs = jarray.array([],java.lang.String) • Upload the RPD
invoke('commit',objs,strs) • Update config files
• Store password in Credstore

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Invoking an MBean using WLST

• All Simple BI WLST scripts have the same common structure


1. Connect to the WLS Admin Server (connect)
2. CD to the correct Systems Management API MBean
3. Prepare input array for invocation
4. Prepare output array for invocation
5. Call the MBean method
connect("weblogic","welcome1","localhost:7001")
domainCustom()
cd ('oracle.biee.admin')
cd ('oracle.biee.admin:type=BIDomain,group=Service')
biinstances = get('BIInstances')
biinstance = biinstances[0]

cd ('..')
cd ('oracle.biee.admin:type=BIDomain,group=Service')
objs = jarray.array([],java.lang.Object)
strs = jarray.array([],java.lang.String)
invoke('lock',objs,strs)
cd('..')

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Other Useful Uses of WLST Scripting

• Stopping, starting and restarting the BI Instance


• Stopping, starting and restarting individual components
• Changing the web catalog path
• Creating roles and mapping them to LDAP groups
• Enabling and disabling caching
• Setting other performance parameters
• Vertical and horizontal scaleout

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Demonstration #4
Scripting the Upload of an RPD using WLST

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


How Have Diagnostics and Logging Changed?

• OBI 11g uses DMS (Dynamic Monitoring Services) to record key system metrics
‣ Metrics and counters accessed via MBeans
• OBI 11g logging is still carried out to nqserver.log, nqquery.log
• Logs are now aggregated and parsed through ODL (Oracle Diagnostic Logging0
• Logs can now be viewed in aggregate using the Log Viewer
‣ Enterprise Manager > Capacity Management > Diagnostics

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The Log Viewer

• On the Capacity Management > Diagnostics page on EM


• Click on individual log name, or click Log Viewer to search all logs

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The Log Viewer

• On the Capacity Management > Diagnostics page on EM


• Click on individual log name, or click Log Viewer to search all logs

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The Log Viewer

• On the Capacity Management > Diagnostics page on EM


• Click on individual log name, or click Log Viewer to search all logs

Messages List

• Most recent errors or warnings


• Headline details

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The Log Viewer

• On the Capacity Management > Diagnostics page on EM


• Click on individual log name, or click Log Viewer to search all logs

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The Log Viewer

• On the Capacity Management > Diagnostics page on EM


• Click on individual log name, or click Log Viewer to search all logs

Error Details

• Error message
• Supplemental details
• ECID
• Component name

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The Log Viewer

• On the Capacity Management > Diagnostics page on EM


• Click on individual log name, or click Log Viewer to search all logs

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


The Log Viewer

• On the Capacity Management > Diagnostics page on EM


• Click on individual log name, or click Log Viewer to search all logs

Search Tools

• Search by name, ECID


• Headline details
• Message type
• Date range

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Adding / Removing Log Targets

• By default, individual log views focus on a single log


• You can broaden scope by adding / removing log targets
‣ ECID searches automatically include all log targets

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Adding / Removing Log Targets

• By default, individual log views focus on a single log


• You can broaden scope by adding / removing log targets
‣ ECID searches automatically include all log targets

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Execution Context IDs

• Every client “transaction” is assigned an Execution Context ID (ECID)


• Log entries are “stamped” with the ECID, to enable tracing across components
• Can be seen in plain-text in log files, or searched for using Log Viewer

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Execution Context IDs

• Every client “transaction” is assigned an Execution Context ID (ECID)


• Log entries are “stamped” with the ECID, to enable tracing across components
• Can be seen in plain-text in log files, or searched for using Log Viewer

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Log Viewer Example 1 : Analysis/Dashboard Error

• To take an example, consider a situation where a user reports a dashboard error


‣ “There’s a problem with the database”
• The only error they report is an ODBC error
• Your job is to now use the logs to diagnose the problem..

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Log Viewer Example 1 : Selecting the Server Log

• A database connection error will normally appear in the nqserver.log server


log file, but you may have many BI Servers spread over many nodes
• From EM, select Capacity Management > Diagnostics
• Click on the Server Log link

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Log Viewer Example 1 : Viewing Server Log, Click on ECID

• View the server log entries, and details, to diagnose problem


• Click on ECID link to see linked and related transactions

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Log Viewer Example 1 : Viewing Server Log, Click on ECID

• View the server log entries, and details, to diagnose problem


• Click on ECID link to see linked and related transactions

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Log Viewer Example 1 : Viewing Server Log, Click on ECID

• View the server log entries, and details, to diagnose problem


• Click on ECID link to see linked and related transactions

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Log Viewer Example 1 : Viewing Server Log, Click on ECID

• View the server log entries, and details, to diagnose problem


• Click on ECID link to see linked and related transactions

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Log Viewer Example 1 : Viewing Server Log, Click on ECID

• View the server log entries, and details, to diagnose problem


• Click on ECID link to see linked and related transactions

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Log Viewer Example 2 : Component Fails to Restart

• You have recently rebooted your OBIEE 11g server, or perhaps uploaded a new
RPD file, but the component fails to start
• What could be the reason?
• Luckily, there’s a View Log Messages button directly in the dialog...

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Log Viewer Example 2 : Log Viewer Opens for ECID

• Clicking on View Log Messages button opens the Log Viewer for this ECID
• All log file targets automatically included in search
• But none of the messages make any sense, and are mostly Java stack traces
‣ These are the various MBeans failing; however, they are not the root cause

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Log Viewer Example 2 : Log Viewer Opens for ECID

• Clicking on View Log Messages button opens the Log Viewer for this ECID
• All log file targets automatically included in search
• But none of the messages make any sense, and are mostly Java stack traces
‣ These are the various MBeans failing; however, they are not the root cause

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Log Viewer Example 2 : Need to Investigate Other Logs

• Root cause is actually in the Server Log, however this has not been tagged with
the ECID for this transaction (limitation in this version)
• Sometimes therefore need to look outside the ECID, especially for system
component failures that are known to be explained by their own logs
‣ MBeans are the process mechanism for EM, but action takes place in
individual system components which don’t always have ECIDs in their logs

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Log Viewer Example 2 : Need to Investigate Other Logs

• Root cause is actually in the Server Log, however this has not been tagged with
the ECID for this transaction (limitation in this version)
• Sometimes therefore need to look outside the ECID, especially for system
component failures that are known to be explained by their own logs
‣ MBeans are the process mechanism for EM, but action takes place in
individual system components which don’t always have ECIDs in their logs

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Demonstration #5
Diagnostics and Logging

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Clustering in OBIEE 11g

• OBIEE 11g Domains can be scaled-out (horizonal clustering)


and/or scaled-up (vertical clustering)
• Adds resilience, reduces single points of failure
• Allows OBIEE systems to take advantage of more/larger hardware
• Installer sets up single-node OBIEE 11g Domain; cluster can then be extended
out and upwards using EM and Universal Installer

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Vertical Scaleout of System Components

• Spare capacity on an individual host can be used by adding additional


system components
• Can add additional BI Servers, Presentation Servers and Java Hosts
• Useful for resilience, and usage of spare capacity
‣ Doesn’t protect against the whole server failing though

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Horizontal Scaleout of Java Components

• Used for adding additional managed servers (Java components) and


system components to an existing cluster
• Used for scalability and failover
• Set up via the Universal Installer > Scale Out BI System option

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


WebLogic Clustering vs. OBIEE Clustering

• Vertical clustering uses just OBIEE clustering features (Cluster Controller)


‣ Cluster controller routes queries to clustered BI Server
‣ Presentation Server, Cluster Controller, Java Host and Scheduler can also be
clustered (active/active and active/passive)
• Horizontal clustering uses WebLogic clustering
‣ Uses MBean method to pack/unpack domain and clone to separate server
‣ Does not itself use/require OBIEE clustering
‣ Does require WebLogic Server EE license

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


So ... Can We Run Multiple RPDs and Webcats on a Single Box?

• No (or at least, we’ve not got it working)


• Internal restrictions within OBIEE 11.1.1.3 that stops complete isolation of RPD /
webcat pairs
• Possibly coming (“Multi-Tenancy”) in OBIEE 11.1.1.5/6?
• Otherwise, need to multiple Middleware Home installations

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


Summary

• OBIEE11g has had a major architectural overhaul, adopting FMW11g technologies


• Core servers stay the same, but new middleware “wrapper”
• Uses WebLogic for J2EE application hosting, security, integration
• Major security enhancements and changes
• Enterprise Manager used for systems managemnet
• WLST scripting can be used for process automation
• EM and WLST both leverage MBeans for modularized administration
• Clustering now extends to WebLogic clustering for horizontal scale-out
• Diagnostics leverage MBeans and ODL for cross-node diagnostics and metrics

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


More Information

• Thank you for attending this presentation


• More information can be found at http://www.rittmanmead.com
• Contact us at info@rittmanmead.com or mark.rittman@rittmanmead.com
• Look out for our book, “Oracle Business Intelligence Developers Guide” due Q1 2012
• Follow-us on Twitter (@rittmanmead) or Facebook (facebook.com/rittmanmead)

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011


OBIEE11g Architecture, Components & Internals
Mark Rittman, Rittman Mead BI Forum 2011
Brighton & Atlanta, May 2011

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Wednesday, 25 May 2011

You might also like