SOA Performance

Performance Testing SOA Applications

 Agenda

» What is SOA?
» Service Level Agreement
» Testing For SLA
» Key To Performance Testing
» SOA Testing Complexities
» Overcoming Challenges
» Methodology
» Approach
» Case Study – Online Trading Application
» Web Services
» Questions & Answers
 What is SOA?
» Service Oriented Architecture
» Architectural paradigm (pattern/model)
» Variety of heterogeneous systems (dissimilar)
» Different locations and owners
» Web Services?

» SOA provides benefits in four basic categories:

» Reduces expensive integration
» Allows for more asset reuse
» Increased business agility
» Reduces business risk

» These four core benefits offer returns at many different levels and
parts of the organization, depending on the set of business problems
to which the company is applying SOA.
 Service Level Agreements (SLA) for SOA

» A SLA is a formal contract between a service provider and a

consumer
» Service availability
» Performance
» Traffic levels
» Messages / queries per hour / minute / second
» Response time Missed SLA
» Rejected transactions
» Errors
» …

Poor Response Time

Noncompliance with industry

and government regulations
 Testing For SLA
» The only real way to verify the capabilities of a system is to conduct
a series of testing efforts
» Load Testing (up to defined SLA transactions per second or other)
» Stress Testing (up to service failure)
» Volume Testing (introducing large amounts of data into system)
» Reliability Testing (high levels of load over long periods of time)
» Performance engineering (achieve the expected level of performance)
» Capacity planning (plan for the forecasted growth)
 Key to Performance Testing
» The keys to successful performance testing in general require:
» Understand the application and the infrastructure
» Understand the consumer of the application
» Generate accurate anticipated volumes of traffic
» Investigate the impact of the traffic on the application and systems under
test
» Performance testing SOA applications requires the same level of
understanding, testing, and analysis.
» Service level testing
» SOA and SME expertise for testing
» Tool strategy
 SOA Testing Complexities
» SOA Adds Complexity to Performance Testing
» Wide range of technologies
» Detailed planning
» Multiple owners – internal and external
» Many different applications and usages
» Heterogeneous hardware/infrastructure
» Knowledge of the application and the technologies
» Replicating traffic patterns
» Availability of information, knowledge
» Environment availability
 Overcoming Challenges
» To simplify performance testing for SOA
applications break them down into the smallest
“logical” components possible:
» Individual Service
» Systems
» Databases
» Technology
» Protocols
» Messaging
» Business process/functionality
» Define:
» Application architecture
» Infrastructure architecture
» Data architecture
» Business Process
 Methodology
Setup & Discovery Plan Design Execute, Analyze &
Report
Activities
Project management / Change management / Reporting

• Requirement gathering • Identify work load • Setup test environment • Execute performance
through questionnaire • Freeze test plan • Setup master data test scripts
• Requirements review • Identify test scenarios • Create performance test • Collect performance
• Application walk through • Identify sleep time scripts metrics
• Establish performance testing • Project approach • Quality control of test • Analyze performance
team • Governance plan scripts metrics
• Project governance • Risk and mitigation plan • Populate performance
• Communication plan test report
• Performance quality risk
analysis
• Discover performance
characteristics of the application
• Testability aspects of the tool
• Establish performance goals

Project management / Change management / Reporting

Deliverables
Performance Performance Performance • Execution results of
requirements test plan test scripts the scripts and metrics
 Approach
» Step 1: Define an objective
» Specific areas we are trying to address
» Plans of adding new services or users
» Plans of moving to or adding geographical areas
» Expected SLAs

» Step 2: Initiate Discovery

» Detailed architecture
» Application
» Network
» Infrastructure
» Business
» Existing information on previous tests – baseline information
» Validate the understanding of the application, services, vendors etc
» Fill the gaps – interview users, process flows
 Approach (continued)
» Step 3: Plan
» Testing plan based on the objectives
» Testing infrastructure and availability of systems
» Components broken by – application, service, business process
» Identify the tools that are apt for the needs
» Identify the metrics to be captured

» Step 4: Execute
» Run tests at varying loads
» Capture the metrics

» Step 5: Analyze & Report

» Analyze results
» Gap analysis – plan vs. actual
» Plan next iteration
 Case Study – Online Trading Application
» The case study involves a leading financial institution with an
existing online trading service.

» Performance Issues
» User response times to be lower than 8 seconds
» Users complain about connections being dropped
» Number of completed transactions
» Support 10,000 concurrent users

» Engagement Expectations
» Planning for growth
» Better user experience
Transactions Break-up
Trading Platform – Business Overview
 Case Study - Architecture

Users Load Balancer FONS

E

FONSE Broadcast
FONSE Interactive
Web Server Trading Server

BSE

App Server
BSE Streamer
BSE Order Server

Gateway Messaging Server

NSE

NSE Broadcast
DB NSE Interactive
 Sequence Diagram
User Web Application Server Gateway 3rd Party
Trade Request
Trade Placed Trade Details Trade Settlement
Process

Personal info and loan request

Trade details

Trade forwarding information

Request - price

Request - quantity

Request – market data

Return market data

Return stock fulfilment

Response
 Execution Life Cycle
Generated Load As per SLA
Scalability Issues Identified
Configuration changes effected
Subsequent Tests
Load Scaling successful
Response Time

Database Issues Identified

Database tuned
Re-tests
Orders per second improved
Adapter limitation identified
More Adapter services added
Orders per second increased
SLA Compliance
Results - Buy Transactions
Results – Sell Transactions
 Key Findings
» Load balancer configuration - prevented the load from being
distributed across the web server farm
» Database configuration – there were quite a few full table scans that
made the queries run very long and translated to extensive disk I/O
» Order per second – could not scale due to the program not able to
pump the orders to the matching engine
» Scalability – the current infrastructure could only support 40% of the
expected user loads
Web Services Architecture

Source – w3.org
Web Services Stack

Source - The Web Services Protocol Stack , Lawrence Wilkes

Q&A
 Thank You

Rajesh Patil
rajesh.patil@applabs.com