Evaluating Quality of Web Services: A Risk-driven Approach

Natallia Kokash Vincenzo DAndrea

26/04/2007

BIS'07 Poznan, Poland

Introduction
Service-centric systems Quality of Service (QoS) Issues Risk-driven service selection
Risk analysis SOA risks Failure risk QoS-driven service selection

Experimental results Conclusions and Future Work References

26/04/2007

Risk management for SOA

BIS'07 Poznan, Poland

Service-centric systems
Partners + Provide r s0 s1 s2 + s3

s4

+ + s5

Sequential operator s1 ; s2 Parallel operator

s 1 | s2

| + si

Choice operator
Web service
Client
26/04/2007

Start state

End state
BIS'07 Poznan, Poland

s 1 + s2 si t0 t

Quality of Service Issues

QoS for web services:
Domain-independent Domain-dependent
Throughput, capacity, latency, response time (duration), availability, reliability, reputation, execution cost (price) Currency converters: accuracy Hotel booking: prices, number of the rooms, availability rate

How to:
1. 2. 3. 4. 5. 6. 7.

specify QoS? measure QoS? specify user requirements and/or preferences about QoS? match user requirements with existing services in terms of QoS? rank services according to user preferences? predict QoS factors under certain environmental conditions? choose web services to guarantee certain QoS level of their composition?

26/04/2007

BIS'07 Poznan, Poland

QoS-driven service selection

Problems in quality-driven service selection:
Lack of QoS statistics Volatility of QoS factors Multidimensionality Subjectivity Context-dependence

Approaches

Multi-attribute optimization [Ardagna and Pernici 2005, Zeng et al. 2004, Yu et al. 2005 ] Constraints satisfaction [Martin-Diaz et al. 2005] Genetic algorithms [Canfora et al. 2006] Fuzzy [Lin et al. 2005]

Problems with existing approaches

Simplified models (e.g., one service for one task) Dependences among QoS factors are ignored Context is not taken into account
BIS'07 Poznan, Poland
5

26/04/2007

Risk analysis
Example:
Movie: title= Rainmaker, format=DVD, languages=Italian, English Convert DVD to AVI: language=English SimpleDivX converter: time=2 hours, language = Italian Impact on time: 2 hours are lost Reason: Unexpected service behaviour (discrepancy with specification)

Requires assessment of inherently uncertain events and circumstances Two dimensions:

how likely the uncertainty is to occur (probability) what the effect would be if it happened (impact)
BIS'07 Poznan, Poland

26/04/2007

SOA Risks
Threats
Loss of service, data, users Unexpected service behavior, changes Performance problems Contract violation

Assessment

Mitigation

Likelihoods and implications of threats Analysis of user expectations Service testing User feedback, reputation systems

Service selection, redundancy, redesign Runtime monitoring Contracts and policies

BIS'07 Poznan, Poland
7

26/04/2007

Risk management for SOA

Information gathering
(service discovery, QoS data)

Analysis of system artifacts Redesign loop

(service testing, conformance evaluation)

Evolutional loop

System configuring
(service selection, adaptation, composition,...)

Risk identification
(Business risks, technical risks)

History

Risk analysis

Risk prioritization Risk mitigation

(policies, SLAs,...)
26/04/2007

No

Decision making
(are risks acceptable?)

Yes

Fix configuration Define controls

BIS'07 Poznan, Poland

Risk-driven service selection

Choose the composition that maximizes the expected profit:

Loss function defines the cost of service failure (money, time, resources)

26/04/2007

BIS'07 Poznan, Poland

Failure risk
probability that some fault occurs resulting impact of this fault on the composite service

where

is the probability of the service failure.

Loss function includes:

Expenses to invoke failed service (its cost and response time) Service failure can cause rollback of the transaction, therefore expenses to execute precedent services are also included The provider may have to pay penalty to a user whose request was not accomplished.
BIS'07 Poznan, Poland
10

26/04/2007

Failure risk of service compositions

b-g + g-t

+
b-e

g-e

+ + e-t

b-g + b-e

g-t g-e

+
+

e-t

26/04/2007

BIS'07 Poznan, Poland

11

Failure risk: examples

Success rate = 0.5; execution cost = 1; penalty = 2
s1 + s2 s1 s3 + s1 s3 s1 + s1

1.75
+
s2 s4 s2 s3 s2

1.375 1.5625 1.625

s2

+
26/04/2007

s3

+
BIS'07 Poznan, Poland

s4

1.25
12

Risk-driven selection algorithm

Select an execution path with minimum risk value Notation:
c composition q(si) quality parameter (response time, execution cost) p(si) probability of success qmax resource limit

Objective function:

where

26/04/2007

BIS'07 Poznan, Poland

13

Experimental results (1)

Goal: Compare QoS of compositions chosen by our algorithm with QoS of compositions chosen by other methods Zeng et al. [2004]
QoS factors: price, duration, reputation, success rate, availability Objective function: linear combination of scaled QoS factors Scaling: QoS factors range from 0 to 1 Weights reflect user preferences
26/04/2007 BIS'07 Poznan, Poland
14

Experimental results (2)

100 simulated service compositions 10 services in each composition

26/04/2007

BIS'07 Poznan, Poland

15

Conclusions and Future work

A novel risk-based method for assessing QoS of web services is proposed Real world case studies Comparative analysis of existing service selection algorithms Risk management framework for automatic web service compositions Questions?
26/04/2007 BIS'07 Poznan, Poland
16

References
1. 2. 3. 4. 5. 6. 7. 8. [Ardagna and Pernici 2005] Ardagna, D., Pernici, B.: Global and Local QoS Constraints Guarantee in Web Service Selection, IEEE International Conference on Web Services, 2005, pp. 805806. [Canfora et al. 2006] Canfora, G., di Penta, M., Esposito, R., Villani, M.-L.: QoS-Aware Replanning of Composite Web Services, Proceedings of the International Conference on Web Services, 2005. [Claro et al. 2005] Claro, D., Albers, P., Hao, J-K.: Selecting Web Services for Optimal Composition, Proceedings of the ICWS 2005 Second International Workshop on Semantic and Dynamic Web Processes, 2005, pp. 32-45. [Gao et al. 2006] Gao, A., Yang, D., Tang, Sh., Zhang, M.: QoS-driven Web Service Composition with Inter Service Conflicts, APWeb: 8th Asia-Pacific Web Conference, 2006, pp. 121 132. [Lin et al. 2005] Lin, M., Xie, J., Guo, H., Wang, H.: Solving QoS-driven Web Service Dynamic Composition as Fuzzy Constraint Satisfaction, IEEE International Conference on e-Technology, e-Commerce and e-Service, 2005, pp. 9-14. [Martin-Diaz et al. 2005] Martin-Diaz, O., Ruize-Cortes, A., Duran, A., Muller, C.: An Approach to Temporal-Aware Procurement of Web Services, International Conference on Service-Oriented Computing, 2005, pp. 170184. [Zeng et al. 2004] Zeng, L., Benatallah, B., et al.: QoS-aware Middleware for Web Services Composition, IEEE Transactions on Software Engineering, Vol. 30, No. 5, 2004, pp. 311327. [Yu et al. 2005] Yu, T., Lin, K.J.: Service Selection Algorithms for Composing Complex Services with Multiple QoS Constraints, International Conference on Service-Oriented Computing, 2005, pp. 130143.
BIS'07 Poznan, Poland
17

26/04/2007