2009 International Conference on Computational Intelligence and Security
Web Service Community Discovery Based on Spectrum Clustering
Xizhe Zhang, Ying Yin, Mingwei Zhang, Bin Zhang
College of Information Science and Engineering
Northeastern University
Shenyang, China
zhangxizhe@ise.neu.edu.cn
Abstract—More and more web services are emerging in the
Internet with development of service computing over the
past decade. The web service in SOA system naturally
forms into some service community during execution
process. Mining and analysis web service community will
help design SOA system and support web service
application effectively. This article addresses the problem
of how to discovering web services community formed by
closely interactive web services. We propose a novel
approach which construct web service execution network
from logs and clustering it using spectrum clustering.
Generally, the web services belong to same cluster have
strong relative to same task object and we call it web
service community. The approach has been implemented in
an experience system for web services dynamic composition
and mining.
Keywords-web services; community mining; spectrum
clustering
I. INTRODUCTION
With the development of web services technology,
Service Oriented Computing (in short SOC) has become the
mainstream of distribution computing. Researchers have
made a great deal of research on service composition,
service selection, service substitution and other area [1-3].
Compared with other distribution software development
environment, SOC has obviously more openness and
dynamic, which are embodied by dynamic variability of
network and execution environment and state of web service,
business requirement from different user (composition logic,
service quality, etc.) , dynamic variability of service space
(emergence of new service, evolvement of existing service,
etc.). In general, the lifecycle for a composition service is
not momentary (maybe a few hours, days or longer), so the
web services selected by method, which is only aiming at
selecting period and user requirement, cannot meet the
requirement of dynamic service computing environment.
This paper is focused on the mining problem for web
service community discovery based on web service logs. It
can be defined that how to mining composition pattern from
service log in dynamic service composition environment.
These service communities acquired from service usage log
can help the users to understand the service behavior
978-0-7695-3931-7/09 $26.00 © 2009 IEEE 187
DOI 10.1109/CIS.2009.162
Authorized licensed use limited to: Dalhousie University. Downloaded on March 04,2022 at 18:31:31 UTC from IEEE Xplore. Restrictions apply.
effectively, as well as guarantee the usability and
compatibility for services interaction.
Mining web services can help the user to better
understand service behavior and use services properly.
Dong [4] describes an algorithm for supporting
similarity search of web services by clustering names of
parameters of web service operations. Zheng [5] proposes a
service mining framework for exploring interesting
compositions of existing Web services. A few researches [6]
perform mining web service logs, but only directly apply
process mining and sequence mining algorithm and no
alteration aiming at service characteristic. 
The domains related to web service log mining are web
usage mining [7], process mining [8]. Web usage mining is
to analyze user behavior by mining visit logs, and its
purpose is to provide customized service or improve the site
structure. Process mining is to find some heuristic rules by
mining the sequence relations of the events in log, and then
summarize process model from these rules. But the data
disposed by process mining are static, and can hardly meet
the requirement of service dynamic variability.
Mining community of web service will facing the
following problem:
1. How to collect web service usage log effectively? The
self-government and distribution of web services cause
difficulty in log collecting. An effective service log
collecting mechanism is necessary for development of
service mining.
2. The collection of web services is dynamic and the
service behavior is variable by time. It is necessary to
determine the size of time horizon, so that the mining result
is workable.
3. How to design the mining algorithm according to
dynamic of service space, so that it is suitable to the
dynamic evolvement of web services, and guarantee the
mining result is proper and effective.
A basic idea for web service composition mining is that
the services related to certain business topic must have some
direct or indirect connection between them. These
connection represent in the service usage log is that the
service tends to forming “community”, in which the web
services interact each other with high frequency. Based on
this idea, we can cluster the service collection into several
communities by their interaction relationship. These web
service communities represent some business topic.
This paper proposes a novel web service community
discovery algorithm based on spectrum clustering. By
analyzing web service usage log, we construct the web
service interaction network and discovery the closely related
service community by spectral clustering algorithm. These
naturally forming web service communities represent the
corresponding business topic, which can provide guidance
for users on service behavior understanding, service
selection, service substitution, etc.
The remainder of this paper is organized as follow: In
Sec.2, we present the structure and collection mechanism of
web service usage log; In Sec.3, we describe the algorithm
for web service community mining; In Sec.4, we discuss the
validity of the proposed approach. Finally, we provide
concluding remark in Sec.5.
II. WEB SERVICE LOG
As the web logs are recorded by the web server that web
services reside on, which is lack of web service execution
information, so it cannot meet the requirement of service
mining. We propose a web service log collecting
architecture, which records service execution information by
web service composition execution engine.
Fig.1 shows a web service composition framework which
supporting web service logging. The framework logs web
service execution information by service composition
execution engine. The execution engine is response for
execute business process and exchange message between
the web service and record log simultaneity.
The log records the interaction information between
execution engine and web service. SOAP message is the
TABLE I.
# PID ws
1 TravelProcess FlightTicket
2 TravelProcess FlightTicket
3 TravelProcess WS-Pay
4 Agent4BuyBook001 WS-Order
5 TravelProcess WS-Pay
6 Agent4BuyBook001 WS-Order
III. MINING WEB SERVICE COMMUNITY
A. Pre-processing web services log
Web service is highly dynamic in service computing
environment. Service can be canceled or modified, and new
service is continuously created. The Internet network and
service execution environment also keep changing.
Therefore web services and their behavior are changing
accordingly. How to effectively find the dynamic variability
of web services is the key point of web service mining.
The dynamicity of web service is an important feature to
service mining. Required by continuously changing log, a
proper time horizon must be determined before mining to
Authorized licensed use limited to: Dalhousie University. Downloaded on March 04,2022 at 18:31:31 UTC from IEEE Xplore. Restrictions apply.
physical basic unit of interaction information and
information transferring between engine and service is the
minimum logic unit.
Figure 1. A composite service log architecture
The service usage log can be defined as message record
during a certain period of service execution. The service
execution log is denote as L={l1 ,l2, …, ln, …}, among which
li is called as log item. It can be described as l = {PID, ws,
time, size, category }, among which:
PID represents the ID of business model which current
log item belong; it is generated by composite service
execution engine.
ws is the web service which current log item executed;
time is the record time of the log item;
size is the message size;
Category indicate the role of execution engine in the
service interaction; its value is {Sender, receiver}, it means
the execution engine is the message sender or message
receiver.
The sample of web service log is showed in Table.1.
SAMPLE OF WEB SERVICE LOG
time size catagory
02/Jan/2007:13:08:21-0700 1410 Sender
02/Jan/2007:13:09:35-0700 2050 Reciever
02/Jan/2007:13:11:02-0700 8140 Sender
02/Jan/2007:13:08:21-0700 1020 Sender
02/Jan/2007:13:08:21-0700 3250 Reciever
02/Jan/2007:13:08:21-0700 2010 Reciever
ensure the result is updated and valid. A proper time horizon
is the key point to mining quality.
Time horizon h is the history of a log at the length of h.
When time horizon h and cluster number k are defined,
service community discovery is to find k service
communities in h time horizon of the log. The detailed
process is as follows. Firstly, figure out the proper time
horizon [tc, tc-h] according to the evolvement speed of
service log, among which tc is present time. Secondly,
construct corresponding service interaction network
according to time horizon, and calculate the similarity
between the nodes, as well as clustering the network, and
finally gain the service community.
188
 When service is changing in high speed, the time
horizon shall be shorten accordingly; while be increased
when service is changing in low speed. So, the size of time
horizon can be determined by service evolvement speed.
For the service space S, let the total log time horizon is
[tstart, tend], let the web service set is (op1,op2,…,opn). We
first cut the time axis into several interval T1,T2,…,Tn
according to the natural time unit such as hour, day, week,
etc. let e ( Ti ) = ( p1 , ", pn ) be the service indicate vector in
the time interval Ti, we have:
⎧1 if opi is executed in Ti
pj = ⎨ (1)
⎩0 otherwise
The difference Diff (Ti , Ti +1 ) between two adjacent time
intervals is defined as follow:
e(Ti ) ⋅ e(Ti +1 )
Diff (Ti , Ti +1 ) = 1 − (2)
e(Ti ) × e(Ti +1 )
The difference Diff (Ti , Ti +1 ) represents the variety degree
of the two adjacent time interval. We can use this
measurement to segment the time axis into several horizons
that the service changes relatively slow.
The segmentation method is similar to hierarchical
clustering. First we segment the time axis into several
intervals, and then compute the difference for the adjacent
interval; Select the two intervals which have smallest
difference and combine them into one interval, and compute
the difference between the new interval and its adjacent
interval; repeat this process until the whole horizon has
combined.
Based on above method, we have the segmentation result
for the web service log. We can select the mining time
horizon which satisfies the difference threshold from
segmentation result. The time horizon selected by above
method can reflect the variability degree of web service in
the service environment. Selecting the log sequence
evolvement relatively slow will guarantee the validity of the
community discovery result.
B. Web Services network clustering
We can represent the interaction behavior of service web
service as a directed multi-graph, called web service
interaction network, denote as G(S,E). It figures the
interaction relationship between web services of the service
environment in a certain time horizon. The web service set
is denoted as S and the edge set E represent the web service
interaction relationship, the direction of edge represent the
time sequence in the interaction.
The web service interaction network is directed multi-
graph for web service is repeatedly executed commonly. So
we can discover the web service community by the density
of interaction relationship. The web service community is a
group in which the web services have tightly interaction.
From the business view, the web service group often relative
to the same business goal. So we can get the web service
composition model by finding web service community.
189
Authorized licensed use limited to: Dalhousie University. Downloaded on March 04,2022 at 18:31:31 UTC from IEEE Xplore. Restrictions apply.
The web service network contains rich information about
service behavior. How to measure the distance between
nodes is an important factor to service community discovery.
An obvious way is to define the edge weight according to
interaction times among nodes. But the outcome is not as
good as expected because of dynamic character of log data.
It is embodied as follows:
1. The importance of the log data is weakening with time,
more weight should be given to new data, so that the
clustering result can inflect recent service behavior pattern
better.
2. In the service environment, lifecycle for different
business process is different. If a certain business exists for
a long time, it is usually executed for more times, while new
business process is executed less times. The internal
interaction density for all the transactions is not even.
So, we should take time element into consideration when
designing weight. For an operation interaction opi → op j , the
mining time horizon is [tstart, tend], the definition for weight
may be given as follows:
op j .t − tstart
wi (opi → op j ) = (3)
tend − tstart
The weight of operation node can by computed by
combine all the edges weight. Let there are n times
interaction between operation op1, op2, which are e1,e2,...
en，the weight of operation op1,op2 is:
n
∑ w (e )
i i
w(op1 , op2 ) = i =1
(4)
max ( ei .time ) − min ( ei .time )
Equation (4) compute the weight by using interact
frequency of the operation, which make the operation group
that execute less time can be found.
For example, let operation op1,op2 interact 200 time in a
month, and op3,op4 interact 30 time in three days, the weight
of single edge is set to 1. The weight computes by execute
time will be:
w(op1 , op2 ) =200; w(op3 , op4 ) =50;
And by frequency, we have:
w(op1 , op2 ) =6.66; w(op3 , op4 ) =20。
This guarantee the new operation group which executed
less time but with high frequency will be founded.
We use the spectral clustering algorithm [9] to find
operation group. Compared to the traditional algorithms
such as k-means or single linkage, results obtained by
spectral clustering very often outperform the traditional
approaches, spectral clustering is very simple to implement
and reasonably fast (for sparse data sets up to several
thousands).
Spectral clustering algorithms have many variations. We
take Ratio Cut method to clustering operation interaction
graph. The parameter is service usage log L and clusters
number k. The similarity matrix W is denoted as follow:
 ⎡ w11 w1n ⎤ existing statistics data.
⎢ " ⎥ We give the measurements for service composition
W =⎢ ⎥ (5)
⎢ " ⎥ model, which includes:
⎢ ⎥ 1. Response time. It is an important measure for evaluate
w
⎣ n1 wnn ⎦
the performance of composite service, which can be
Wij = w(op1 , op2 ) is the interaction weight for operation calculated by service execution records in the log. To sum
opi and opj, which can be computed by equation(4). Fig.2 the each service response time in service composition model,
shows the overview of clustering algorithm. we will get the measurement of response time for service
Input：Service usage log L， Clusters number k
composition model.
Output：Clustering result G’
2. Robustness. It is the measurement for success
execution rate of service request. As the service function
Algorithm Spectral Clustering ( L，k )
1. Construction operation interaction graph based on and usability is variable with time and environment, service
service usage log L; failure may take place during execution. The robustness of
2. Compute similarity matrix W of operation interaction service composition model can be defined as follows:
graph G； ( )
R = E fail / E fail + Esuccess (8)
3. Compute Laplacians matrix L=D-W,
n
Efail is the failure time of service request in service
where D = diag ( d1 , d 2 ,", d n ) , di = ∑ wij ; composition model, and Esuccess is the success time of service
j =1 request.
4. Compute the eigenvector v1 , v2 , " , vk corresponding to
the eigenvalue from the first smallest to the kth smallest,
let V = [v1 , v2 ," , vk ] , so V ∈ R n×k . Let V = [ y1 , y2 ," , yn ]T ,
and we get a k-dimension set V ' = { y1 , y2 , ", yn } ;
5. For the data point y1 , y2 ," , yn ，clustering them into
k clusters by K-Means algorithm.

Figure 2. The spectral clustering algorithm for service network

IV. EXPERIMENT Figure 3. Compariation for robustness between mining result and
real composite services
Based on above research, we design a web service
mining system to test the validity of above algorithm. This
system creates web service usage log by simulating service
execution environment. On the basis of these logs, we are
able to evaluate the usability of web service community
discovery algorithm. The composition model construct by
service community can be evaluated by reliability of service
and response time.
First, we create network topology graph by Internet
topology generator [10]. The network delay among nodes is
calculated by the shortest route algorithm. And then the
Figure 4. Compariation for response time between mining result
service structure is built. Each service contains service name,
and real composite services
service type, parameter and other information. Then we
select the actual business process to build web service Fig.3 and Fig.4 provides the comparison of response
composition model. Finally, the simulating execution based time and robustness between mining result and real
on above environment is carried out and the service usage composite services. We can see that the response time and
log is generated. robustness of service composition mined by log is in better
We generate 1500 web service, which belong to 200 level than that of real service composition. This is because
service types, and random distributed in 800 network nodes; that after a certain time of execution, the real service
50 service composition models are created; the total time composition process may be lagging behind the service
horizon for service log is 30 weeks. As the business process dynamic variability, which will result in its quality decrease.
and web service in real service space is highly variable, we
set random existing periods to service composition model V. CONCLUSION
and web service, so that new model and service will The service log contains plenty of information about
continuously enter into, while the old model and service exit.
service usage pattern, the quality and performance of service
The degree of variability of web service can be decided by

190

Authorized licensed use limited to: Dalhousie University. Downloaded on March 04,2022 at 18:31:31 UTC from IEEE Xplore. Restrictions apply.
application may be improved by mining service log. This
paper proposes a web service community discovery
algorithm based on Spectrum Clustering. By analyzing web
service execution log, we construct the web service
interaction network, and find out the closely related service
groups with same usage pattern by spectral clustering
algorithm. These naturally forming service groups represent
the corresponding business process, which can provide
guidance for users on service behavior understanding,
service selection, service substitution, etc.

ACKNOWLEDGMENT
This work was supported by a grant from National
Natural Science Foundation of China (No. 60903009),
Postdoctoral Science Foundation of China (No.
20080430185), National High Technology Research and
Development Program of China (863 Program) (No.
2009AA012122).
REFERENCES
[1] G. Zheng, A. Bouguettaya. A Web Service Mining Framework. IEEE
International Conference on Web Services (ICWS 2007), 1096-1103.
[2] Mohsen J. Asbagh, Hassan Abolhassani. Web service usage
mining: mining for executable sequences. Proceedings of the 7th
Conference on 7th WSEAS International Conference on Applied
Computer Science, Venice, Italy, 266-271.
[3] Robert Gombotz, Schahram Dustdar. On Web Services Workflow
Mining. BPM 2005 Workshops, LNCS 3812, 216–228, 2006.
[4] Xin Dong, Alon Halevy, Jayant Madhavan, Ema Nemes, Jun Zhang.
Similarity Search for Web Services. Proceedings of the 30th VLDB
Conference, Toronto, Canada, 2004, 372-383.
[5] G. Zheng, A. Bouguettaya. A Web Service Mining Framework. IEEE
International Conference on Web Services (ICWS 2007), 1096-1103.
[6] Mohsen J. Asbagh, Hassan Abolhassani. Web service usage
mining: mining for executable sequences. Proceedings of the 7th
Conference on 7th WSEAS International Conference on Applied
Computer Science, Venice, Italy, 266-271.
[7] J. Srivastava, R. Cooley, M. Deshpande, P-T. Tan. Web Usage
Mining: Discovery and Applications of Usage Patterns from Web
Data. ACM SIGKDD Explorations, (1) 2, January 2000.
[8] van der Aalst W., Weijters T., Maruster L. Workflow mining:
discovering process models from event logs. IEEE Transactions on
Knowledge and Data Engineering, 16(9), 2004, 1128 – 1142.
[9] Ulrike Luxburg. A Tutorial on Spectral Clustering. Statistics and
Computing, 17(4), 2007, 395-416.
[10] Medina A, Lakhina A, Matta I, Byers JW. BRITE: An approach to
universal topology generation. In: Proc. of the 9th Int’l Workshop on
Modeling, Analysis, and Simulation of Computer and
Telecommunication Systems (MASCOTS 2001).Cincinnati, 2001.
346−356.

191

Authorized licensed use limited to: Dalhousie University. Downloaded on March 04,2022 at 18:31:31 UTC from IEEE Xplore. Restrictions apply.