VtCIMS 2003 . lntemational Symposium on Vinual Envirom,cnts. Hum-Computer Intrrfaco. and Mrarurrmcnt Syuems Lqann. Switzrrland.

27-29 July 2003

Modelling E-Commerce Systems Quality with Belief Networks

Antonia Stefani School of Science and Technology Hellenic open University 16 Sachtnuri Str.,Patra, GR26222, Greece Phone: +30 2610 362586
Fax: +30 2610 361410

Michalis Xenos School of Science and Technology Hellenic Open University 16 Sachtnuri Str., Patra, GR26222, Greece Phone: +30 2610 361485
Fax: +30 2610 362349

Dimitris Stavrinoudis Computer Engineering and Informatics Department Patras University,

Rion, GR26500, Greece

Phone: +30 2610 362556

Fax:+302610361410

E-mail: stefani@eap.gr

E-mail: xenos@eap.gr

E-mail: stavrino@ceid.upatms.gr

-A -This pnper focuses on qunlily aspects o e - c o m r c e f systems nnd proposes n method for modelling such system based on Belief Networks. The pnper discusses fhe Iheordicnl f bnckground o the proposed model, as wellprneriecll issues nrising f from its npplicntion. The basic norotion and concept o Belief Networks is briem presenle4 while enphnsis is plnced on lhe models stmdure nnd its usage. The presented model cnn be utilisedfor assessing Ihe qunli& o e-commerce syslem, as well as f for aiding in qunlily nssurnnce during the design nnd development f phnse o such syslem

This paper proposes a model based on Bayesian Networks that can be used for assessing the quality of E-commerce systems, as well as defming specific quality requirements during the design process of E-commerce systems. Section 2 discusses quality issues that relate to E-commerce systems characteristics and that formulate the models background. In section 3 the notation and formation of Belief Networks are described, while section 4 discusses the shucture and performance of the proposed model. Section 5 presents the models use and applicability. Finally, in section 6 conclusions and future work are discussed.
11. THE MODELS THEORETICAL BACKGROUND

1. INTRODUCTION

E-commerce is a constantly expanding field. This fact is confirmed by the increasing number of enterprises that invest into the creation of e-commerce systems and the continuous expansion of economic and commercial transactions through the Internet. E-commerce can be defmed as follows [I]: sharing business information, maintaining business relationships and conducting business transactions by the means of telecommunication networks. Depending on the type of transactions performed electronically, there are two basic categories of e-commerce systems [2]: Business to Consumer (B2C) and Business-to-Business (B2B). In E-commerce systems, interaction with the end-user is conducted through web-based applications including a human-computer interface. Since all user-system communication is realized based on such interface, it is self evident that the quality of an E-commerce system is directly related to the quality of the human-computer interface through which the end-user interacts with the web-based applications. Usually, end-users value E-commerce systems that are flexible, usable, easily adaptable to their needs and that offer a full range of applications. But how can one evaluate E-commerce systems and define the extent to which they meet end-users requirements? To this end, it is necessary to define what constitutes a high-quality Ecommerce system as well as a methodology for evaluating the quality of E-commerce systems [3].

Most E-commerce systems seek to provide high quality services to the end-users, i.e. the clients, and to this end they include specific applications (modules) so as to meet specific end-user requirements. Examples of such requirements are searching capabilities, flexible navigation or the ability to group goods and applications like search engines, site maps and shopping carts have been developed in order to meet such requirements. Even if the type of applications that an Ecommerce system integrates changes in the future, the user requirements relating to the E-commerce system will remain unchanged. It is thus reasonable to conclude that the quality and evaluation methods of E-commerce systems will always be dependant on the quality of similar applications and their ability to meet end-user requirements. Such quality factors should be taken under serious consideration during the development of E-commerce systems. Past approaches about the quality of E-commerce systems are emphasizing on usability standards using techniques like feature inspection methods and collecting data about endusers opinion by questionnaires. These methods provide an important feedback to the researcher and their results can be utilised as a useful background for future work, however, they do not contribute directly to a dynamic model. On the contrary, the importance of the proposed model lies on its dynamic character. In the proposed model the results derived from its application are utilized for the models constant

Corresponding author: Michalis Xenos is IEEE Member since 1998 0-7803-7785-0/03/$17.00 0 0 3 IEEE 02

.
13

improvement, thus contributing to a continuous evolvement and upgrading. The proposed model is based on the I S 0 9126 quality standard [4]. Specifically, it relies on the set of those quality characteristics and sub-characteristics that are directly related to quality as perceived by the end-users. These quality characteristics are: functionality, usability, reliability and efficiency. The importance of each of the above mentioned quality characteristics depends on each E-commerce systems specificities as well as the user requirements and developer priorities for the specific system. It should be mentioned that the development of the proposed model was mainly based on Business to Consumer (B2C) systems. Ill. BELIEF NETWORKS The proposed model is based on the notation and formation of Causal Probabilistic Networks, also called Belief Networks (BN) and Bayesian Networks [5, 61. The mathematic model on which Bayesian Networks are based is the theorem developed by the mathematician and theologian Thomas Bayes. The BN are a special category of graphic models where nodes represent variables and the directed arrows the relations between them. Therefore, a BN is a graphic network that describes the relations of probabilities between the variables 171.

in a Node Probability Table. This table presents the probability that a child node is assigned a certain value for each combination of possible values of the parent nodes. For example, figure 1 presents two parent nodes (nodes B and C) and one child node (node A). The probability table of node A reflects the probability P(AlB,C) for all possible combinations of A, B, C. Thus, since there are two possible states for node B (bl, b2) of figure 3, three possible states for node C ( e / , c2, c3) and three for node A (al,02, o ) then the 3, NPT of node A will include 3*2*3=18 elements.

IV. MODELS DESCRlPPlON

The philosophy underlying the proposed model is the creation of a dynamic network that concentrates and exploits the knowledge gained from the analysis of data gathered during previous researches and that can also use its own results for future estimations. A graphical presentation of the network is illustrated in Figure 2. The model uses nodes to represent the quality factors, characteristics and sub-characteristics of E-commerce systems. Each node is characterized by a set of possible states called evidence and is connected to i& parent nodes by directed arrows. In figure 2 the central node Quality appears in grey. This node represents the E-commerce system quality as a whole and is characterized by three possible states (evidence): good, average, and poor. The parent nodes of quality are the nodes: Functionality, Usability, Reliability and Efficiency, namely the quality factors that end-users value based on IS0 9126. These quality factors are marked with bold letters in the corresponding nodes of figure 2 and can also be characterized by three possible states: good, average, and poor. Each quality factor node is connected to the corresponding E-commerce systems quality characteristics, based on I S 0 9126, which in turn are assigned three possible states as evidence: good, average, poor. Finally, each of these quality characteristics is connected to a number of child nodes comprising the quality sub-characteristics of E-commerce systems. The evidence in all nodes simply answers the question posed to the user whether a specific characteristic or sub-characteristic exists in the E-commerce system or not. This is a way to minimize subjectivity at this level as much as possible. The model has been developed using the Microsoil 0 MSBNx Authoring and Evaluation tool version 1.4.2. An example of the tools user interface is shown in Figure 3. Each node of the model has a Node Probability Table that presents the discrete conditional probability distribution. This table presents the relations between this node (child node) and its parent nodes. For example, the quality subcharacteristic of Learnability is represented as a child node connected to two parent nodes, as indicated by the directed arrows. Each parent node represents the relevant e-commerce characteristics, namely: Easy help functions and Correct placement of tools. The Leamability node has three

Fig. 1. A simple Bayesian Network

The use of BN not only makes it possible to define the relation between the various nodes (variables), but also to estimate consistently the way in which the initial probabilities influence uncertain conclusions, such as the quality of an ecommerce system. In this case, BN are used for future estimation, or -as also c a l l e 6 forward prediction. Furthermore, BN can be used to speculate about the states of the initial nodes, based on a given final and snme intermediate variables. This is called backward assessment. In order to defme the relations between the variables, firstly the dependent probabilities that describe the relations between a child node and its parent nodes must be determined for each node. If the values of each variable are distinct, then the probabilities for each node can be described

14

possible states as evidence and the parent nodes have two states for evidence. The probability table for Leamability has therefore 3*2*2 = 12 elements. One of the most important factors affecting the successhl application of the model is the definition of the Node Probability Table of each

node. In the Node Probability Table of the quality subcharacteristic Leamability, which is presented in figure 3, the values of the probabilities vary between 0 to I, scaling hy 0.05. The probabilities of the model are based on data taken from previous studies of E-commerce systems [SI.

Fig. 2. Graphical presentation oflhe proposed model

Fig. 3. Example of a Belief Network

The user can insert data (evidence) for one or more nodes. This evidence can activate the conditional probabilities of other nodes and provide an estimation using bar charts. For instance, in figure 3 evidence has been inserted for the parent nodes of Leamability. The corresponding bar chart shows that there is 54% probability that the systems Leamability is good is 54%. Another advantage o f the model is that it can utilize the results from its applications in order to improve the accuracy of future measurements. Namely, the results are used for the improvement of the Node Probabilities Tables thus contributing to better accuracy.

V. APPLICATION OF THE MODEL

The application of the model is based on the input of evidence for some nodes. If no evidence is inserted by the user, the estimations provided by the model are based on previously collected experience of the system, as this has been inserted in the Node hobability Table. New evidence affects the probabilities of the other nodes and the estimation for each node is different. This is shown clearly in the bar charts. The model can be used forwards and backwards. Forward use can be utilized to assess the overall quality o f an Ecommerce system. In this case, the end-user inserts in each

node of the model the available evidence (measures) related to the E-commerce system. The model can then be used to provide an estimation about the systems quality and characterize it as good,average and poor also providing the corresponding probability values. It is worth mentioning that the model can provide estimations even if evidence has not been inserted in all of its nodes. Of course, more evidence inserted into the model improves the results accuracy. In a similar way it is possible to apply the model for obtaining results about only one of the quality factors or quality sub-characteristics. In this case, the estimations of the model can be utilized by E-commerce system developers to assess the importance of the quality sub-characteristics as well as the interaction level between parent nodes and child nodes. For instance, if the developer wants to assess the extent to which characteristics such as search engine, shopping cart, shopping list, altemative presentation methods and comparative presentation of the product affect the systems accuracy, as shown in Figure 4, it is possible to d o so by inserting various evidences for each of these characteristics. The different probability values obtained by the application of the model can assist the developer to conclude which system characteristics affect its accuracy more, and based on such conclusions to decide about the type and the number of applications to be developed.

16

Fig. 4. Exampb of forward use ofthe model

On the other hand, backward use of the model provides assessments regarding the intermediate nodes, when the value of the fmal state of quality is defined. For example, if it is known that the accuracy of an E-commerce system is good and all the other characteristics related to accuracy at the proposed model have low probability, the model would give the result that the probability of an accurate search engine must be at least 85%. This indicates the importance of this characteristic according to the users demands. Similarly, for a good overall system quality andgood system usability the models probabilities about attractiveness would be 80% for good, 16% for average and 4% for poor. Therefore, in the backward use, inserting evidences concerning the child nodes, enables the model to provide estimations about the corresponding probability values of the parent nodes. It should be stressed that the model does not provide estimations by itself. It builds on the experience of the developer. This experience is based on three components: Ecommerce systems applications (modules), end-users demands and quality characteristics. The models estimations are based on discrete probabilities inserted into each Node Probability Table. Thus, the insertion of new evidence may change the models estimations. The models dynamic character is based on the creation of the Node Probability Tables. If the probabilities are based on accurate data that have been systematically collected, the estimation will be accurate. However, even in the case that the data of the Node Probability Tables are not completely accurate, the model can still provide results. It can leam (collect experience) and improve the results it provides. The initial values of the Node Probability Tables have been

derived from measures and experiments conducted by the authors. Affer the initial application of themodel and the consequent improvement of the values in the Node Probability Tables, the probability values resulting from the models application were in agreement with relevant studies of end-users opinions that were conducted using questionnaires.

VI. CONCLUSIONS AND FUTURE WORK

This paper presented a model applied to assess the quality of E-commerce systems as far as the end-user is concerned. The model does not provide results by itself but is based on end-users experience and the accuracy of the evidence inserted into it. The model can be utilized as an important tool for the provision of estimations conceming the quality of Ecommerce systems under development and can therefore aid developers during the design phase. It can also be used backward for the assessment of already developed Ecommerce systems in order to identify problematic or high quality applications (modules). The proposed model is abstract enough so as to provide a general framework for assessment and estimation that can be utilized even if the form of applications comprising an E-Commerce system changes over time. Regarding future work, the authors goal is to provide a model with even lower level of subjectivity. This can he achieved by analysing E-commerce systems characteristics in a way that the user cannot provide estimation but only predefined answers. Furthermore, the nodes that correspond to the E-commerce system characteristics can be further

17

analysed into one or two levels. This is a way to improve the model's accuracy even further.

REFERENCES
V. Zwass, "Elecrroniecommerce: s~mclures and issues". International Joumal of Electronic Commerce, Vol. 1, pp. 3 -23, 1996. H. Chan, R. Lee and T. Dillon, "Commerce Fundamenlals and Applications".John Wiley & Sons,2001. 1. Bunows, "Infomarion Technology srmdords i o changing world: n the role o/ihe uerr". Computer standards & Interfaces, Vol. 20, pp. 323-331.1999. f [41 ISO, "ln/ormnation lechnology - Evolmlion o so/Iwore - @lily ehoracrerisrics and guides for their use': International Standard: lSOllEC9126, 1991. F. Jensen, "An inlroduclion lo B q a i o n Neworb". UCL press, 1996. S. Lauritzen and 1. Spiegelhalter, '"Loco1 eompuloliom wirh probobililies on graphical s~ruciweiand rheir applicaiions lo experl systems". Readings in Uncerlain Reasoning, San Mateo, California, w. 415-448,1990. [71 N. Fenton, P. Krause and M. Neil, ''Sofwme Meoruremenl Cincenloimly and C m d Modellmg". IEEE Soilware, pp. 116-122, JulylAugust 2002. A. Stefani and M. Xenos, "A modclfor assessing the qunliry o/ecommerce Svslems". Proceedinas of the Panhellenic Conference with International Participation in Human Computer Interaction (PC-HCI 2001),pp. 105- 109,2001.

18