You are on page 1of 19
formation S)stems Vol. 28, No.6 9p. 425-493, 1999 Pergamon (© 1999 Published by Biever Science Lu. Al rights reserved Printed ia Great Brin 0306-43799 $20.00 + 000 Il: $0306-4379(99)00028-9 DEVELOPING INTERNET E-COMMERCE BENCHMARKS’ DAWN JUTLA', PETER BODORIK’, and YIE WANG? ‘Faculty of Commerce, Saint Mary’ University, 923 Robie Steet, Halifax, Nova Scotia, Canada B3H 3C3 Faculty of Computer Science, DalTech, Dalhousie University, P.O. Box 1000, Halifax, Nova Scotia, Canada B3J 2X4 (Received 28 February 1999; in final revised form 10 August 1999) ‘Abstract — A benchmark i a standard for measuring and comparing the performance of like systems. For new product maker, a benchmark can provide imporant statistical infomation so products canbe fine-tuned before their deployment. For end user, on the other hand, a benchmark ean be used to compare the strengths and ‘weaknesses of diferent products so that an informed decision can be made about system adoption. Benchmarks Aid in estimations of scalability in terms ofthe numberof users andor transactions tata system can support, and system response times under various loads and hardwaresoftware deployment platforms. ‘This paper focuses on the design issues in developing benchmarks for e-commerce. Because of the mlidscplinary aspects of e-commerce and the various emerging and distinct e-commerce business models, creating a single benchmark forthe e-commerce aplication is ot feasible. Add to this the diverse neds of small {to medium enterprises (SMEs) and big business and we motivate the nocd fora benchmark site for e-commerce. Its the thesis of this paper that the Business model plays the primary role inthe development of e-commerce benchmark. It is the business that determines processes and transactions and thus also the database and navigational designs. For illastative purposes, we step through the design of an e-commerce benchmark specification, WebEC, based on a e-broker (cybermediary) Inemet business model, An example implementation Of the benchmark specification, based on Microsofts COM technology, and sample beachmark results ar also presented. © 1999 Published by Elsevier Science Lid. All ighs reserved Key words: E-Commerce Benchmark, E-Commerce Performance Measurement, Intemet Business Model, COM Technology, Web Transaction Processing 1, INTRODUCTION Critical success factors for an Internet e-commerce business channel can be categorized as strategic, technical and functional. Strategies embody benefits gained from first-to-market, brand establishment, customer focus, targeted marketing, outsourcing and development of a customer or user community. Technical issues encompass quality of service items such as response time, transactional throughput, site availability, reliability, video and sound quality. Scalability, the interoperability of technologies, and security are also in the technical category. The supported transaction set (e.g. buy, shopping cart, browse, search and order status, negotiation, delivery) overlap in both user and system application functions. Functional customer care features include facilitation of product customization, support for negotiation and access to a similar-interest user community. Strategic benefits are not quantified in any exjsting benchmark application; they are difficult to quantify and return on investment formulae/calculations for e-commerce are not yet fully developed. However metrics for quality of service items are relatively easy to define for an e-commerce benchmark application, Functionality is determined by the underlying e-commerce business model and can be captured in the benchmark e-commerce application itself. ‘The diverse emerging e-commerce business models complicate e-commerce benchmark development. ‘We identify the need to develop different e-commerce benchmarks for each individual e-commerce business model. Business models for e-commerce are classed into the following categories: the e-broker (cybermediary) model, the manufacturer model, and the auction model [11]. The cybermediary model is being used successfully by companies such as 1-800-FLOWERS, amazon.com and abe.com. Dell and Cisco are example successes of the manufacturer business model. The auction model can be found in companies such as priceline.com, Because of continuous change in e-commerce we expect hybrid and completely new models to evolve: the categorization is far from complete at this stage. ‘Recommended by Michael P. Papazoglou and Aphrodite Tsalgatidou a5 416 DAWNJUTLA et al The business model determines processes and transactions and thus also the database and navigational designs. Because a benchmark must model these aspects, development of an e-commerce benchmark must be targeted to a specific business model. Other aspects of benchmark design include definitions of performance metrics, probability distributions for workload characterization, and reporting formats for results. ‘The paper is organized as follows. Section 2 outlines the goals and suggests users of an e-commerce benchmarking application. Three classes of e-commerce business models are briefly described in Sec- tion 3. Section 4 details components of e-commerce benchmark design. Section 5 describes an example benchmark design. Section 6 briefly outlines an example implementation and raises other technological issues in the e-commerce environment. Section 7 presents example results. Related work is detailed in Section 8. Section 9 gives a summary and concluding remarks. 2. BENCHMARKING GOALS AND METRICS E-Commerce benchmark applications can aid in system sizing, capacity planning, system tuning, and quality of service measurement and validation for transactional eCommerce business models. System sizing refers to the short-term initial sizing activity such as selecting processor speeds, memory capacities and number of servers involved in hardware acquisition. Capacity planning covers what-if scenarios, It ‘can occur many times in the business cycle; for example to handle peakedness, scalability issues, introduction of new services or on business expansion. System tuning refers to the “tweaking” of a system to get maximum performance out of the configuration. Acceptable quality of service refers to reasonable site response times, transaction throughput, availability of connections to the site and other site services, reliability as in persistence of connections and uptime, and acceptable levels of video stream and audio rates. “Everyone” is going online to support business activities. E-Commerce hardware vendors, commerce service providers (CSPs), IT departments of big businesses, and consultants can use the benchmark to do system sizing and capacity planning for their E-Commerce customers. Example management questions that the benchmarks can address are: © Will e-commerce site performance continue to be acceptable when X as many people visit the site? © Are servers and internal network capacity adequate to handle load spikes? © Atwhat will capacity be saturated? © Can the current system support load increase while preserving the response time below 3 seconds? ‘* Where will the system bottleneck if the site scales from 10,000 customers to 500,000? ‘© What will the site's response time be (in seconds) if 1. another CPU is added to the web server? 2. another web server is added? 3. the database is partitioned and housed on two data servers? Clearly from the above list, the response time parameter is very important. It can be used to answer most of the questions once an allowed range for the parameter is defined. However a benchmark consists of 2 major components - business logic as well as performance metrics definitions, ‘The business logic component requires fairly accurate representation of an e-commerce business model, the supporting web navigational and database designs, transaction definitions, and integration with payment systems and security. The performance metrics take into account the rich multimedia environment and interactions with servers and the network components. Performance metrics involve quality of service parameters such as transaction throughput, response time, reliability, and audio and video quality. CPU, disk, network and memory usages are also tracked to identify system bottlenecks. Developing Internet E-Commerce Benchmarks an Fig, 1: The Cybermediary Business Mode! 3. BUSINESS MODELS 3.1. Introduction ‘The Internet e-commerce business models impact the e-commerce benchmark application in terms of distribution types and makeup of the business transactions, frequency of invocation of individual transactions, and number of accesses to remote databases over the Internet. Various definitions of business models for e-commerce include “‘an approach a company takes to make money online” and “entire collection of activities that support commercial activities on a network”. We broadly categorize current e-commerce business models into three classes: cybermediary, manufacturer and auction models [11]. These business models are described in terms of entity (e.g. customer, supplier) interactions for facilitation of benchmark design. 3.2, Electronic Commerce CyberMediary (E-Broker) Model ‘The cybermediary or e-broker model is characterized by an intermediary between suppliers of goods and/or services and the customer. The intermediary or cybermediary adds value to its on-line supplier sites, either by marketing a large range of similar type products from one site, or through enabling ‘comparison shopping, or through facilitating coalition industries such as real estate and the automotive industry that provide multiple company listings. Other ways of adding value is through creation of community, facilitation of customization, and enhanced customer care through user services such as personal account information. Negotiation facilities also provide added value on an online site. The cybermediary model can support the sourcing of a product or service from many suppliers and may provide a customer with a choice of products/services, or the best price/delivery time for a product/service, or bulk discounts. Figure 1 shows the architecture for the cybermediary model described in terms of interactions among the business entities. This model incorporates access to databases internal to the company and to external databases of some other companies. Security and payment system(s) are implicitly incorporated in the electronic commerce application itself. Normally limited stock is held in the cybermediary company. Delivery can be made either from the cybermediary or from the supplier. Some suppliers are not set up to deliver goods of one unit size but can only accommodate delivery of multiple units thus in some instances the cybermediary may need to mediate delivery. 418 DAWNJUTLA eral Supplier Supplier of Raw of Raw Materials: Materials, oO © @ Finance fe Customer Fig 2; The Manufacturer Business Model ‘The architectural model consists of querying multiple online supplier databases, requesting service from third party businesses and responding to the consumer. In a brokerage a function may be applied to select the supplier with the least cost, shortest delivery time or sufficient stock. In addition to the online consultation of the suppliers’ databases, credit verification at a third party financial-service company (e.g. bank) is required. One cybermediary can communicate with other similarly organized cybermediaries to find the best deal for the brokerage and the client. Figure | shows how the electronic broker model works. In (1), Enterprise X (the broker) electronically sources quantities of ABC (the product or service) from multiple suppliers or other cybermediaries over the Internet to source the best price and delivery terms. In (2), the customer adds an ABC to his/her shopping cart. In (3), the customer clicks on the buy button, which prompts a query for credit card information and user registration (4). The information is sent to a third party, such as a credit card ‘company (5), and the company responds with an authorization number (6). Finally, Enterprise X sends the customer a notification that the order is confirmed. The order tracking service (7) allows the customer to check at what stage of delivery the product is at present time. ‘This model has many advantages: It reduces the inventory management overhead in terms of staffing and space and reduces capital tied in inventory. Most importantly, it provides the specialization in areas of expertise (marketing, delivery) across the supply chain from manufacturers to customers. E-broker companies are marketing specialists while suppliers have experience in production planning, inventory ‘management and product expertise. The more specialized the functionality, the better the quality of the service the company provides, and at lower costs. This model argues for the strengths of outsourcing. 3.3. Manufacturer Model Dell, Cisco and General Electric are example successes of the manufacturer business model. The ‘manufacturer model implies that marketing and distribution are part of the company’s operations. Note that here manufacturer encompasses the re-packagers as well. The virtual supply chain and forecasts for product demand support production planning. Unlike the cybermediary model where the finished good is bought from suppliers and resold, the manufacturers create value-added products through their internal manufacturing processes. This model works best for organizations with configurable products, mature marketing staff and sophisticated customer service processes. Established businesses such as car and ‘computer product manufacturers use this model. Figure 2 shows how the manufacturer business model works. In (1) the customer selects product components and receives pricing. In (2) the customer issues a buy transaction. In (3) internal access is made to production data to determine delivery timing and inventory requirements. The customer's Developing Internet E-Commerce Benchmarks a9 financial status is verified in (4). Confirmation of order and delivery dates notification is issued in (5). In (6) B2B e-commerce (buying side) is performed, 3.4, Auction Model ‘The auction model, sometimes referred to as the Internet Exchange model can be found in companies such as priceline.com and eBay.com. Basically this model works like a stock auction market where the buyer sets the price of the product through submission of bids and the willingness of suppliers to sell at the bidding price (Figure 3). Many auction sites charge a small fee to the suppliers when a sale is made and also may charge a fee for the listing of goods for sale at their site, Businesses that use this model require large customer bases because of low cost services in order to make money. The auction model attracts price sensitive/aware customers and customers with a host of varying product requirements. Businesses derive their income from charging small fees to sellers when a sale is made through their site and to list goods for sale. Buyers are not charged any fees. Another revenue model for these businesses can be solely through the sales of advertisements at their site. Since they attract a huge customer base, they are also in the position to act as a portal v © Buyer (Customer) Seller (Customer) Fig. 3: The Auction Business Model Figure 3 shows how the auction business model works. In (1) the customer submits a bidding price. Internal processing of customers’ bids is shown in (2). In (3) the customer with the winning bid is notified and the seller who has made the sale is notified in (4) of the customer's identity. The seller and buyer transact the final settlement details in (5) without the intervention of Enterprise Z. In (6) the seller remits a transaction fee for the sale to the business Z. In (7) sellers maintain listed items at the site. 3.5. User View of the Business Models From the user point of view the cybermediary and manufacturer models have similar interactions e.g. shopping cart, browse, buy and so on. However despite this the benchmarks for these models will differ since they support different workloads. Also the business logic within the transactions are different for the manufacturer and cybermediary models. Manufacturers or re-packagers will have integrated e-commerce services to their back-end production and planning systems whereas the cybermediary requires far less integration with other enterprise resource modules. From the user point of view the auction model is clearly quite different, supporting bid and sell processes and the removal of the payment and deliver processes. Thus each model requires its own benchmarking system. See [1, 4, 14, 15, and 17] for further discussion of business models. 480 DAWNJUTLA et al. 4. COMPONENTS OF E-COMMERCE BENCHMARK DESIGN 4.1. E-Commerce Business Processes and Transactions A full description of an e-commerce business model for the benchmark specification includes process model and transaction model components. Table 1 shows a listing of a few business processes and their ‘mapping to business transactions. Business transactions in turn map to one or more database transactions. The ACID (Atomicity, Consistency, Isolation and Durability) properties of database transactions must be enforced in the distributed web environment and forms a central technology issue for online businesses. Business | Business Process | Tran- | saction Buy “Customer clicks buy button on commerce senerierface The buy Wansacton Extemal ) | complex for goods that canbe electronically delivered ‘Shopping| Customer adds order line to temporary electronic oder ieluding quanti, em Cant Payment | Customer selects a method of payment option rom the commerce server interface, Some __| persons payment information may be required at this point such as eredit card number ‘Order Sakus”| "Customer tracks order through parner web sites. Most commonly, delivery sites such as Fedex and UPS are used to support this funtion __| | Customer | New customer provides personal information such as name and mailing address OR an Registrat-ion_| existing customer is recalled from the customer database Description Retail (Credit check | A requests electronically transmited toa third party finance company, ea bank, and ternal) ‘some authorization indicator is returned electronically E-broker | Acces (othe parmering Extranet e-broke's online databases fs made. Wt ebroker can contact | supply our needs hen procurement processing starts ie. purchase order is generated and slecinically anced Supplier | Acces tothe partnering Ean! supple online datbaics Tora sales dst i ade contact’ | Comparison of owes prices, and delivery time windows may be optimized electronically thereby facitating the determination of the odes suppliers Onder Sends order confirmation fo the Customer citer va webpage o eal Confirm Delivery] The supple ialtnes delivery of goods ely wo oa carom. The supplier provides our business witha tacking number thats aseisted with our intemal order number | Customer J Customer orders archived for future data mining purposes "A Purchase order is automatically generated when a cient buys and the e-business ‘cannot supply out ofits inventory (either because stock isnot kept on hand, or for inventory replenishment JOR A PO is automatically generated if stock falls below & ‘certain inventony threshold. ‘Supplier issues electronic confirmation of PO receipt ‘Acceat suppliers web se to tack PO statis ‘Shipping not (racking number) is electronically set wo the Business Receipt of | Ifthe ebusines stocks goods, then goods received must be matched tothe PO. Aso goods triggers the creation of an accounts payable record Payment | Payment can be automatically generated within the credit conditions set out by the suppliers to optimize discounts, ‘Table 1: E-Commerce Business Processes and Transactions 4.2, Database and Navigational Designs E-Commerce applications require database and web navigational designs to support the business transactions. A good database design is essential for maintaining data integrity in an operational environment. An e-commerce benchmark must specify the contents of the data tables that will support the required database and hence business transactions as well as details on the data table sizes and description of the data for database population. Developing Inmet E-Commerce Benchmarks 48 ‘The navigational design is responsible for the layout of the e-commerce site and provides the users’ access points to the indexes of the site's database(s) and hence catalogs. Many studies have tracked customer paths within the traditional offline store for identification of the strategic positioning of products. Similar activities occur at online e-commerce sites that aid in fine-tuning a navigational design. 4.3, Security ‘The perceived or real lack of security over the Internet has been identified as a barrier to consumers’ acceptance of online purchasing. Hence an e-commerce benchmark application must incorporate delays due to the overhead of the presence of a security protocol. For most e-commerce sites, security is provided on the transmission of financial information such as credit card numbers. ‘The Secure Sockets Layer (SSL) security protocol [18] provides for data security and is layered between the service protocols HTTP and TCP/IP. SSL provides data encryption, server authentication, and message integrity. The SSL protocol employs public key cryptography. Secure Electronic ‘Transactions (SET) is a complex security protocol with a very high overhead due to its use of several different subprotocols. SET was devised by the credit card consortium of VISA and MasterCard. SET has not been widely adopted by Industry unlike SSL, which leads in the number of Commerce Service Providers that use it for security purposes. Another fairly widely used security protocol is PGP. Hidgins- Bonafield [9] reports on surveys that show that SSL is the most widely adopted security method today. It is thus currently the common-case security protocol that should be adopted in any e-commerce application benchmark. 4.4, Payment Systems ‘An e-commerce application must support one or more payment systems. The load and hence delays due to processing payments must be incorporated in the instrumentation of the e-commerce benchmark application. Payment systems are available for products that cost cents or fraction of a cent to thousands of dollars. Micropayment systems tend to be expensive but have a niche market The Mondex system based on Smart Cards that can support multiple applications and store cash is one of the more promising Payment systems. However the most common payment system found in the majority of present-day e- ‘commerce sites is the traditional credit card system, which means that it should be included as the base payment system into present e-commerce benchmark specifications. 4.5. Performance Metrics, Workload and Probability Distributions Quality of service issues for an e-commerce site such as transaction throughput, response time, video quality, sound quality, availability, and reliability are measurable and are defined as benchmark metrics. Ultimately, the benchmarking application should be tooled to measure time spent at the individual components such as web servers and database servers, measure O/H due to security, payment protocols, and describe load at the web, database and application servers such as CPU, disk and memory usage. ‘The traditional definition of transaction throughput is the total number of completed business transactions divided by the elapsed time of the measurement interval. That is, the throughput is a number ‘of business transactions processed per minute. Subtleties for the throughput definition involve the perception of when a transaction is completed. For example, is a browse transaction completed when the first character of the results appears on the client or when all results are displayed? ‘The traditional definition of response time is the total individual response time of each business transaction divided by the number of transactions during the measurement interval. The individual response of a business transaction is the time between the first timestamp and the second timestamp. The first timestamp is taken after the last character of the required input data has been entered and the appropriate action is triggered. The second timestamp is taken after the last character of the output data is received at the client machine. Again itis easy to see how definitions of response times can vary from one benchmark application to another depending on when the timestamps are taken. ‘The waiting times between transactions are key times and think times. The key times for the transactions are chosen to be approximately proportional to the number of characters input, and the think nal to the number of characters output. Response time does not include think and key or click times. The workload characterization for a site includes the required minimum percentage of mix for each transaction type operating over the measurement interval. Other characterization is for the distribution of access to data held at the site, Probability distributions describe data access distributions in a benchmark application that are used to select various data from data tables. Workload on e-commerce sites may also vary over time, due to seasonal activities and other external factors such as advertising campaigns, or through other avenues of publicity. The benchmark needs to vary workload to reflect sudden surges in site activity. This also pertains to the scalability of a site's e-commerce application. 4.6. Results Format / Disclosure Reports Results of the e-commerce benchmark must be reported according to a standard format. Full specifications of the hardware and software components including the workload, number of users, think and key times are to be disclosed. The metrics such as transactional throughput, response times, availability, reliability and so on, described in Section 4.5., must also be reported along with clear definitions of these metrics. 5. AN EXAMPLE BENCHMARK SPECIFICATION: WEBEC Since any benchmark specification must be based on a single business model, we select the cybermediary model as the underlying business model in WebEC. The benchmark is targeted to the small-to-medium enterprises (SMEs). Business transaction definitions (browse, user registration, buy, shopping cart and order status), except for the buy transaction, provided in this section are generic to any cybermediary based e-commerce site. WebEC specifies a complex buy transaction. WebEC’s buy transaction consists of a set of nested transactions which perform credit checks, user authentication and product availability searches. There are ‘many instances, particularly in the immediate delivery distribution, where these checks need to be done as part of the buy transaction. Examples are in information exchange and sales of software, news and journal articles, The security protocol used is SSL and the payment method is through credit card. For simplicity within the WebEC benchmark example, the e-commerce business is assumed to be constrained to the North American domestic market and we further restrict this example benchmark to two performance ‘metrics: transaction throughput and response time. 5.1. WebEC Benchmark Business Transactions ‘The WebEC benchmark specifies a web e-commerce transaction processing (OLTP) workload. It is a mixture of online transactions that simulate the activities found in the cybermediary model environments. Similar to other benchmarks, the purpose of the WebEC benchmark is to retain the essential characteristics for a given environment or context, namely, the level of system utilization and the complexity of the operations (19). The buy transaction is distributed in WebEC. The distributed transaction spans different remote resource managers, e.g., database management systems (DBMSs) residing on the partner supplier sites. Work can be committed as an atomic transaction even if it spans multiple resource managers on separate ‘machines. The participants in one atomic transaction of the WebEC benchmark include a web browser, a web server, and several remote databases. A WebEC distributed transaction refers to a unit of work with full ACID properties (atomicity, consistency, isolation and durability). From the view of execution time, a WebEC transaction starts when the request of a web page is triggered, and ends when the committed ot aborted result is completely loaded on to the browser's screen. Another feature of the WebEC transactions is that an external business transaction may consist of several internal business transactions (alternatively called sub-transactions). External transactions are accessible to the customer. For example, the transactions, such as the buy transaction, and the shopping cart transaction, are external transactions. Internal transactions, such as the internal online supplier contact transaction, are performed by the e-broker itself and the customer has no way of accessing the results of this transaction. Developing Internet E-Commerce Benchmarks 483 External Transactions Internal Transactions Shopping Cart Internal Credit Verification Customer Registration Internal Online Supplier Contact, Buy Internal Delivery of Purchase Order Order Status Internal Customer History Update Welcome Page Browsing Internal E-broker Category Page Browsing Internal Order Confirmation Product by Category Page Browsing Product Page Browsing Stock Level Update ‘Table 2: Summary of WebEC Business Trans Any internal transaction’s execution status affects its external transaction. Abortion in any internal transaction results in the rolling back of its associated external transaction. A summary of WebEC’s transactions is provided in Table 2. 5.2. Database Entities, Relationships and Characteristics ‘There are three database types in the WebEC benchmark system: cybermediary, supplier and finance. The first is the database type residing on the local cybermediary, called ‘ebroker’. In the benchmark specification, four partner suppliers hold trusted relationships with the e-broker. The second database type is that residing on the remote suppliers’ sites. There are four instances of the supplier database, si to s4 and one e-broker database, el. e1 is simulated as the other cybermediary site (see Figure 1) with which ‘our WebEC e-broker can communicate in case there is not enough stock from these four partner suppliers. Databases si to s4 have similar schemas. The thitd type is the database, fl, sitting on the finance company site. ‘The components of the local cybermediary database are defined to consist of twelve tables. The relationships of tables in the e-broker database are shown in Figure 4. The database for a remote supplier consists of two tables shown in Figure 5. In the finance company’s database, shown in Figure 6, two tables that can be accessed for credit information are held. In Figure 4, SC_Line stands for the shopping cart lines and PO_Line for the purchase order lines. Figures 5 and 6 show a subset of the tables found in the remote database and there are no relationships between them, 5.3. Navigation Design ‘The web pages include: Welcome Page, Category Page, Products by Category Page, Product Page, Shopping Cart Contents Page, User Registration Page, Credit Card Info Page, Order Status Page, Order Confirmed Page, and Thank-you Page. Navigation amongst the pages is also defined. 484 DAWN JUTLA eral History Customer Supplier " + Order Supplier_ttem Item 1 SC_Line Product_by_Category —> EN nen Category Delivery Company Lookup Table Fig. 4: Relationships of Tables in the Cybermediary’s (E-Broker) Database Remote_Item ‘Tracking Fig. 5: Tables in a Remote Supplier's Database Credit_Info Auth_Number Fig. 6: Tables ina Finance Company's Database 6. AN EXAMPLE WEBEC BENCHMARK IMPLEMENTATION ‘The WebEC specification may be implemented using different underlying distributed technologies such as CORBA, COM/DCOM or Java APIs and base classes. Indeed it can be implemented on any platform, heterogeneous or not. E-Commerce application development toolkits such as IBM's net.commerce and Microsoft Site Server allow for the specification of business transactions and thus can support the WebEC specification, The WebEC benchmark specification does not limit the use of any database server, OS, commerce server, payment server etc. However, due to space limitations this section briefly describes one example implementation of the WebEC benchmark using Microsoft's Component Object Model (COM) / Microsoft Transaction Server 2.0 (MTS) / Internet Information Server 4.0 (IIS) / ActiveX Server Page 2.0 (ASP) platform. The business logic of the WebEC benchmark is implemented as COM components. MTS groups the COM components under transactional control. IIS serves as the web server that accepts ASP requests from the ASP page sent from the client’s browser. Clients’ application is written using ASP script language. The three-tier client and server model, web server, middleware, database connectivity and transaction control, implementation of the WebEC benchmark coordinator and client/server applications are briefly described next. Developing Internet E-Commerce Benchmarks 485 ca User Services Business Services Data Services Client Application |, | Business Rules and

You might also like