This action might not be possible to undo. Are you sure you want to continue?
By Angelina Samaroo, Steve Allott and Brian Hambling, ImagoQA, UK
What is e-commerce? For the purposes of this paper, e-commerce (also known as ebusiness) is defined as the software and business processes required to allow businesses to operate solel or primaril usin! di!ital data flows. "-commerce is often associated with web technolo! and is commonl transacted #ia web portals, but e-commerce is much more than the pro#ision of a web pa!e as the customer interface. $he creation of inte!rated business processes ("nterprise %esource &lannin!), the inte!ration of collections of disparate software applications, each desi!ned to facilitate a different aspect of the business ("nterprise 'pplication (nte!ration), the e)tension of software and business processes to embrace transactions with suppliers* s stems (+uppl ,hain -ana!ement), the need for increased securit for transactions o#er public networks, and the potential #olume demand at e-commerce sites all pro#ide new and unique challen!es to the e-commerce de#elopment communit . challen!es which will require no#el and inno#ator solutions and which will need thorou!h testin! before the are allowed to !o li#e. Wh is testin! important in the e-commerce en#ironment? $he first and primar reason is because e-commerce is, b its #er nature, business critical. (n the third quarter of 1//0, 1ell*s e-commerce site e)ceeded 213 million in dail sales4 the "5$rade site currentl e)ceeds 62, 333 transactions per da , !i#in! a cost of one-da failure of around 2033,3334 and the tra#el industr in "urope will be worth 22 billion b 2332, accordin! to 1atamonitor. $he immediac of the customer, with its implied promise of rapid deli#er at competiti#e prices, and the sheer accessibilit of the web, all combine to create potentiall massi#e demand on web sites and portals. $he second reason is that e-commerce is a massi#e and !rowin! market place but one which requires lar!e up-front in#estment to enter successfull . $here are alread 6.0 million web sites worldwide, 2.6 million of which ha#e been created this ear (1///). $he (nternational 1ata ,orporation ((1,) estimates that the e-commerce market will !row from o#er 26billion in 1//0 to 21trillion in 2337. $he a#era!e cost of de#elopment of an e-commerce site is 21 million, sa s the 8artner 8roup, and will increase b 269 annuall o#er the ne)t 2 ears. $he third reason is because the histor of e-commerce de#elopment has been littered with e)pensi#e failures, at least some of which could ha#e been a#oided b better testin!
Technical Issues $he de#elopment process for e-commerce has unique characteristics and some associated risks. $hese characteristics relate in part to the web technolo! that usuall underlies ecommerce applications. so chan!e is imposed on e-commerce sites e#en when the site itself is not chan!in!. $hese chan!es increase risk and create new challen!es for testers. %apid 'pplication 1e#elopment (%'1) techniques predominate in the e-commerce en#ironment. &roblems with user interfaces lose clients. 't the same time. +uccess will brin! increasin! demand. Failure is unthinkable for a business critical s stem. The Testing Challenges Business Issues ' successful e-commerce application is. thou!h.before the site was opened to the !eneral public. %'1 techniques are not new. and in some cases de#elopment is e#en done directl in a production en#ironment rather than in a separate de#elopment en#ironment. &ri#ac . the technical en#ironment of front-end s stems is chan!in! #er rapidl . %eliable. $his requires more re!ression testin! than would be e)pected in a con#entional application to ensure 2 . -aintainable. other aspects of e-commerce are at least as important as the front-end. :the site* means the entire architecture from suppliers throu!h back-end s stems and front-end s stems to the customers4 it t picall includes (ntranet. "-commerce inte!rates hi!h #alue. hi!h performance business critical s stems. authentication. (n other words. access control. For this reason. ((n e-commerce terms. but the are also dependent on effecti#e inte!ration and effecti#e back-end applications. +ecure. and it is !enerall a!reed that the work best where functionalit is #isible to the user . <sable. inte!rit and non-repudiation are bi! issues. hi!h risk. <nfortunatel . +caleable. 1owntime is too e)pensi#e to tolerate. $he end-to-end inte!ration of business processes and the consequent se#ere constraints placed on intermediate processes make them less than ideal application areas for %'1. (nternet and e)tranet applications as well as le!ac s stems and third part middleware). because time pressures militate a!ainst spendin! a lon!er time testin! sites before the are released. and it is these characteristics that must dominate the approach to testin! because it is these characteristics that determine the success of e-commerce at the business le#el. so web site de#elopment would seem to be an ideal application area. =i!h rates of chan!e are fundamental to e-commerce. a credible update strate! would need to !enerate e-commerce site updates rou!hl monthl . (t is !enerall reco!nised that a :web ear* is about 2 months lon!. =i!hl a#ailable.
$he site must be s ntacticall correct. Fortunatel . 'pplications attached to an e-commerce site. Bac End Systems $he back end of e-commerce s stems will t picall include "%& and database applications. +ecurit is an issue in its own ri!ht. includin! the turnaround time for each ser#ice and the o#erall ser#er response. useful for re!ression testin! and to sa#e time in checkin! basic functionalit . >ew issues ha#e also come to the fore for testers. to ensure that ima!es seen across browsers are of the same qualit . a back end (the software applications underl in! the ke business processes) and some middleware (the inte!ratin! software to link all the rele#ant software applications). $his. What is essential. must be e)ercised across alternati#e platforms. which is a fairl strai!htforward issue. $he back end ma well pro#e to be a bottleneck for user ser#ices. but also has potential to impact on performance. either b . such as ser#er load balancin!. is to appl the ke front end testin! scenarios to the back end s stems.8( pro!rammin! or ser#er e)tensions. notabl securit of transactions and the performance of web sites under hea# load conditions. will need to be tested b creatin! scenarios that !enerate calls to these attached applications. for e)ample b requirin! database searches. the back end s stems should be dri#en b the same real transactions and data that will be used in front end testin!. browsers and network connections. "-commerce applications are essentiall transaction-oriented. and will require effecti#e interfacin! between intranet-based and e)tranet-based applications. $he front end of an e-commerce site is usuall a web site that needs testin! in its own ri!ht. (t should be tested a!ainst a #ariet of browsers. so performance under load and scalabilit are ke issues to be addressed. Dynamic Testing. is about business application testin! and does not pose an new or poorl understood problems from a business perspecti#e. therefore. we can consider each component in isolation. but it must also offer an acceptable le#el of ser#ice on one or more platforms. 7 . clientser#er s stem testin! has tau!ht the testin! communit man #aluable lessons that can be applied in this situation. too. For e)ample. (f we consider an e-commerce site as made up of a front end (the human-computer interface). -an of these tests can be automated b creatin! and runnin! a file of t pical user interactions . (n other words. and ha#e portabilit between chosen platforms. search en!ines and portals. but there are potential new technical problems. $he ser#ices offered to customers must be s stematicall e)plored. ?ack end testin!. the functionalit of buttons on a screen ma be acceptable in isolation. but can a user na#i!ate around the site easil and does information printed from the site look !ood on the pa!e when printed? (t is also important to !ain confidence in the securit of the site. based on ke business processes. howe#er. Front End Systems Static Testing. <sabilit is a ke issue and testin! must adopt a user perspecti#e.that the site continues to function acceptabl after chan!es to browsers.
this adds a new dimension to the problem. since the demand e)pertise in. (f le!ac code is in#ol#ed. one or more of the followin! components are usuall inte!rated. their compatibilit with the pre#ious #ersions. 'lso. and the likel impact of an chan!es. usin! realistic transactions. web ser#er and pa ment ser#er from different #endors. understandin! connecti#it -related issues and inte!ratin! them into a sin!le de#elopment (e)ecutable) en#ironment. is an essential component. there is considerable effort in#ol#ed in networkin! these components.orrectl functionin! back-end and front-end s stems offer no !uarantees of reliable o#erall functionalit or performance. Web ser#er administration. commerce adds e)tra le#els of comple)it . (n order to build an e-commerce application. E . $echnical support should also be borne in mind. (t is also crucial to keep in mind the steep learnin! cur#e associated with cuttin!-ed!e technolo!ies. &a ment ser#er administration. 1atabase administration. and in#esti!atin! all the new features for buildin! optimal solutions for performance can be a dauntin! task. 1atabase +er#er +er#er-side application scripts@pro!rams 'pplication ser#er =$-A forms for user interface 'pplication scripts on the client &a ment ser#er +cripts@pro!rams to inte!rate with le!ac back-end s stems $he process of de#elopin! an e-commerce site is si!nificantl different from de#elopin! a web site . . "nd-to-end testin! of complete inte!rated architectures. (f an application is bein! built that uses a database ser#er. since e-commerce applications on the web are a relati#el new phenomenon. there are unlikel to be an metrics on similar proDects to help with proDect plannin! and de#elopment. Bne hi!hl comple) feature is that of inte!ration.!iddle"are and Integration (nte!ration is the ke to e-commerce. since time will need to be in#ested in understandin! the interfaces to the le!ac code. Ceepin! pace with the latest #ersions of the de#elopment tools and products to be inte!rated. $he maintenance tasks of installin! and up!radin! applications can also become #er in#ol#ed. 'dministration of an other special tools that ha#e been inte!rated into the site.
&lans should include measures of risk and #alue and incorporate testin! and other qualit -related acti#ities that ensure de#elopment is properl focused on achie#in! ma)imum #alue with minimum risk. Principle '. "ffecti#e testin! adopts a strate! that is tailored to the t pe of application or ser#ice bein! tested. with test scripts !i#in! precise instructions and e)pected results. (n the case of e-commerce. due to the ease of searchin! related sites or another totall unrelated site. Kno the value of the applications !eing tested. (n order to ensure that a site loads and functions properl from all supported 6 . $hese criteria .ross-platform testin! is. $he most important lesson we ha#e learned about software testin! is that it is one of the best mechanisms we ha#e for mana!in! the risk to businesses of unsuccessful ($ applications. Ce testin! principles ha#e emer!ed and these can be successfull applied to the e-commerce situation. $he business communit must be in#ol#ed in settin! #alues on which the risk assessment can be based and committed to deli#erin! an a!reed le#el of qualit .Ten Key Principles of Effective E-Commerce Testing B#er the decades since (nformation $echnolo! (($) became a maDor factor in business life.!i#e us the critical e#idence that we need in decidin! readiness to make the site accessible to customers. !i#en the #ariet of platforms and the use of the (nternet as a communications medium. and the risks that would accompan its failure. $here will also need to be some cross-referencin! back to the requirements and obDecti#es. naturall . Create an effective test environment. and minimisin! the risk of a si!nificant failure. but the metrics will at least enable us to decide whether it would be wise to release an application for li#e use. which should be a!reed with the business communit . $his is wh the test pro!ramme must be properl planned. $o mana!e risk effecti#el . $he detailed plannin! of the testin! and the desi!n of the tests can then be conformed b the strate! into a business-focused acti#it that adds real business #alue and pro#ides some obDecti#e assessment of risk at each sta!e of the de#elopment process. Principle 1. When testin! an e-commerce site. (t would be #er e)pensi#e to create a completel representati#e test en#ironment for e-commerce. an important part of testin! an multi-platform software application. testin! enou!h of the requirements to be confident of the most important beha#iour of the site.riteria for successful completion are based on deli#erin! enou!h business #alue. the term :cross-platform* must also e)tend to include :crossbrowser*. Testing is a risk management process. so that some assessment can be made of how man of the requirements ha#e been tested at an !i#en time. the business #alue of the application or ser#ice. %eal proDects ma not achie#e e#er thin! that is planned. it would be #er eas for the testin! to de!enerate into surfin!. Principle ". #et clear testing o!$ectives and criteria for successful completion %including test coverage measures&. . we must know the business #alue of success as well as the cost of failure. Principle 2. . problems and challen!es such as those now faced b the e-commerce communit ha#e been met and sol#ed.
*ser +cceptance Testing %*+T&. se#eral people should be able to lo! into the site and access it concurrentl . e#en if de#elopers continue to impro#e the site durin! beta testin!. should not be seen as a beta-testin! acti#it . to assure continued ali!nment of !oals and responsibilities. %'1 utilises product protot pes. to ensure that product de#elopment does not drift from its ori!inal obDecti#es. $he client or ultimate owner of the ecommerce site should perform field testin! and acceptance testin!. Web-based applications that reference e)ternal links F . for performance@load testin!. as well as the immediate cost of fi)in! the fault. Principle . Principle (. a fault found after shippin! will ha#e been detected as a failure of the site b the marketplace. "arl e)posure of users to sites with problems increases the probabilit that the will find the site unacceptable. at the end of the de#elopment process. is to subDect the site to representati#e usa!e le#els. $he !oal of stress and load testin!. Where %'1 is not used. but testin! strate!ies ha#e been de#eloped b the %'1 communit . Principle ). howe#er. (t is alread well understood and accepted in the software en!ineerin! communit that the earlier faults are detected. the cheaper the cost of rectification. $his has the added complication of loss of interest and possibl the loss of customer lo alt . "commerce users are becomin! increasin!l intolerant of poor sites. with in#ol#ement from the pro#ider where needed. allowin! users to interact with the de#elopers and #alidate product beha#iour continuousl from the be!innin! of the de#elopment process. (n the case of an e-commerce site. so this must be a maDor feature of an e-commerce testin! strate! . "#en if %'1 is used with its continuous user testin! approach. +ome form of final testin! that can address issues such as performance and securit needs to be included as a final confirmation that the site will perform well with t pical user interactions. <'$. $his st le of web de#elopment makes testin! an inte!ral part of the de#elopment process and enhances risk mana!ement throu!hout the de#elopment c cle. 's an absolute minimum. and these can be mobilised for support. fi)ed periods of time durin! which the protot pe can be de#eloped and tested . $he fact that e-commerce de#elopment is rapid and often based on chan!in! requirements makes earl testin! difficult. &erhaps the most important idea in %'1 is the Doint de#elopment team. 'pplications that chan!e need re!ression testin! to confirm that chan!es did not ha#e unintended effects.. the scope of the pro#ider*s internal testin! co#era!e and user acceptance testin! co#era!e should be defined earl in the proDect de#elopment lifec cle (in the $est &lan) and re#isited as the proDect nears completion. performance or reliabilit ha#e been cited as primar reasons wh customers ha#e abandoned sites. -egression testing. de#eloped in a series of strictl controlled :timebo)es* . which is potentiall as lar!e as the number of (nternet users. dele!ated to users in the field before formal release. (t would. therefore. be beneficial to use automated tools. in some cases) to #alidate in this wa . and technical issues related to functionalit . howe#er.platforms. such as +e!ue*s +ilk&erformer or -ercur (nteracti#e*s Aoad%unner. there are some attributes of an ecommerce site that will not be eas (or e#en possible. as much stress and load testin! as possible should be performed. from a mi)ture of the browsers and platforms supported. Test as early as possi!le in the development cycle.
1anage change properly to avoid undoing all the testing effort. $he ke is to take testin! processes sufficientl seriousl that ou document them and control them so that automation becomes a feasible option . Het the time pressures in the e-commerce world militate a!ainst the thorou!h I . misunderstandin!s or deliberate chan!es to s stem functionalit . Principle 10. (t will not be quick or cheap . Capture test incidents and use them to manage risk at release time.ase.+ and . the chances of !ettin! adequate testin! done in the ti!ht time scales for an e-commerce proDect and without automation are e)tremel slim. (t has been said that a fool with a tool is still a fool. . it is in the more familiar areas of the technolo! that the most serious problems arise. purchase and install the tools. because the en#ironment is chan!in! continuousl . 'll incidents found must be recorded #ia an incident mana!ement s stem ((-+). >e#ertheless. such as &G. ' test incident is an discrepanc between the e)pected and actual results of a test. in order to minimise the impact on the test schedule. $hin!s chan!e quickl and often in an e-commerce de#elopment and mana!ement of chan!e can be a bottleneck.onfi!uration -ana!ement tools. $his is a risk principle because test automation is frau!ht with difficulties. Conclusions "-commerce is both familiar and no#el. Principle . +ome of the technolo! is relati#el no#el. Bnl some test incidents will relate to actual faults4 some will be caused b incorrect test scripts.need re!ular re!ression testin!. but the discipline is the most important thin!. and the application of that technolo! to a complete business is certainl no#el. and both of these are true. and that the outcome of automatin! an unstable process is faster chaos. Butstandin! incidents can be one of the completion criteria that we appl . &arado)icall . re!ression testin! should be automated. because the emer!ence of e-commerce has placed new and challen!in! requirements on this relati#el old technolo! that was desi!ned for a quite different purpose. but it mi!ht Dust a#oid a #er e)pensi#e failure. Where#er possible. $estin! is crucial to e-commerce because e-commerce sites are both business critical and hi!hl #isible to their users4 an failure can be immediatel e)pensi#e in terms of lost re#enue and e#en more e)pensi#e in the lon!er term if disaffected users seek alternati#e sites. can help to minimise the o#erheads of chan!e mana!ement. but the problems of creatin! business processes to operate a business in a wholl new en#ironment o#ershadow all of that no#elt with some familiar and intractable problems. e#en if their functionalit does not chan!e.lear. so the abilit to track and e#aluate the importance of incidents is crucial to the mana!ement of testin!. but there is little point in testin! one #ersion of a software application and then shippin! a different #ersion4 not onl is the testin! effort wasted.. but the risk is not reduced either. Principle /. +utomate as much as possi!le. which can then be used to ascertain what faults are outstandin! in the s stem and what the risks of release mi!ht be. then ou select.
Aike most new #entures. (n this paper we ha#e su!!ested some testin! principles that ha#e stood the test of time and intermin!led them with some lessons learned from similarl challen!in! de#elopment en#ironments to !i#e e-commerce testers a starin! point for their Dourne of disco#er . 0 . %apid 'pplications 1e#elopment (%'1). su!!ests some promisin! approaches. $he #er familiarit of much of the technolo! means that tried and true mechanisms will either be suitable or can be modified to fit. thou!h. so a new approach is needed to enable testin! to be inte!rated into the de#elopment process and to ensure that testin! does not present a si!nificant time burden.testin! usuall associated with business criticalit . in particular. ecommerce must find its own wa and establish its own methods.
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue listening from where you left off, or restart the preview.