he testing of Web-based applications has much in common with the testing of desktop systems: You need to test the usual func-
tionality, configuration, and compatibility, as well as perform- ing all the standard test types. But Web application testing is more diffi- cult because complexities are multi- plied by all the distributed system components that interact with the ap- plication. When we see an error in a Web environment, it\u2019s often difficult to pinpoint where the error occurs, and, because the behavior we see or the error message we receive may be the result of errors happening on dif- ferent parts of the Web system, the er- ror may be difficult to reproduce. So how do we analyze errors within a Web-based system, and what consid- erations should be made for reproduc- ing such errors?
When we have an understanding of the underlying technology, we are better able to maximize testing effi- ciencies\u2014writing more reproducible bug reports and finding more errors in less time. This is easier said than
with error-prone technology vari- ables. Here are five fundamental con- siderations of Web-application test- ing:
client side, we are seeing the symptom of an error\u2014not the er- ror itself.
erating environments\u2014static versusd y n a m i c\u2014demands dif- ferent approaches.
Without diagnosing the environment, we can\u2019t be certain what causes a symptom to appear. If one of the en- vironment-specific variables from ei- ther the client side or the server side is removed or altered, we might not be able to reproduce the problem.
Here is an example. I\u2019m testing a Web-based defect tracking applica- tion, and going through the process of creating a new bug report. When I select the NEW button, I receive an error message:
replicate both the exact sequence of activitiesa n d the environment con- ditions (operating system, browser version, add-on components, data- base server, Web server, third-party components, server/client
sources, network bandwidth and traffic, etc.) in which the application operates. For example, when you try to log into your Web application while using a 28.8 kbps dial-up con- nection, you experience login fail- ures due to timeout in the authenti- cation process\u2014but the same login steps will authenticate successfully if you are on a T-1 connection at 1.54 mbps. In this case, you have an environment-dependent error where the dependency is in the bandwidth.
Environment-independent er- rors, on the other hand, are relatively easier to reproduce\u2014it\u2019s not neces- sary to replicate the operating envi- ronment. With environment-inde- pendent errors, all that need be replicated is the steps that reveal the error. For example, if the company name is misspelled on all of the prod- uct\u2019s online pages asWe b Te s s t i n g .
operating environment. More com- monly, we refer to environment-in- dependent errors asf u n c t i on a li t y -
posed errors) may be resolved with code fixes (assuming the errors are in fact real) or system reconfigura- tion (client, server, or network). Don\u2019t jump too quickly to the conclu- sion that it\u2019s ab u g !
Here is an example illustrating the challenge of identifying possible configuration problems as opposed to actual software errors. It shows an error message caused by a \u201cfailed login\u201d that has been generat- ed by a Web application. By simply looking at this error message, it is impossible to determine whether this error is the result of a software bug, a server-side configuration is- sue, a compatibility issue, a browser configuration issue, or all of the above.
After further analyzing the fail- ure, I discover several possible con- ditions that might generate this error message:
directory is not properly configured, the requested files, scripts, or data will not be found. Typically, this is a
server configuration issue. However, if the installation program failed to programmatically configure the Web server according to specification, then this is a software error. If a system administrator fails to proper- ly configure the Web server accord- ing to specification, this then be- comes a user error.
typical application-server directory contains scripts to be executed when they are called by a Web server on the behalf of a client. For security reasons, a Web server can be config-
ured to allow or disallow scripts to be executed within certain directo- ries. If your application-server direc- tory is designed to contain scripts that will be executed\u2014but the Web server is configured to disable script execution in that directory\u2014the ap- plication will not work. Is this as of t -
cation server needs to connect to the backend database living on the SQL server in order to execute queries, store procedures, and access data. If the SQL server process itself is not running, then obviously the applica- tion will not work.
installation program failed to copy all theDLLs used by the application server during setup. If anyDLL need- ed by the application server is miss- ing, the application will not work.
Perhaps the installation program correctly copied all the needed mod- ules, but failed to register one or
To reproduce an environment-dependent error
we have to perfectly replicate both the exact sequence
of activities and the environment conditions.
more of them. For example, withOLE- based objects such asCOM orDCOM, their class ID (CLSID) must be regis- tered in the Registry Database before they can be used. If an application tries to access aCOM object that was not registered successfully, the appli- cation will not work.
This problem is often caused by errors in the installation procedures. If, on the other hand, the compo- nents must be manually registered then this becomes ac on f i g u r a t i on
Errors in Web systems are often dif- ficult to consistently reproduce be- cause of the many variables intro- duced by the distributed nature of client/server architecture (i.e., serv- er, client, and networking compo- nents). There are at least three usu- al suspects in a Web environment: Thec l i e n t , thes e r v e r, and then e t -
ty issues that are similar to PC envi- ronments, where all components are in one box. Issues multiply within client/server systems, however, be- cause there may be many clients and servers connected on a network. Typical client/server configuration and compatibility issues involve the hardware and operating system mix (UNIX-based boxes versus Windows- based boxes, for example) and the software mix on the server side (Web server packages, database server packages, firewalls,COM objects,
involve the software mix on the client side (TCP/IP stacks, dialer software, helper components, brows- er brands, and browser versions). Additionally, browser settings, such as general settings, connection set- tings, security settings (including Ac-
This action might not be possible to undo. Are you sure you want to continue?