Testing a Web Application
Harry M. Sneed
Ancon GmbH, Vienna, Austria & Budapest, Hungary
Email: h.sneed@axelero.hu
Abstract: The following paper describes the
requirements and contraints of testing a web
application system for the Austrian Chamber of
Commerce. It seperates the testing task into two
parts – one for testing the web architecture and one
for testing the internet application itself. The high
costs of testing an internet application are driven
not only by the complexity of the architecture, but
also by the complex usage profiles of the
application. Non-functional quality characteristics
such as usability, performance, security and
interoperability take on particular importance in
web-based systems. Without a staff of skilled
testers equipped with automated tools, it is not
possible to ensure the reliability of e-commerce
applications. The evolution of such systems
requires that the evolved application be tested
against the old one. In the project referred to here, a
comparison tool was developed to validate both the
backend data base contents and the frontend web
page contents.
Keywords: Web Applications, Evolution,
Regression Testing, Assertions, File Comparison
1. The high costs of testing a web
application
The more complex an environment is, the greater
the effort required to demonstrate the reliability of a
software system operating in that environment.
Testing has always been difficult and time
consuming even in the days of stabile, closed,
monolithic mainframe environments. Considering
the fact that that the operating system was almost
totally predictable, that the teleprocessing monitor
and database systems were equally reliable and that
security was no real issue, it still required up to
30% of the total project effort to test new
applications.[1] The test research community has
always insisted that testing is underestimated and
that it requires equally as much effort to test a
function as it does to program that function. [2]
This applies to projects where the operating
environment is assumed to be correct and not part
of the testing problem. It turns out to be an
understatement when, as is the case with the web,
the environment becomes a part of the testing
problem.
This was already the case to a certain extent with
distributed systems. The middleware, which is
connecting clients and servers became a part of the
Proceedings of the Sixth IEEE International Workshop on Web Site Evolution (WSE’04)
1550-4441/04 $ 20.00 IEEE
test domain. On top of that, the user application
took over many of the service functions, such as
memory allocation and error handling, which were
once handled by the operating system or the
teleprocessing monitor. Not only that, the number
of potential interactions between dispersed
functions was doubled as was their
interdependencies. This led Boris Beizer to claim
that the testing of object-oriented software in a
client/server architecture could cost three times
more than the testing of conventional software in a
mainframe environment. [3] This claim may be a
exaggerated, but the fact is that the relation of test
effort to development effort has changed
significantly with the advent of client/server
systems, so that testing may require up to 50 % of
the total project effort according to P. Goglia [4]
The emergence of e-business applications in the
internet has added a further dimension of
complexity to the testing problem. Not only is the
user software distributed between clients and
servers, but now there are many potential servers.
A client request may be processed by several
alternative servers, which may, in turn, route the
request to other servers further down the line. If the
application provider is cautious, he will also be
employing at least two web servers to route his
transactions. The operating environment becomes a
complex network of client, server and router nodes,
exploding the number of interactions between the
application software and the support software.
Assuming that testing effort is driven by the
complexity of the environment, then testing web
based applications could be even more expensive
than testing client/server systems which are already
more expensive than testing conventional stand
alone systems. The applications themselves may be
simple, but the environment in which they operate
is not and that environment is, by nature, part of the
testing problem. [5]
That being the case, the significance of the
architecture test increases when testing applications
on the web. Clients are connected to web servers
,which are connected to application servers which
are connected, in turn, to function servers and data
servers. Every connection is handled by another
kind of middleware. Data can also be passed in
various forms such as HTML or XML forms, by
means of parameters in a CORBA or DCOM
interface, or as a combination of both. [6] Each
potential connection and the corresponding
 interface causes a unique set of circumstances
which has to be tested in all variations.
As a result of the many interactions between remote
nodes, there are also more exception conditions,
which can be triggered. No test is complete without
raising all possible exceptions at least once if not
twice. In this respect the rollback, restart and
recovery functions of a system have to be tested, as
well as the multithreading features, not to mention
such conventional database functions as locking
and committing and guaranteeing integrity of data
content.
Another special aspect of the web architecture is
that of net security. As opposed to closed systems
running in a restricted environment, internet
applications are running in an open environment
where they are exposed to intruders. If they are not
protected by means of firewalls or other security
features, intruders may break in and intercept
messages, access databases or even distroy
application programs. These aspects only cover the
perimeter security. In reality, much of the security
expolits are coming from outside of the perimenter.
For example, buffer overflow will not be protected
by inward directed measures. Most of the attacks
will not be detected by a firewall. Many other forms
of attack are based on the principle ´that the client
side is not a controlled environment, and this is
where the application producer has the biggest
risks. In light of the fragility of the web, security
becomes a major issue. [7] There are software
techniques available to circumvent unwanted
access, but these techniques have to be retested for
every environment. This again adds to the total test
effort.
As a consequence of the many dependencies in the
web, more attention must be given to testing the
application software. This additional expense will
be enhanced by the costs of testing for novice,
unpredictable users. Business to business e-
commerce systems have more complex user
interactions, more objects to deal with and more
business rules to process, but here it is possible to
train the users and to oblidge them to work within
givin constraints. As employees their behaviour can
be controlled. They can even be held accountable
for irrational actions. In dealing with customers,
this is not the case. Customers are kings and can do
what they want. Therefore, anything they might
want to do has to be tested. This makes the costs of
testing business to customer applications much
greater than testing business to business systems. In
fact, this aspect is critical for the success of such
applications. [8]
In the end web testing costs are determined by the
size and complexity of the application plus the size
and complexity of the architecture plus the
complexity of the user interface and the degree of
Proceedings of the Sixth IEEE International Workshop on Web Site Evolution (WSE’04)
1550-4441/04 $ 20.00 IEEE
user freedom relative to the productivity of the
testers. The latter is driven by the knowledge,
experience, skill level, motivation, and, above all,
the tools of the testers. Without adequate tools they
cannot do their jobs.
2. A Dual Strategy for Testing Web
Applications
Testing on the web involves two distinct goals. One
is to test the compatability of the web application
with the technical environment. The other is to test
the functionality of the web application itself. The
effort required to test the functionality of an
application is driven by the size and complexity of
the application independent of what environment it
is running in. That effort is a factor of the number
of data attributes processed – arguments, results and
predicates, the number of dependencies between
interactive parts – inheritance, collaboration,
aasociation, etc., and the number of decision nodes
and branchs – selections, iterations, exceptions,etc.
The effort required to test the compatability of an
application with it‘s environment is, on the other
hand, driven by the complexity of the environment,
independent of the application. It is a factor of the
number of interfaces and dependencies between the
application and the environment. [9]
In the case of a web application this can be very
high, so that even trivial applications with little
programming effort require a significant testing
effort relative to applications on a host or classical
client/server systems. Web-based systems are
highly intertwined with the environment, which
they operate, e.g., they have a high number of
interdependencies with that environment, and it is
this high degree of dependency which makes them
so expense to test. It is mainly for this reason that
web applications cost two to three times more to
develop than conventional systems. This experience
was borne out by the project concerned here. The
costs of developing the new web application were
assumed to be 50% less than the costs of
developing the old mainframe CICS application or
8 person years as opposed to 16 person years for the
old application. As of now, and the new application
is still evolving, the costs have exceeded 25 person
years, or 50% more than what the older mainframe
application cost. Most of the cost overrun is due to
the additional test effort required.
Managers of web site development projects should
be aware of this fact. Using a sophisticated
development environment like DotNet or Java
Beans Enterprise Edition with a ready made
framework and many standard components, may
enable one to quickly produce a running system, but
it will still require a much longer period of testing
before that system becomes a deliverable product,
especially if the non-functional requirements such
 as security, reliability and efficiency are high. It
could even turn out that the first solution will never
be accepted and that they will have to start over
again. In any case, testing will prove to be a long
and tedious task with lots of unpleasant surprises,
something which has to be planned for from the
beginning.
The fact that many standard components are used in
internet applications might save the effort required
to produce them,, it will not save the effort required
to test them. Kaiser and Perry have demonstrated in
a land mark research paper on object-oriented
testing that one cannot assume that inherited or
reused classes will perform correctly in every
context, since they are all in some way context
dependent. [10] The same applies to components.
They too must be validated in each and every
context in which they are used. Referring to a
standard reusable method implies that each
reference has to be tested. If a reference is
embedded in a nested control structure it entails
fulfilling all of the conditions leading up to it. This
may sound easy, but if the condition predicates are
parameters or return values from other remote
methods, it can become extremely tedious. The
literature on object-oriented testing is abundant
with proofs of the need to retest reused software.
[11] Since web architectures offers a wide range of
reusable components, these insights are particularly
relevant to web-based applications.
3. Testing a Web Architecture
The architecture of the project in question is
DotNet. The programming language is C#. XSL
style sheets are used to depict the web pages. These
are then converted via XSLT to XML documents,
which are forwarded to the application server for
processing the SQL Server database. One could say
that this system is a complex structure with many
elements and even more interrelationships between
them. There are client components, server
components and database access components.
There are many standard components used,
including the MS-Explorer web browser SOAP as a
component connector, XSLT as an interface
transformer and the ODBC database access
components. There is also the MS-Transaction
Monitor. All of these components and their
interactions with one another had to be tested prior
to starting the actual application development since
there has to be a technical proof of concept and the
burden of proof is on the side of the user.
It is true that a web architecture provides a wide
range of useful functions, everything from search
machines to interface interpreters. One might
expect from the architecture that
long transactions are managed including
rollback, recovery and restart
Proceedings of the Sixth IEEE International Workshop on Web Site Evolution (WSE’04)
1550-4441/04 $ 20.00 IEEE
exceptions are intercepted and handled
events are caught and reported
messages are transferred and converted
data is stored, indexed and retrieved
memory is allocated and deallocated
intermediate states are logged
unauthorized entry is recognized and thwarted.
[12]
These are only some of the many functions required
to run a web application. It cannot be assumed that
they will all function as one expects. They have to
be tested and validated in the target environment.
That entails sending messages, causing events,
triggering exceptions, overloading memory,
disrupting transactions and attempting unauthorized
entry, all cases, which only experienced testers
would think of. Testing the architecture demands an
insider’s knowledge of the system as a whole.
Testers have to plan their test well, design a test
concept and specify all of the special cases they
intend to test, placing particular emphasis on
extreme values and boundary conditions.
Testing a web application architecture can be done
from outside-in over the user interface, i.e the web
page, or from inside-out starting with the data
server. If the outside-in approach is followed, it is
necessary to simulate the servers via test stubs. If
the inside-out approach is taken, it is necessary to
simulate the clients via test drivers. These two basic
approaches are definitely not new and are certainly
not unique to the web. They have been used both to
test host online applications as well as to test
client/server applications. [13] What is new are the
techniques for implementing them. In a web
environment drivers must generate messages in a
standard web format such as HTML or XML and
forward them to the object under test via a message
routing service. Stubs must, on the other hand,
intercept messages, i.e. prevent them from going to
the message router and generate an artificial
response in the form of standard communication
protocols. Doing this requires a good tool and
expert knowledge of web interfaces. In the project
described here, the tool Empirix was used to
generate messages to be sent over the web [14].
Another important aspect of testing the web
architecture is the ability to generate extreme
transaction loads which will push the system to its
limits. The system in question is a system for
collecting corporate taxes from companies. Thus,
millions of requests have to be generated in a
limited period of time. For this purpose special
tools are needed which can duplicate and mutate
web messages. Emprix was used for this purpose as
well, but the data produced was not really
representative of the actual production load,
probably because it was never intended for this
purpose.

Finally, the transaction management ability of the
web architecture has to be tested with selected long
transactions, which cause transactions to be raised
and the transaction to be rolled back. Data updates,
which have occured up to that point have to be
annuled, transient objects deleted, persistent objects
restored and allocated memory freed. If there are
several servers involved in handling one
transaction, as was the case here, this will prove to
be difficult. Yet since it will undoubtably come up
in production, it had to be tried.
In testing the web architecture there will always be
a certain degree of non-determinism to deal with.
This has to do with the evolving nature of the
components and the user’s lack of control over
them. One can never be sure that the browser one
tested yesterday is the one you are using today. It
might be the responsibility of the service providers
to sustain the same interface, but a user can never
be sure he is using it correctly. Interactions may
appear to be working when in fact they are only a
coincidence. The moment an error is corrected in
the server, the interface to that server may no longer
respond in the same manner. Then one is left
wondering why it ever functioned in the first place.
Web testers have to be aware of these architectural
inconsistencies and test to ensure that the client
software is prepared to handle rejected requests. It
needs to be checked that it will react accordingly to
any potential failure. Nowhere is Murphy’s law
more true than in the web. [15]
4. Testing a Web Application
The web application in question here was conceived
to interact with both accountants at the different
Chamber of Commerce offices at the state level –
there are nine states in Austria, each has its own set
of business rules - and with the companies and
persons being charged. Companies and self
employed persons liable for taxation, must fill out
web forms giving financial data on the status of
their company – the number of employees, the fixed
costs, investments, income, profit, etc. This
information is checked by the Chamber of
Commerce state offices, which then enhances it and
send out notices requiring the companies to pay a
given corporate tax. If they do not pay by a certain
deadline, up to five reminders are sent, after which
their data is forwarded to the courts for filing a
claim.
Due to the complexity of the system, it could not be
developed all at one time. Besides, most of the
users had never worked on the internet, so they had
no idea of what they wanted. They first had to be
shown working web pages before they could make
up their minds as to how they wanted them. The
Proceedings of the Sixth IEEE International Workshop on Web Site Evolution (WSE’04)
1550-4441/04 $ 20.00 IEEE
project turned out to be a long learning process,
both for the users and the developers. Here one can
really speak of evolutionary development. The web
site of the Chamber of Commerce has slowly
evolved to become that, what it is now, and it is still
evolving at an average rate of 18 new requirements
per month. This amounts to a change rate of about
3% per month.
4.1 Validating the XML Outputs
This presented a problem to the testers of the web
site. Since new versions of the software were being
released every month, there had to be a way to
repeat previous tests with a minimum of effort. The
solution chosen was to record the dialog tests with a
capture replay tool - Empirix - and to play them
back. However, to check the results another method
was used. The XML documents produced by the
server to be converted to XSL-Stylesheets and then
displayed to the user were intercepted upon leaving
the server and stored in a sequential file. Afterwards
the XML file generated by the test of the current
version is compared to that produced by the
previous version.
GU-
System
old
Reports REPTOXML
XML-
Files
Comparison
XMLComp protocol
Missing Objects,
Matching Objects,
Non-matching Results
GU-
System Test
XML-
Files
Figure 1: Comparing XML Output Files
The tool for making the comparison does so at the
element level, since the XML structures may vary.
First the old documents are read in and stored in a
direct access XML database. Then the new file is
read and processed document for document. From
the new document the key is taken to access the old
document. If it is not found, then the old document
is considered to be missing. If it is found the
elements marked for comparison are compared.
Those, which do not match are reported. Those old
documents found are marked so that at the end of
the run, the old file is read and all unmarked old
documents reported as missing in the new file. (see
Example 1)
A test is always a test against something else. To
determine whether data is correct or not, the user
must formulate assertions about the relationship
between one set of data and another. One such
 relationship is that a record with a certain key also
exists in the other file with the same key. Another
relationship is that, if the keys match then certain
fields, or attributes, must also match. Thus, a simple
first order predicate logic is required to describe the
relationships in a script file. This file has to be read
and checked. If the statements are not syntactically
correct, the execution of this particular file
comparison is terminated. If they are correct, the
assertions with keys and compare values are stored
in a table to be interpreted at compare time. The
assertion interpretation is a preprocessor and a
prerequisite to the file comparison. It is
automatically invoked for each assertion procedure
selected.
Assertions are, as used here in this context, claims
about the relationship between data items. [16] One
can claim that the value of an attribute in one object
should be =, < or > than the value of the same or
another attribute in another object, e.g. assert
Object_A.Attribute_1 = Object_B.Attribute_2.
One can also claim that the attribute of a given
object is equal to a constant, e.g
Object_C.Attribute_3 = “10”. The values must be
ASCI character strings. There is no comparison of
internal numeric data types, i.e integer, decimal,
double or floating, only character. It is assumed
that the values of a database are converted by the
SQL query to character format when they are
extracted from the database. The Assertion Script is
a text file. It must start with the name of the file to
be compared, e.g:
file: This_File;
The file name is preceeded by the word ‘file:’ in
lower case followed by at least one blank, then the
name of the file to be processed. This is done to
ensure that the correct file is being compared. The
file name must be the same as the name of the
assertion script, since it is the name that connects
the two.
After the file assignment declaration it is optional to
write when statements to replace the value of fields
in the files to be compared. As in all asserion
statements the keywords are in lowercase and are
separated from the variables by spaces. All field
names must be qualified by either new. or old. In
order to identify to which of the files they belong.
The when statement identifies the key field by
value and replaces the value of that key field or
another one by a given value, e.g.
when ( new.Field1 = “ ” ) then replace new.Field2
by “GT “;
when (old.Year = “00”) then replace old.Year by
‘99’;
There are some restrictions to this replacement
statement. The field whose value is queried and the
Proceedings of the Sixth IEEE International Workshop on Web Site Evolution (WSE’04)
1550-4441/04 $ 20.00 IEEE
field to be replaced must both be in the same file,
either in the old file or the new file. They may also
be the same field. The replacement statement
allows key values in either the old or the new file to
be overridden so that the keys will match.
The statement which connects the old XML objects
to the new ones is the if statement. The syntax is:
if ( <Condition> &
<Condition> &
<Condition> )
There can be up to 20 conditions. The conditions
actually identify the keys. The condition is a
comparison of a new key field with an old key field.
new.KeyField1 = old.KeyField1
The comparator can be =, > or <. The key field
names must correspond exactly to the titles of the
SQL table columns or the tags of the xml file. The
field names must not be the same in both files. One
can compare new.Field-B with old.Field-C. It is
also possible to match sub strings of the fields. In
this case the beginning position and the length of
the sub string must be in brackets, separated by a
colon, immediately after the field, e.g Field_A[2:8]
means the key begins with the second character of
Field_A and is 8 characters long.
There is also an OR clause in the condition. By
inserting the character ‘!’ followed by a spaces and
a constant, it is possible to say a field in the new file
must match the field in the old file or it matches to
the constant, e.g.
if (new.Field_A = old.Field_B ! “0” &….
That means either the new Field_A is = the old
Field.B or it has the value 0.
The conditions are included in parenthesis. The
closing parenthesis terminates the conditions.
For comparing XML files, the if statement may also
refer to an object tag to identify which object is to
be compared. This must be the first condition, e.g.
If (object = “Object_1” & new.Attr-A = old.Attr-
A)
When coimparing XML files, one object is
compared at a time. This means that the file will be
parsed as many times as there are objects in it. This
rather inefficient approach was taken to avoid the
problem of dealing with nested objects. The
prerequisite is that each object has a unique
identifier. In this way, even if objects are embedded
at a deeper level inside another object, they will still
be stored as individual objects in the database,
albeit with a concatenated key, which includes the
key of the higher level object, e.g.:
Object1_ID.Object12_ID.Object121_ID
 The assert statement is dependent on the if
statement, i.e, it will only be executed when the ‘if’
statement is true. Since the if statement is actually
linking the new object to the old object, there can
be no assert without an if.
If ( <Condition> & <Condition> )
assert <Comparison>,
<Comparison>;
The assert statement begins with the key word
assert followed by a sequence of comparisons. A
comparison compare two file attributes with each
other or an attribute with a constant, eg.
assert new.Field_A = old.Field_A,
new.Field_B = old.Field_C;
If other assertions are to follow, an assertion is
ended with a comma. It it is the last assertion, it is
ended by a semicolon ;. There can be any number
of assertions in a sequence. The operator can be
either =, > , >=,<=, or <. Here too it is possible to
compare substrings, e.g.
assert new.Field_1[3:5]
old.Field_1[10:5]
The assertion script is terminated by an end;
statement in the last line. This signals to the
interpreter that the script file is ended. If there are
several if statements, then each if statement is
terminated by an end; This is necessary because in
the case of XML there can be many objects in any
one file.
The following script was used to test the charges
file:
file: Vorschreibung;
if ( object = "Mitglied" &
new.MitgliedsNummer =
old.MitgliedsNummer )
assert
new.Name[1:8]=old.Name[1:8],
new.Datum[1:10] = old.Datum[1:10],
new.RechnungsBetrag =
old.RechnungsBetrag,
new.Ocr_Zeile[1:20]=
old.Ocr_Zeile[1:20],
new.Kundendatenfeld=old.Kundendate
nfeld;
end;
if ( object = "Forderung" &
new.ForderungsId[1:4] =
old.ForderungsId[1:4] )
assert new.Jahr = old.Jahr,
new.Betrag = old.Betrag,
new.Fachgruppe = old.Fachgruppe;
end;
end;
Example of an Assertion script for an XML Output
Proceedings of the Sixth IEEE International Workshop on Web Site Evolution (WSE’04)
1550-4441/04 $ 20.00 IEEE
Comparing the XML messages returned from the
server revealed some 72 errors in the output to the
end user. This accounts for 14% of the total errors
reported. Altogether some 520 errors have been
registered in the error tracking system “SCARAB”
as of now. Some 221 errors have been reported by
the users, which are testing parallel to the regular
testers. That makes up for 42% of the total errors
reported. The other 58% of the errors have been
reported by the testers. Of these 296 errors, 117
errors, or 45%, were revealed by the tools described
in this paper. The remaining errors were found
mainly in the dialogue test.
4.2 Validating the SQL Databases
The same tool used to validate the XML outputs is
also used to validate the status of the SQL
databases. In this case there are standard SQL select
statements, which extract the same views from both
the previous and the latest database tables. These
views are then stored as comma seperated value
files. It is important to note that this preselection is
done so as not to compare the entire content of the
<= database tables, but only those attributes, which are
considered to be crucial. The old csv file is
compared with the new csv file using the assertions.
Tuples of the database can be missing in the old
database, missing in the new database or not
matching. (see Example 2)
The file comparison is made here in four steps. In
the first step, the records of the base file, the old
data, are read sequentially and stored in a random
file. While reading, key values may be altered to
match the new keys. For this purpose the replace
statement is used. The keys are extracted and stored
in an indexed table. The data fields of the old record
are stored in a random file. In the second step, the
index table is sorted so that the keys are in
ascending order and can be accessed directly. In the
third step, the compared file is read sequentially and
with the key of each record an access is made to the
index table. Here too key values may be altered to
match the old keys. If the key is not found, even not
by alternating the keys, this record is assumed to be
missing in the base file. If the key is found, the base
record is retrieved from the random file and its
selected contents are compared with the contents of
the current record. Field values that do not match
are documented in the compare report. In the fourth
step, the indexed file is searched thru for all keys
which were not selected. Those that were not
selected are reported as being missing in the
compare file.
As with the XML files, the user gets a report on
missing records and non-matching attributes as well
as some statistics as to the degree to which the two
files coincide. This is equivalent to a correctness
metric.
 Comparing the database tables of the last release
with the database tables of the new release proved
to be an effective means of detecting erroneous data
in the database. The database comparison has
revealed more than 45 serious errors or 9% of the
total number of errors reported. The severity of
these errors was, however, much greater than that
of the errors discovered in the user interface.
GUN-
GUN
SQL
System Test
DB
process and to enforce that process rigidly. The
process must state explicitly the test steps to be
taken and it should be supported by tools. The two
key tools are
a capture/replay tool and
a comparison tool
The capture/replay tool serves to record and play
back the user interface test cases. The comparison
tool should be able to compare both the system
outputs and the system databases. Both tools have
to be built into a well defined test process. [17]

The managers of the project have complained about

the fact that 42% of the errors have been reported
SQL- CSV- Fields which map to the
Query Files Fields of the old Database by the users. They claim, the testers should have
found more. This may be true – there were only
three testers, as compared to 11 testing users - but
GU- Comparison
VSAM
System
Files
TabComp protocol without the testing tools, even less errors would
old
Missing Records,
have been found by the testers. It is very difficult to
Matching Records,
Non-matching Fields
quantify the value of errors found, so this will
FileTran
CSV-
Files
always be a bone of contention. As a tester one is
always left with the frustrating feeling, one could
have done more.
Figure 2: Comparing SQL Tables
Nonetheless, in the project described here it would
have been impossible to retest every new release
5. Lessons learned without the availability of the tools. Therefore,
since web site evolution is a test driven activity and
The main lesson learned from this project is that the test is dependent upon the availability of tools,
web site evolution is a test driven activity. It is test tools are the key to a successful web evolution.
important to have a precise and practical test

References:
1] Poore,J and Trammel,C.: “Bringing Respect to Testing through Statistical Science“, American
Programmer, Vol. 10, No. 8, August, 1997, p. 15-22
2] Gelperin,D. and Hetzel,W.: “The Growth of Software Testing“, Comm. Of ACM, Vol. 31, No. 6, June, 1988,
p. 687-695
3] Beizer,B: “Testing Technology – The Growing Gap“, American Programmer, Vol. 7, No. 4, April, 1994, p. 9
4] Goglia,P.: Testing Client/Server Applications, QED Publishing Group, Boston, 1993, p. 25
5] Downs,T.: “Extensions to an Approach to the Modeling of Software Testing with some Performance
Comparisons“, IEEE Trans. on S.E., Vol. 12, No. 9, Sept., 1986, p. 979-987
6] Bazzana,G./Fagnoni,E.: “Testing Web Applications“ Proc. of ICSTEST – Int. Conf. on Software Testing,
Software Quality Systems, Cologne, April, 2000, p. 65-74
7] McDonald,M.: “From manual Commerce to e-Commerce“, in Cutter IT-Journal, Vol. 13, No. 4, April, 2000,
p. 12-24
8] Becker,S./Berkemeyer,A./Zou,B.: “A Goal-Driven Approach to Assessing the Usability of an e-Commerce
System“, Cutter IT Journal, Vol. 13, No. 4, April, 2000, p. 25-34
9] Sneed, H. “Testing Software for Internet Applications“, Software Focus, Vol. 1, No. 1, Sept. 2000, p. 15-22
10] Perry,D./Kaiser,G.: “Adequate Testing and object-oriented Programming“, Vol. 5, No. 2, Jan. 1990, p. 13-19
11] Kung,D./Hsia,P./Gao,J.: Testing object-oriented Software, IEEE Tutorial, Los Alamitos, CA., 1998, p. 89
12] Nguyen,H.Q.: “Testing Web-based Applications“, Software Testing & Quality Engineering, Vol. 2, No. 3,
May 2000, p. 23-30
13] Siegel,S.: Object-oriented Software testing – A Hierarchical Approach, John Wiley & Sons, New York,
1996, p. 235
14] Fewster,M./Graham,D.: Software Test Automation, Addison-Wesley, New York, 1999, p. 248
15] Anderson,M.: “The top 13 Mistakes in Load Testing Web Applications“, Software Testing & Quality
Engineering, SQE Pub., Vol. 1, No. 5, Sept. 1999, p. 30-41
16] Binder,R.: Testing Object-oriented Systems, Addison-Wesley Longman, Reading Mass., 1999, p. 810-832
17] Kaner, C., Falk, UJ., Nguyen, H.-Q.: Testing Computer Software, Wiley & Sons, New York, 1999, p. 177

Proceedings of the Sixth IEEE International Workshop on Web Site Evolution (WSE’04)
1550-4441/04 $ 20.00 IEEE
 Samples:
+------------------------------------------------------------------------+
| Object: Report XML Comparison Report Date: 04.04.04 |
| Type : XML System: REPORTS |
| Key Fields of Record(new,old) |
+------------------------------------------------------------------------+
| Non-Matching Fields | Non-Matching Values |
+-----------------------------------+------------------------------------+
| RecKey:4711 | |
| New: Zip | 39508 |
| Old: Zip | 39507 |
+-----------------------------------+------------------------------------+
| RecKey:4711 | |
| New: Total_Costs | 364.25 |
| Old: Total_Costs | 363.75 |
+-----------------------------------+------------------------------------+
| Total Number of old Records checked: 10 |
| Total Number of new Records checked: 10 |
| Total Number of Fields checked: 15 |
| Total Number of non-Matching Fields: 10 |
| Percentage of matching Fields: 67 % |
| Percentage of matching Records: 100 % |
+------------------------------------------------------------------------+
Example 1: XML Comparison Report
+------------------------------------------------------------------------+
| Object: Bewegung File/Table Comparison Report Date: 21.06.04 |
| Type : CSV System: TEST |
| Key Fields of Record(new,old) |
+------------------------------------------------------------------------+
| New:MGNR BNR TYP JAHR |
| Old:MitgliedNr BerechtigungNr Type Jahr |
+---------------------------------+--------------------------------------+
| Non-Matching Fields | Non-Matching Values |
+---------------------------------+--------------------------------------+
| RecKey:238132 1 GU 0 | alternate key in new File/Table |
+---------------------------------+--------------------------------------+
| RecKey:254036 1 GU 2000 | missing from the old File/Table |
+---------------------------------+--------------------------------------+
| RecKey:238132 0 MK 2003 | |
| New: VS | 0000000000001.45 |
| Old: VS | 0000000000002.00 |
+---------------------------------+--------------------------------------+
| RecKey:238132 0 MK 2003 | |
| New: OP | 0000000000001.00 |
| Old: OP | 0000000000000.00 |
+---------------------------------+--------------------------------------+
| Total Number of old Records checked: 10 |
| Number of old Records found in new File: 02 |
| Number of old Records not in new Table: 07 |
| Total Number of new Records checked: 13 |
| Number of new Records found in old File: 03 |
| Number of new Records not in old File: 08 |
| Total Number of Fields checked: 09 |
| Total Number of non-Matching Fields: 02 |
| Percentage of matching Fields: 78 % |
| Percentage of matching Records: 38 % |
+------------------------------------------------------------------------+
Example 2: SQL Comparison Report

Proceedings of the Sixth IEEE International Workshop on Web Site Evolution (WSE’04)
1550-4441/04 $ 20.00 IEEE
 This document was created with Win2PDF available at http://www.daneprairie.com.
The unregistered version of Win2PDF is for evaluation or non-commercial use only.

Proceedings of the Sixth IEEE International Workshop on Web Site Evolution (WSE’04)
1550-4441/04 $ 20.00 IEEE