Au Green Software PDF

Making software testing greener
Skill Level: Introductory
Nageswararao V. Gokavarapu (nagesh.gokavarapu@in.ibm.com)

Software Test Specialist
IBM
11 Aug 2009
Recently there has been an increased emphasis on the effective utilization of

resources. Testing plays a key role in any software development lifecycle. Hence,
effective utilization of resources in testing becomes the prerogative for any product
team. Read about the effective utilization of resources available for testing on both
hardware and software, thus leading to more efficient testing.
Introduction
There are two software testing methods, black-box testing and white-box testing,
which are designed to cater to the Functional Requirement Specification (FRS) and
System Requirement Specifications (SRS). SRS contains the complete description
and behavior of all the functions of software to be developed. FRS contains the
expected outputs for list of inputs for each function of the software.
In white-box testing, the test engineer has access to the internal data structures and
algorithms.
Black-box testing treats the software as a "black box," without any knowledge of
internal implementation and the test engineer will perform tests on the software as a
customer would use the product. System testing is a method of black-box testing
and is conducted on a complete, integrated system to judge the software compliance
in the context of a FRS and SRS. The different types of testing that should be
considered during product testing are:
• Installation testing
• Build verification testing

© Copyright IBM Corporation 2009. All rights reserved. Page 1 of 18
developerWorks® ibm.com/developerWorks
• Functionality verification testing

• Usability testing
• GUI software testing
• Security testing
• Load testing
• Stress testing
• Performance testing
• Scalability testing
• Regression testing
Each of these testing methods ascertains the software product quality in different
functional areas.
Virtualization is a modern approach for enhancing a system's ability to complete

tasks. Virtualization can be adopted in different areas in the computing world like
hardware virtualization, operating system virtualization, network and storage
virtualization, presentation virtualization, and application virtualization. Operating
system-level virtualization is a server virtualization method where the kernel of an
operating system allows for multiple isolated user-space instances. Each of these
instances may look and feel like a real server from the point of view of its end user.
Each of these instances are isolated and the kernel often provides resource
management features to limit resources for every instance. This article address how
to adopt operating system virtualization methods in different types of testing and
thereby minimizing the utilization of resources in terms of hardware, software, and
test engineers.
Types of software testing

The following are some examples of testing that will be performed during the testing
phase of a software product.
Installation test
In this test, the test engineer installs and sets up the software. The testing process
may involve full, partial, or upgrades to the installation/removal of the product. The
simplest installation approach is to run an install program, sometimes called
Package Software. The test engineer should perform every possible configuration
with every mode of installation and should receive a better quality of testing so that
the software can be released to customers with confidence.

Page 2 of 18 © Copyright IBM Corporation 2009. All rights reserved.
ibm.com/developerWorks developerWorks®
Build Verification Test (BVT)
This test involves the basic functional tests on the product to make sure the code
changes by the development team in that particular build will not affect or regress
any pre-existing features in the product. Once this test is passed successfully and
the build is certified, then the functionality of the product is tested by the other
methods of testing. As part of globalization, the BVT will be executed in supported
locales of the software product that is available.
Functional Verification Test (FVT)
The purpose of this test is to check that a product conforms to its functional
specification. As part of FVT, the functionality of the product is tested for all its
supported features and then certifies the product readiness for the system test.
In Globalization Verification Test (GVT), the FVT is also tested in all supported
locales to make sure the product functionality is consistent.
GUI software test
The GUI test tests the software that supports the graphical user interface to ensure it
meets the required specifications in FRS. Most products deliver system functionality
to the clients using a GUI. The test designer must take care that the test suite covers
all the functionality of the product with GUI, and also ensure that the suite fully
exercises the additional functionality provided by the GUI. The GUI test also needs
to be done in all supported locales and the test engineer needs to verify the
standards defined for different objects in GUI, for different locales.
Load test
In load test, multiple users access the program concurrently to simulate the
expected usage of a software program. To simulate the load, vendor-acquired
software may be used. As many users access the application or product
concurrently, the CPU usage, network or disk I/O operations, and disk space
requirements are greater for these type of tests.
Stress test
Stress test refers to tests that put a greater emphasis on robustness, availability, and
error handling under a heavy load on the product. The main goal of these tests is to
ensure that the software does not crash in conditions of insufficient computational
resources such as memory or disk space. It also determines the stability of the
product by testing beyond normal operational capacity. As the product is tested in
heavy load conditions, no other testing can be done on the server.
Regression test

This test reveals the software regressions. Regressions are determined whenever a
functionality that was previously working correctly stops working as intended with
new test run. Typically, regressions occur as an unintended consequence of
program changes during a bug fix. Regression testing include re-running previously
run tests and checking whether the previously fixed issues have occurred.
Regression testing is used to determine the correctness and quality of the product.
The different types of tests discussed here require different environments to run the
test. Some of the test types require less CPU and disk space, whereas some of the
test types requires high CPU usage, I/O operations, and disk usage. So, the
resource utilization of a particular test type depends on the sequence of tests
performed in that test type. Adopting operating system virtualization for all test types
is not possible because of dependencies. The following section describes how to
choose the test area.
Choosing the test area

The following general questionnaire helps in identifying the test area to use
operating system virtualization technologies to minimize system and human
resources.
General questionnaire for choosing test type:
• How much CPU and memory allocation is required?

• How many processes will run for executing the test?
• How much virtual memory is required for running the test?
• Does the test require any physical devices?
• Will the test require modifying the kernel parameter at runtime?
• Will the test access raw or routing sockets?
• Does the test require mount/dismounting file system?
Inside the operating system virtual instance, some facilities like mounting the file
system, reading/writing the data to raw sockets, and changing the kernel parameters
dynamically are not available. The file system mounting can be done from the global
system environment to a particular OS instance. The virtual OS instance can access
the mounted file systems without any issues, but direct mounting from OS instance
is not possible. Similarly, accessing the physical devices is not possible from virtual
instances. Even though there is a way to access these devices by exporting from the
global environment, applications that do not support these exported devices will fail.
So, based on the resources required for test types, the virtual opersting system
techniques are adopted for product testing.

Example test areas for consideration
As discussed previously, the product testing involves multiple test types and each of
the test types needs different hardware, software, and network resources. For
example, load test, stress test and performance tests, where the product is tested
under high load and each of these tests uses the same system resources from the
system. So, these tests should not be run on the same physical server at the same
time. If these tests are running on different virtualized instances on a physical server,
you need to be careful when configuring your system for allocating resources for
these operating system-virtualized systems. The load tests require more resources,
and all will share from the same physical server, so these benchmark tests should
not be performed on operating system-virtualized systems.
For installation test, build verification test, globalization verification test, and GUI
software test, you can use the operating system virtual instances. In installation test,
the product is tested in all supported modes of installation. Most of the time, the
installation tests are executed in a freshly installed operating environment. Once the
product is installed, it will add a set of configurations in the system environment.
Even though the product is removed after testing, some of these environments will
remain in the system. The subsequent installation on the same machine will use the
existing configurations. Sometimes this will hide the installation issues that can be
noticed in a freshly installed operating system environment. But practically speaking,
it is not possible to perform each of the installation test scenarios in freshly installed
operating system environments, because it includes additional operating system and
third-party prerequisite software setup in the server. As part of globalization, the
installation test is done in all supported locales. So, the number of machines
required will be more. The operating system virtualization technology helps for
installation tests. Once the operating system and other third-party prerequisites are
installed in the physical server, you can make as many as virtual operating systems
instances and all of these will have the same system operating environment as the
global physical system. Creating these virtual instances will take less time.
Table 1. Different test areas with effort in conventional methods

Test type Number of servers Administration Test effort
required effort
Install tests 2 4x 12x
BVT 2 4x 12x
FVT 3 6x 24x
GUI tests 2 4x 12x
Table 1 shows different test areas, their server usage, and the administrator effort
needed for the operating system, software setup,and testers effort in configuring and
executing the test cases where x is the unit of time.

Most of the products support a minimum of three modes of installation modes like
GUI, Console, and Silent. For installation test, if the product is tested in all modes
with all supported locales (assume the product supports 10 locales), then installation
must be tested in 30 various configurations. Similarly, BVT/FVT needs to be run in
all locales to certify the product functionality. By using operating system virtualization
technologies, the administration and test efforts are reduced, as shown in Table 2.
The administration effort is reduced as the number of servers used is less and the
test effort reduced, because operating system virtual instances contain all the
software installed in a global system. Hence, there is no need to install
vendor-acquired products inside each instance. It also saves power and contributes
less heat, thus reducing the investment required to cool the systems.
Table 2. Different test areas with efforts required using OS virtualization

methods
Test type No of servers Administration Test effort
required effort
Install tests 1 2x 8x
BVT 1 2x 8x
FVT 1 3x 12x
GUI tests 1 2x 4x
Another test area where virtualization methods is used is GVT. As part of the GVT,
BVT and FVT execute in all supported locales. If the product is installed in a global
physical system and the virtual instances are created inside, then there is no need to
install the product inside each instance. Only some additional configuration needs to
be done to execute these test suites. Even if the product image refreshes in the
global system, it reflects immediately in all virtual instances. Hence, it reduces the
test efforts.
Operating system-level virtualization technologies
Virtualization is done in different levels and each virtualization technique has a

different advantage based on the usage. For product testing, hardware and
operating system-level virtualization helps in reducing the number of physical
servers required and time required to set up the servers. In operating system
virtualization, different isolated instances of the operating system are created and all
share the physical resources available in the system, as shown in Figure 1. The
maximum resource usage for a each instance can be configured at the time of
creation. This article discusses the different server virtualization methods to be used
in product testing.
Figure 1. Operating system virtualization

There are different operating system virtualization technologies available like AIX®
V6.1 workload partitions (WPAR) and Solaris 10 containers.
AIX 6.1 provides a WPAR facility that provides an application-isolated environment

and the resource control for applications. WPAR monitors the processes running on
a particular WPAR from the global environment. Different automated test scripts can
be added in the startup profiles of the WPAR. Once the WPARs are started, all the
automated tests are executed on different WPARs simultaneously. Manual tests can
also be executed on each WPAR if needed. Using shared-system WPAR
configuration, there is no need to install the product on each WPAR. The test suites
will use the product installed in the global environment to run different test areas of
product testing. Once the tests are started, Global environment can monitor the
status logs for the initiated tests. Some of the test areas needs minimum system
resources and these tests can be run at the same time. This saves the resources,
mainly in terms of servers and cost.
Solaris Container allows completely isolated virtual servers within a single operating
system instance called zones. The zone facility allows sharing the system resources.
Each zone behaves as a separate system. The similar virtualization technologies are
also available on other operating systems like HP-UX, Linux and Windows.
WPARs in AIX 6.1
AIX 6.1 supports two types of workload partitions, Application WPAR and System
WPAR. Application WPAR creates and starts with the application in an isolated
resource environment and automatically removes when the application terminates.
System WPAR is a complete virtualized opersting system environment running

multiple services and applications. There are two different types of System WPAR:
• Shared system WPAR

• Non-shared (private) system WPAR.
Each of these behaves as a completely isolated operating environment.
In shared system WPAR, it uses the software installed in the global system and in
private WPAR it allows for installation of the product inside the WPAR.
1. Create a WPAR
You can create System WPAR by doing the following steps:
1. Log in to an AIX 6.1 system as a root user. Note that this is for a
global system in a WPAR environment.
2. Select the WPAR name that you want to create.
3. If you want to mount the WPAR file systems on a specific mount

point, create a directory; otherwise, by default WPAR uses the
/wpars directory.
4. If you have created a directory, then change the directory

permission to 700 (that is, drwx------).
5. Add the IP address of the WPAR in the /etc/hosts file.

For example:
123.456.789.128 txwpar.domain_name txwpar # IP address of WPAR
where:
123.456.789.128 is the IP address of the WPAR.
txwpar is the WPAR name.
6. Run the mkwpar command as shown in Figure 2 to create a shared

system WPAR.
Figure 2. Creating shared system WPAR

In the example shown in Figure 2, above, txwpar is the WPAR name.
-r is copy global network name resolution configuration into the workload.

This copies the following:
/etc/resolve.conf
/etc/hosts
/etc/netsvc.conf
/etc/irs.conf
/etc/networks
-N is network settings ( [-N attr=value ...] )
address= 123.456.789.12 is the IP address
interface = en0
To find the interface number, run the ifconfig -a command. For example:
# ifconfig -a en0:
flags=5e080863,c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,
GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),PSEG,LARGESEND,CHAIN>
inet 123.456.789.126 netmask 0xffffff00 broadcast 123.456.789.255
tcp_sendspace 131072 tcp_recvspace 65536
netmask=255.255.255.0 is the netmask value
For non-shared (private) WPARs only, you can use the -l option. This
creates a private, writable version of /usr and /opt.
Upon successful creation, it shows a successful message as shown in

Figure 3. It also updates the network configurations to access the WPAR
from outside.
Figure 3. WPAR creation

2. Start the WPAR:
startwpar -v WPAR_name
where -v is verbose and -m is maintenance mode. Once the WPAR is

started successfully, as shown in Figure 4, it returns the status as a
success.
Figure 4. Start WPAR and check the WPAR status
3. Check the status of the WPAR:

In Figure 4, the running status of WPAR is known by using lswpar
command, where S means that txwpar is a system WPAR and A means
that txwpar is active.

4. Log in to the WPAR:
clogin txwpar.
5. Now WPAR is ready to use for testing.

As it is a shared system WPAR, it contains all the software installed in the
global system. Assume that two WPARs are created and WPAR1 FVT
and WPAR2 BVT are configured as shown in Figure 5. Both tests can
start in parallel. Once it checks and makes sure that all test case
execution is completed inside each WPAR, it can start the next test case.
The results are written in a test log file inside the WPAR. The application
running on AIX 6.1 system keeps on monitoring these logs and takes the
necessary steps. In case of GVT, the same WPARs can be used to run
the BVT and FVT in different locales by setting the locale one after the
another. As a result, it saves lots of resources like servers, power
consumption, installation, and configuration of different products on each
server, test execution time, and server administration.
Figure 5. Product testing with WPAR
Zones in Solaris
The Solaris Zones feature provides the facility to create separate operating system
environments for running applications with dedicated resouce allocation like CPU
and memory. There are two types of zones, global and non-global. A server on
which the zones feature is enabled has one global zone and up to 8191 non-global
zones. Non-global zones contains subset of the Solaris operating installed in the
global zone and some additional packages. Each non-global zone is completely

isolated, has its own package database, and does not share package information
with the global or other non-global zones.
1. To create zones, you need to have an IP address for each zone that you
configure.
• Connect to the Solaris 10 system as an administrator.
• Choose the name for the zone.
1. Create a zone using the zonecfg command and follow the

steps as shown in Figure 6.
Figure 6. Create a Solaris zone
2. Set the IP address and physical network card interface ID

for the zone created as shown in Figure 7. Get the additional
information about the zone by issuing the info command.
2. Enter the ifconfig -a command to get the interface card name.

Figure 7. Configure the zone with network interface

3. Once the zone is created successfully, install the zone. This will copy the
different softwares installed from the global zone to the newly created
zone. The time taken for installing the zone depends on the number of
packages installed in the global zone, as shown in Figure 8. Once the
zone is installed, it shows the following output: Successfully
installed zone <zone name> ".
Figure 8. Install the zone once it is created
4. Now you are back to the normal login system prompt. To check the zone

status, run zoneadm list -cv .

If the value in the ID column against the zone name is '-', it means the
zone is created and installed but not running. Run the boot command, to
start the zone and the ID column against zone name will show '1,' as
shown in Figure 9.
Figure 9. List all zones from the global zone
5. For allocating resources to zones, use zonecfg from the global zone and
then specify the resources for the particular zone. If you are creating
multiple zones, share the resources appropriately, as shown in Figure 10.
In this example, zonecfg is allocating 40% CPU to the txzone.
Figure 10. Allocating resources to zones
6. Reboot the zone to effect the resources allocated for a particular zone.
Using prctl from the global zone, the CPU share for global zone can
also be defined. The following example defines 60% CPU share to the
global zone.
prctl -r -n zone.cpu-shares -v 60 -i process $$.
prctl -i zone global
7. Once the zone is created and resources are allocated, connect to the
zone for setting up additional configurations, such as setting up the
password, DNS name, IP address, Locale, and the like. Connect to the

zone using the zlogin command, as shown in Figure 11. Once you log
in inside the zone, the global zone is not visible. To verify, run uname -a.
To log out from the zone enterthe ~.(tilde dot) command.
Figure 11. Login to the zone
8. Now the zone is ready with the required configuration. The newly created
zone is completely isolated from other the zone as well as the global
zone. To the end user, it looks like as a separate Solaris system. The
Solaris zones are very useful for product installation tests, GVT tests, and
resource manager tests.
Figure 12. Solaris zones
In Figure 12, three zones are configured inside the global zone and
resources allocated as 15% CPU share for zone 1 and zone 3. For zone
2, a CPU share of 25% is allocated from the global zone. All of these

zones, including the global zone, use the same kernel to access the
hardware available in the physical system. There are some test
applications running inside zone 2 and the database servers are also
configured inside this zone. The database clients installed in zone 1 and
zone 3 will connect to zone 2 to execute the database-based test
applications.
Conclusion
Adopting virtualization technologies saves resources like servers, installation and
configuration of different products on each server, test execution time, and also
manpower. It also saves the administration effort to maintain the servers.
On AIX WPAR, even if one server goes down for maintenance, it is easy to move
the WPARs from one server to another server with minimal effort, using the
workload partition manager.
With all these advantages, adopting virtualization techniques helps the testing
environment to move closer towards green IT.

Resources
Learn
• The AIX 6.1 Information provides technical details about AIX Operating system.
• See Introduction to Workload Partition Management in IBM AIX Version 6.1 to
understand AIX 6.1 WPAR operating system virtualization capability.
• Workload Partition Management in IBM AIX Version 6.1 provides information of
enhancements on WPAR with AIX 6.1 TL2.
• The AIX and UNIX developerWorks zone provides a wealth of information
relating to all aspects of AIX systems administration.
• Stay current with developerWorks technical events and webcasts.
• Browse the technology bookstore for books on these and other technical topics.
Get products and technologies
• Download IBM product evaluation versions or explore the online trials in the
IBM SOA Sandbox and get your hands on application development tools and
middleware products from DB2®, Lotus®, Rational®, Tivoli®, and
WebSphere®.
Discuss
• Check out developerWorks blogs and get involved in the developerWorks
community.
• Participate in the AIX and UNIX forums:
• AIX Forum
• AIX Forum for developers
• Cluster Systems Management
• IBM Support Assistant Forum
• Performance Tools Forum
• Virtualization Forum
• More AIX and UNIX Forums
About the author

Nageswararao V. Gokavarapu

Nageswara Rao is a Post-graduate student from Andhra University and joined IBM in
2006 as Software Engineer. He is presently at the India Software Labs, IBM, working
with the TXSeries test team and has expertise in designing and developing test
applications for middleware like TXSeries on UNIX-like systems. His areas of
interests are vVirtualization, transaction processing management, and UNIX systems.


Au Green Software PDF

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Au Green Software PDF

Uploaded by

Copyright:

Available Formats

Making software testing greener

Skill Level: Introductory

Nageswararao V. Gokavarapu (nagesh.gokavarapu@in.ibm.com)

Recently there has been an increased emphasis on the effective utilization of

Making software testing greener

• Functionality verification testing

Virtualization is a modern approach for enhancing a system's ability to complete

Types of software testing

Making software testing greener

Build Verification Test (BVT)

Functional Verification Test (FVT)

GUI software test

Making software testing greener

Choosing the test area

General questionnaire for choosing test type:

• How much CPU and memory allocation is required?

Making software testing greener

Example test areas for consideration

Table 1. Different test areas with effort in conventional methods

Making software testing greener

Table 2. Different test areas with efforts required using OS virtualization

Operating system-level virtualization technologies

Virtualization is done in different levels and each virtualization technique has a

Figure 1. Operating system virtualization

Making software testing greener

AIX 6.1 provides a WPAR facility that provides an application-isolated environment

WPARs in AIX 6.1

Making software testing greener

• Shared system WPAR

2. Select the WPAR name that you want to create.

3. If you want to mount the WPAR file systems on a specific mount

4. If you have created a directory, then change the directory

5. Add the IP address of the WPAR in the /etc/hosts file.

123.456.789.128 txwpar.domain_name txwpar # IP address of WPAR

123.456.789.128 is the IP address of the WPAR.

txwpar is the WPAR name.

6. Run the mkwpar command as shown in Figure 2 to create a shared

Making software testing greener

In the example shown in Figure 2, above, txwpar is the WPAR name.

-r is copy global network name resolution configuration into the workload.

Upon successful creation, it shows a successful message as shown in

Figure 3. WPAR creation

Making software testing greener

2. Start the WPAR:

where -v is verbose and -m is maintenance mode. Once the WPAR is

Figure 4. Start WPAR and check the WPAR status

3. Check the status of the WPAR:

Making software testing greener

4. Log in to the WPAR:

5. Now WPAR is ready to use for testing.

Figure 5. Product testing with WPAR

Making software testing greener

1. Create a zone using the zonecfg command and follow the

2. Set the IP address and physical network card interface ID

2. Enter the ifconfig -a command to get the interface card name.

Making software testing greener

Making software testing greener

status, run zoneadm list -cv .

Figure 9. List all zones from the global zone

Making software testing greener

Making software testing greener

Making software testing greener

About the author