Mobile Application Testing

Mobile applications are booming,so mobile apps are big business, but with
this hyper-growth comes the challenge of releasing apps that are stable on
multiple portable devices and platforms. Unfortunately, when it comes to
mobile apps, quality assurance can be a complex, time-consuming and costly
task. This can be especially true if you rely only on manual testing or on
traditional tools for testing.
Mobile Application testing is a process in which Mobile applications are
tested for its functionality,usability and consistency.
Mobile devices differs from screen sizes,input methods with different
hardware capabilities.There are different Mobile Operating Systems in the
market. The major ones are Android, IOS, BREW, BREWMP, Symbian,
Windows 7.5 Mango, and Blackberry (RIM). Each operating system has its
own limitations. Testing a single application across multiple devices running
on the same platform and every platform poses a unique challenge for
testers.
There are over 400 mobile network operators in the world;out of which
some are CDMA(Code division multiple access), some GSM(Global system
for Mobile communications).Each network operator uses a different kind
network infrastructure and this limits the flow of information.
As devices differ in keystrokes, input methods, menu structure and display
properties a single script doesnt function on every device.

Types of Mobile Application Testing

Functional Testing: Functional testing ensures that the application is
working as per the requirements.
Laboratory Testing: Laboratory testing, usually carried out by network
carriers, is done by simulating the complete wireless network. This test is
performed to find out any glitches when a mobile application uses voice
and/or data connection to perform some functions.
Performance Testing: This testing process is undertaken to check the
performance and behavior of the application under certain conditions such as
low battery, bad network coverage, low available memory, simultaneous
access to applications server by several users and other conditions.
Performance is checked on both client and server side.
Memory Leakage Testing:Memory leakage happens when a computer
program or application is unable to manage the memory it is allocated
resulting in poor performance of the application and the overall slowdown of
the system. As mobile devices have significant constraints of available
memory, memory leakage testing is crucial for the proper functioning of an
application.
Interrupt Testing: An application while functioning may face several
interruptions like incoming calls or network coverage outage and recovery.
The different types of interruptions are:

Incoming and Outgoing SMS and MMS

Incoming and Outgoing calls
Incoming Notifications
Battery Removal
Cable Insertion and Removal for data transfer
Network outage and recovery
Media Player on/off
Device Power cycle

 Usability testing: It is carried out to verify if the application is achieving its

goals and getting a favorable response from users as the usability of an
application is its key to commercial success.
Installation testing: Certain mobile applications come pre-installed on the
device whereas others have to be installed from the store. Installation testing
verifies that the installation process goes smoothly without the user having
to face any difficulty. This testing process covers installation, update and
uninstallation of an application.
Certification Testing: To get compliance certificate each mobile device
needs to be tested against the guidelines set by different mobile platforms.

Mobile Application Testing Tools

1. eggPlant:
eggPlant is a GUI(Graphical User Interface) based automated test tool
delivering a robotic solution to mobile testing across all operating
systems and devices.
eggPlant delivers a robotic solution to mobile application testing
across all operating systems and devices. Using RealVNC, generic
VNC or emulators, eggPlant deploys patented intelligent image
recognition technology to see any device display. This approach to test
automation is easy to use, non-invasive and requires no special
hardware.
Write one script and use this script to test across multiple devices and
emulators. eggPlant can be used in a continuous integration
environment, for regression testing and provide statistics in a load
tested environment. eggPlant provides a scalable and flexible solution
for all functional testing and data driven testing objectives.
With eggPlant mobile application testing solution QA test automation
engineers can :

1. Test on real mobile devices

2. Test across mobile OS with a single script

3. Test within your firewall

As a result, testing applications with TestPlant is much faster than using

traditional test tools, meaning robust mobile applications can be brought to
market very quickly. In addition, it brings great manual test productivity, so
costs can be dramatically reduced.
Link for eggPlant testing Smart phone:
http://www.youtube.com/watch?v=fHzPlJXici4&feature=player_embedded
Cross Platform Testing.
eggPlant scripts work regardless of the environment you are testing.
Supported platforms include:
a. Windows 95, 98, 2000, NT, XP, Vista, and Windows 7
b. Mac OS X
c. LINUX
d. Solaris
e. HP-UX
f. Even BIOS and other hardware devices
Here is a link for eggPlant in action:
http://www.youtube.com/watch?v=M8-8UkKSGG8&feature=player_embedded

Rich Internet application testing.

I.

HTML5:Its is the new RIA and growing rapidly. With growing adoption in
mobile app development HTML5 is set for great growth. But testing it
remains a challenge. But with eggPlant you can test it as any other RIA
application.

II.

Adobe Flex:Because eggPlant doesnt rely on code hooks for application

testing, its perfect for GUI testing graphic rich applications designed and
built in Flash where no code-level tool integration is possible.

III. AJAX:As

eggPlant does not rely on objects, testing AJAX functionality is

simple and easy as well as allow for performance metrics to be used.

IV.

Silverlight:With eggPlant, scripting a Silverlight application is no different

from scripting any other application.

V.

Java:As GUIs are developed using Java with Swing or SWT. Testing them
for functional validation becomes more important.With eggPlant you can
conduct diversified GUI testing.

For more information on how it works,click on the following link:

http://www.youtube.com/watch?v=TthpZ45wdEw&feature=player_embedded
Performance Testing using eggPlant
i. Application Performance Testing is conducted to determine if an
application, system or component meets such metrics as responding in
an appropriate amount of time or handling an acceptable number of
connections.
ii. Stress or Load tests are one type of Performance Testing that can often
be automated with a test tool that simulates multiple users. Common
deficiencies encountered during this phase of performance testing are
related to things like slow execution cycles, data corruption, or
complete system failure.
iii. There are a number of different methods one can use to create a
performance test with eggPlant.
iv. eggPlant application performance testing tool can time individual
events or sequences of events, and it can be used either to generate
small loads or to enhance an existing load test scenario by simulating
real user interactions while another tool creates an artificial load on the
application.
v. Performance testing example
GIMP is the GNU Image Manipulation Program. It is freely distributed
and used for such tasks as photo retouching, image composition, and
image authoring.

Objectives:
To determine the performance impact of having multiple images
open for GIMP.
To determine the number of open images that the system can

handle the threshold that the system will allow.

Test Specs:

Operating System: Linux RedHat 7.1

Kernel: 2.4.2
Desktop: Gnome
Application: GIMP 1.2.1

Test Details
Our test example will use one 678435 jpeg image with a file size of 35754
bytes. The sample image must reside in the home directory of the current
user and will be named eggplant.jpg. For consistency, no other UI
applications will be allowed to run concurrently with GIMP.
The following settings for GIMP will be used:
Tip of the day will be disabled.
All other windows, with the exception of GIMPs main window, will be
hidden from view.

Test execution will consist of the following:

1. Launch GIMP from the Gnome menu bar.
2. Open a new image using the Control-o keyboard shortcut.
3. Wait for the Load Image window to appear.
4. Wait for the Load Image window to appear.
5. Proceed by selecting eggplant.jpg.
6. Press Enter to open the image.
7. Measure the amount of time that it takes for GIMP to load the image. The
time limit for an image to open will be set to 10 seconds. Once the threshold
is reached end the test.
8. Log the time.
9. Repeat steps 2 through 8.

Integration and validation with IBM Rational Quality Manager 4.0.

eggPlants integration with IBM Rational Quality Manager, eggPlants
market-leading automated GUI and screen testing is available to IBM
Business Partners and customers as part of the Ready for IBM Rational
program. This opens up opportunities for new users to harness eggPlants
capability for consistently delivering powerful business-driven GUI
automation and software testing results across legacy, desktop and mobile
environments.The eggPlant integration allows you to call eggPlant scripts
running on any platform (Mac, Windows, Linux or Mobile) and return the
results to your RQM server.

http://www.youtube.com/watch?
v=7yniUJh8hww&feature=player_embedded
Benefits
Non-invasive does not load to test devices or interact with underlying code
Transparent processes, results and metrics vital for ensuring that
application development teams, Quality Assurance teams and business teams
are all on the same page
Works with Windows 95, 98, 2000, NT, XP, Vista and Windows 7; Mac OS
X; LINUX; Solaris; HP-UX; plus BIOS and other hardware devices
Supports HTML 5; Adobe Flash/Flex; Silverlight; AJAX and Java

2. DeviceAnywhere :A cloud based platform for carrying out testing and

monitoring the functionality, usability, performance and availability of
mobile applications.

3. elusivestars:A crowd sourced Mobile Application service where Developers

have their Apps tested by real users with real devices.

4. Cucumber-Functional Automation tool

Cucumber is a functional test automation tool for lean and agile teams.
It supports behavior-driven development, specification by example
and agile Acceptance testing. You can use it to automate functional
validation in a Form that is easily readable and understandable to
business users, developers and testers. This helps teams create
executable specifications that are also a goal for development,
acceptance criteria and functional regression checks for future
changes.
Cucumber is one of the rare tools that try very hard to stay out of your
way, to let you do your work without making you worry about the tool
itself too much. It helps teams automate their specifications efficiently
in several ways:
It is relatively easy to set up.
It supports multiple report formats, including HTML and PDF.
It supports writing specifications in about 30 spoken languages,
making it easy for teams outside of English-speaking territories or
those working on internationally targeted software to engage their
business users better.
It supports different ways of describing executable specifications
including story-like prose, lists and tabular data.
It allows scripting, abstraction and component reuse on several levels,
allowing both technical and non-technical users to efficiently describe
specifications.
It generates the tricky parts of the code so that you don't have to write
most of the boiler-plate automation or make mistakes doing it.
It integrates nicely with the rest of the development ecosystem. It does
not try to impose a version control system, but works off plain-text
files that can be stored in any version control system. For continuous

build integration, it emulates JUnit (and everything else in the world is

already integrated with Junit).
Although it is a Ruby tool, people who work on other platforms do not
have to learn Ruby to use it. You can use Cucumber with .NET or
JVM languages almost natively.
It's integrated with all the most popular web testing libraries.
It allows you to mark tests with tags so that you can quickly run a
group of related tests (eg quick tests, slow tests, integration tests,
accounting tests).

5. MobileCloud-Interactive by Perfecto Mobile

The MobileCloud-Interactive service enables users to test mobile
applications, websites or services on a multitude of real handsets and tablets
via the web and control them as if holding them in their own hands. Using
MobileCloud-Interactive, you'll be able to streamline development, testing
and support processes, perform more tests and ensure optimal compatibility
of your product or service, on more handsets, networks and locations.

6. Robotium-Its like a Selenium but for Android