Next Gen Software Testing

Trends

Software Testing Trends

Software Testing Job Trend - Case for Selenium

Selenium & products based on Selenium are the most popular and preferred testing tools

Where is SW Testing industry heading?

What are the Key Influencers

New development methodology trends (Agile)
SaaS / Service based models
Higher emphasis on quality - reputation & business risk
Time to market
Increase in transformational projects

Cost

Where is SW Testing industry heading?

Over 2/3rd of software development projects using agile method to deliver software quickly.
As software releases become more frequent, testing processes have to keep pace
Software testing budgets rising and likely to overtake software maintenance budgets
IT budget allocation towards QA projected from 18% in 2012 to 29% in 2017

Increase in transformational projects mobile, cloud & big data applications

Increase in development means more Risk
Application failure consumer backlash & reputational risk

Key focus is on cost optimization via process & technology

Testing of SaaS Applications

Big push towards Cloud Migration
SaaS has changed the way users think about software
Rather than purchasing, users are now renting
Renting vs Buying mentality - Users will move out quickly if you dont provide what they want, when they
want it

Large SaaS projects are onboarding clients with requirements that are unique & complicated
Verification of data migration (ETL) & functional flow within framework of the offered solution for each client is
complicated challenging

Advantages of SaaS - no infra, on demand, cheap, scalable, create and run & access
anywhere & anytime
Software Testing becomes extremely important - attention to user experience, user satisfaction,
and performance

Big Data and Analytics

Rapid growth 90% of worlds data has been created in the past 2 years (IBM)

With Big data comes bad data

Average organization loses $14.2 million annually because of bad data (Gartner)
75% of businesses waste 14% of revenues due to poor data quality (Experian Data Quality)

Testing Challenges

Testing huge amounts of data and their correctness & accuracy because the data is
sourced from multiple platforms
Testing focused on data irregularities and quality issues, accuracy of ETL and if source
data is relevant and accurate for analytics

Internet of Things (IoT)

IoT covers all devices that connect to Internet

Thermostat, Automobiles, Washing Machines, Smart Watches + 1000s more that collect, display,
react to and send data all the time

Challenge is for companies that serve data to many devices they cannot have all of
them for testing
IoT Testing will be very focused largely dependent on who the customer is, e.g.
Performance & security for internet connected storage devices
Data collection & delivery for medical devices

Testing Centers of Excellence (TCOE)

TCOE - A group of people, processes, tools & best practices for enhancing testing
effectiveness
Enterprises tasked with introducing innovative technologies & new systems at record
speed in-memory databases, mobility, portal, and cloud technologies are driving the
pace of innovation

Many Fortune 1000 firms find TCOE the best approach to reduce IT costs, accelerate
innovation, gain testing competency, and improve the quality of business systems
Push towards Agile & DevOps practices
Continuous & automated testing is becoming a key strategy

Testing the SMAC pack

What is SMAC? - Social media, Mobile, Analytics and Cloud technologies
Social media & mobile demands of "zero tolerance" will drive QA demands higher
As more mobile apps come to the market - ones that are built well and offer better value
through frequent feature updates will probably survive - Mobile QA to support all types of
apps becomes very important
SMAC Testing applications need to
Allow insight into customer behavior & become more business centric
Support multiple devices & platforms
Focus on testing algorithms to secure SMAC data and applications

10

Software Testing Trends at Granular

Level

11

Continuous QA (CQA)
What is CQA? Process of executing automated tests as part of the software delivery
pipeline & get immediate feedback on risks associated with software release
Continuous Integration (CI) & Continuous Deployment (CD) aim to introduce smaller code
/ features into production in shorter time,

CQA means test early & often. Integrated approach bridges the gap between CI & CD
CQA needed for faster releases, quality software, catch bugs before deployment
On demand, fast, flexible, comprehensive - traditional QA does not work with CD
Goal Test software builds as rapidly as they are being produced

12

Crowd-sourcing

Allows for remote usability testing Testing is user centric

Software is put to test in diverse, realistic environments. It is more reliable, cost effective,
bug free
Pool of testers is diverse and unbiased
Parallel testing by multiple testers makes it faster

13

Cloud based Testing

Ideal for SaaS applications hosted on the cloud
Opportunity for collaborative testing without boundaries
Allows on-demand scalability without infrastructure concerns
Great for small to medium sized development projects

Easy to manage & offer Subscription based model based on usage which is attractive to
software projects of all sizes
Ready testing infrastructure means quick startup times

14

Test Pricing models

Flexible pricing models are becoming more popular
Pay per usage
Use case based
Results based
Time and material

Risk reduction makes it attractive to businesses of all sizes

Free trial periods
Fast evaluation cycles - businesses cant afford to get caught up in month(s) long proof of
concept (POC)

No long term vendor lock-ins

15

Testing concepts
Testing concepts to reduce risk, cost, time and effort & increase test coverage

Risk driven testing - testing tools that identify what to automate rather than automate everything e.g. if feature 3 &
5 not tested the risk is 80% compared to 1 & 2 which is 20%

Flexible & configurable codeless testing Allow business users to actively get involved in testing & automation &
perform complex functional tests (record and playback with advanced features)

Cross-browser & multi-device testing test application compatibility on multiple browsers & user experience in
various resolutions on multiple devices

Modular testing use multiple data sets on base tests. Reuse common data sets in multiple test (ex. User login)

Test management & reporting ability to effective control a test environment

Quickly identify defect patterns & monitor the pace of corrective measures
Prioritize test execution by criticality or functional areas of the software
Schedule regression / routine tests

Performance Testing benchmarking & grading application performance over a period of time

16

Security & Penetration testing

Driving forces rapidly increasing cyber security threats, strict government regulations &
compliances, automated testing tools
Growth projected to be from $2.47 billion (2014) to $4.96 billion (2019) (Markets and Markets)
Pentest is a software attack on a computer system to identify security weaknesses & potentially
gaining access to the computer's features and data.
Can help determine whether a system is vulnerable to attack
Goal
Identify vulnerabilities that may be difficult to detect with automated network scanning tools

Assess the potential business & operational impacts

Test ability of network defenders to defend & respond
Justify additional spend on security tools & resources

17

Some helpful tips on selecting a

test automation platform

18

Identify your Pain Points

Resources

Pain Points

Timeliness

Affordability

Complexity

Flexibility

Scalability

19

Create an evaluation criteria that covers your

pain points (see example)
Time

Setup Time
Learning Time
Evaluation Time
Test Creation Time
Ongoing Maintenance Time

Cost

Features

Coding effort needed vs. Codeless

Record / Playback

Test Coverage

Test Management

Startup expense

Test Execution

Ongoing expense

Extensibility

Risk
Startup time

CI/CD tools

Pilot project

Custom scripts

Hardware Cost

Licensing terms

Lab Cost

Resource pool

Scheduler

Support

Site monitoring

Responsive testing

Reporting

Data Driven

Multi-Browser testing

API testing

Infrastructure Cost
Resource Cost
Training Cost

Reliability

Licensing Cost

Security

Portability (export/import selenium)

20

Perform Cost/Benefit analysis (see example)

21

Some information on CloudQA

22

CloudQA & how it is adapting new trends

What is CloudQA?
Test automation and test management of web applications
CloudQA design influenced by emerging trends & demands
Make it fast & easy no programming skills required
Cloud enabled service - zero infrastructure, hassle free startup effort & accessibility
Low risk implementation time, cost and effectiveness
Appeal to development environments of all sizes & budgets

23

What does CloudQA offer?

Basically, CloudQA is a cloud based test automation tool that has
taken care of a lot of work automation testers have to do
themselves:

It provides a readymade framework

Has a robust test recorder that even handles recording of

complex web technologies and even the ones that the
selenium recorder has difficulty with

Has built in test management capability so you can manage

1000s of test cases

Has a powerful execution engine that runs in Amazon cloud

and allows you to run your tests against multiple browsers

Allows massive parallelization that will allow you to run 100s of

non-sequential tests in parallel literally cutting down
execution time to minutes instead of hours or days

Has a powerful reporting engine that keeps a track of your

run history for months

Integration with Bug reporting, CI/CD tools like Jira,

Jenkins, etc and extensions via Zapier
Allows Data driven testing where you can record a
test scenario once, create a data template, load
new data sets and run them without creating new
tests
Allows codeless recording of tests and also allows
extensions using javascript for those rare situations
where the recorder does not work out of the box
Include API testing
Includes site monitoring
Has a scheduling feature so you can create sets of
tests and run them unattended at predetermined
times
sumant@cloudqa.io

24

Benefits of CloudQA over other tools

1.

2.

3.

More time savings and staff efficiency

Fastest & easiest setup

Codeless testing, and hassle free maintenance

On-demand or scheduled remote test execution

Quick integration with productivity tools you use (Jenkins, JIRA, Slack, CircleCI, Teamwork + many more)

Comprehensive Notifications, Reports and Dashboards

Higher quality in business processes and software applications

More web technology coverage than any other tool (Angular JS, Ember JS, iFrames, Telerik)

Data driven testing to reduce test creation & maintenance time

Modular & Extensible without the complexity of custom coding

Cross platform and browser compatible

Minimal Risk

Evaluation Risk - get started in minutes & find out quickly if CloudQA works for you

Resource Risk - Zero Infrastructure. Extremely short learning curve - use existing resources for testing

Cost Risk Pay as you go subscription model vs. licensed model with vendor lock-ins

25

Please feel free to contact CloudQA if you have any

Test Automation related questions or inquiries.
Thank you!
visit https://cloudqa.io

info@cloudqa.io

26