Continuous Testing

Continuous testing is the process of executing automated tests as part of the deployment
pipeline to obtain immediate feedback on the business risks associated with a software release
candidate.
• Functional
• Unit tests • Test driven development (TDD)
• API
• Integration
• Performance testing
• System testing • End-to-end use-case testing
• Non-functional • Security testing
• Performance
• Security • Manual testing
• Compliance • Canary & Blue/Green deployments
• Capacity

“Shifting left” is about building quality into the software development process. When you shift
left, fewer things break in production, because any issues are detected and resolved earlier.
NaradaCode
© DevOps Institute unless otherwise stated
1
 Type of Software Testing

Types of Testing

Functional Non-Functional Maintenance

• Unit Testing • Performance Testing

• Integration Testing
• Smoke Testing
• Sanity Testing
• User Acceptance Testing
Localization Testing
• •
• Globalization Testing
• Interoperability Testing
• ... and more
• Endurance Testing
• Load Testing
• Scalability Testing
• Regression Testing
• Usability Testing
• Maintenance Testing
Reliability Testing
• Portability Testing
• Availability Testing
• … and more

NaradaCode
© DevOps Institute unless otherwise stated
2
Team collaboration and incremental delivery starts on the Jira Scrum Board

Let’s Compare Testing in

Waterfall, Agile and DevOps
NaradaCode
© DevOps Institute unless otherwise stated
 Testing: Waterfall Model

Requirement
Gathering

Analysis

Design

Development

Phase of Testing, Testing

once processed and completed

Deployment

NaradaCode
© DevOps Institute unless otherwise stated
4
 Testing: Agile Model
Requirement
Gathering

Analysis
Ø Agile testing is a software
testing process that follows
the principles of agile software
Design Development
development process.

Phase of Testing,
Ø Testing in agile is a continuous Testing continuously
throughout the
process rather then development
sequential.
Deployment
NaradaCode
© DevOps Institute unless otherwise stated
5
 Testing: DevOps Model

Ø Testing as process is automated and triggered continuously

Requirement Gathering

Code Ø Quality Gates needs to be passed before moving the

Check-In
automatically forward in the delivery process
Unit Test

Static Analysis
Packaging

Deploy to Test
Integration Test

Deploy to Staging
Acceptance Test
Deploy to Pre-Prod

Security Test
Deploy to Prod

Performance
Test
Quality Gate

NaradaCode
© DevOps Institute unless otherwise stated
6
 Test Pyramid : What Your Test Should looks like

Use Session Based Testing to focus

on the test objective

NaradaCode
© DevOps Institute unless otherwise stated
7
 Test Pyramid: [ANTI-PATTERN] The Ice Cream Cone

Teams skip unit testing (or delay them) in order

to reach development milestones quickly
Manual / Exploratory because of deadline driven development.

This path will cost the project much more time

E2E and money later on.

Ø The Ice Cone describes testing dependent on

Integration Testing
UI and exhaust manual tester to keep up

Unit Tests
Ø Waiting to write unit tests until features are
completed can result re-learning (Waste)

Ø Unit tests written early save time and money

by closing this feedback loop before the build
gets unstable

NaradaCode
© DevOps Institute unless otherwise stated
8
 What to Automate? The Automation Test Quadrant

Automated &
Business-Facing Tests Manual
Manual

Functional Acceptance Tests Exploratory Testing

Examples Scenarios
Story Tests Usability Testing
Support Programming

Prototypes UAT (User Acceptance Testing)

Critique Product
Simulations A/B
Q2 Q3
Q1 Q4

Non-Functional Acceptance Tests

Unit Tests
(Performance-, Load-, Security-, Availability-
Integration Tests
Testing)
Component Tests
Any “ility” Testing

Automated &
Automated
Technology-Facing Tests Manual

NaradaCode
© DevOps Institute unless otherwise stated
9