Food Safety, Consumer

and Health Services

Revamp
​Capability building bootcamp
– Engineering

​09 August 2023

Agenda

Introducing engineering practices in Agile

teams

Engineering principles and standards

Deep dive DevSecOps

2
The broad popularity of the term “Agile” came with the creation of the Agile
Manifesto in 2001, which outlined Agile values

 In 2001, representatives The Agile Manifesto

of alternative software
We are uncovering better ways of developing software by doing it and
development methodologies
helping others do it. Through this work we have come to value:
gathered (the “Agile alliance")
and published the Agile manifesto
Individuals and interactions Process and tools
 The Agile Manifesto sets
principles explaining how Agile Comprehensive
methods differ from "classical" Working software
documentation
software development over
 Based on practical experience, Customer collaboration Contract negotiation
the values and principles of the
manifesto lay out guidelines for
Responding to change Following a plan
effective teamwork that can be
applied to any type of
knowledge work That is, while there is value in the items on the right, we value the items on
the left more.

Source: www.Agilemanifesto.org (March 2008) 3

Agile development is iterative and reduces the risk of incremental “big bang”
project delivery…
​Traditional development (“Waterfall”) ​Modern, short-cycle delivery (“Agile”)
​Delivery Requirements Customer input
process and feedback
Design Analysis
Incre-
Development mental 2-4
release weeks User
Testing stories

Maintenance Testing/
Develop-
review
ment
1-3 years

​Characteris-  Sequential process with long cycle times
tics  End users typically involved only during
requirements and testing
 Limited ability to adjust for customer feedback
 Slow and expensive to respond to changing
requirements
 Quick iterations, short time to market
 Business and IT working together and involved throughout
 Feedback as an integrated part of the development process
 Adaptive to evolving requirements

​Value
delivery

4
…However, most organizations have been designed for efficiency and stability vs.
speed and flexibility; they are not setup to face challenges and opportunities from a
fast changing and digital world

Siloed and inflexible Top talent not assigned to top

organizational structures priorities

Monolithic IT systems
Rigid processes and complex application
landscape

5
Most digital organizations are adopting the agile methodology to reduce the cycle
to release software because at the core of the methodology is the requirement to
release working software
Agile delivery deploys products and Traditional “waterfall” approach builds full digital platform
Customer value processes that deliver impact right away that is only deployed at the end of the build process

Modern Agile delivery:

Quick iterations, short
time to market
Feedback as an
integrated part of the
development process
Short cycled, agile Adaptive to evolving
IT delivery cycles requirements

Traditional “waterfall”
delivery:
Long cycle times
Limited ability to
Traditional projects adjust for customer
take a “big bang” feedback
approach Slow and expensive to
respond to changing
requirements
Time
6
However, even if weekly release cycles, when agile is not extended to the end-to-
end IT value chain, organizations experience significant operational bottlenecks

3-5 days 1-2 days 5-10 days 2-4 days 1 day 0.5 day 0.5 day
Rapid 1-2
week sprint

cycles
Product Code Deploy Full suite Release Prod Smoke Monitor,
discovery merge E2E test testing mgmt. deploy test roll-back

Business/IT single Large Long wait Long cycles Heavy Over 8 Multiple Reactive
team collaboration code times for of manual paper hand- business monitorin
merges environmen end-to-end release offs to owners to g and
Rapid 1 week issues t scheduling and non- process different coordinat manual
sprint cycles after functional with 10+ IT op e manual roll-back if
Feedback cycle weeks 15-20 fields testing forms specialist smoke needed
with users of to manual (e.g., with test
Continuous develop fill for regression, multiple
improvements -ment deploy form security, manual
etc.) approvals

15-20 Infrastructure
days provisioning

Manual provision and

setup of new environments
for team (e.g., servers,
platforms, network, etc.)

~30-40 days of product ~15-25 days “code complete” to production

7
development
Agile development needs advanced engineering practices to fully realize values of
efficiency and effectiveness
Deploy binary
to production
Deploy binary to
pre-production Automated
Deployment
Commit code to
main version Continuous  Run regression tests in
integration pre-production
Put code in
review queue Streamlined  Integrate code into
code review main version
Submit code for  Build main version
testing and review Automated testing  Review code
 Approve code for
submission
Test as  Run unit tests
code  Run integration tests
 Run regression tests
 Fix bugs/errors/
changes
 Agree test cases
 Code test cases
 Code functionality
Show failed tests Specify required Show code conflicts Show errors from pre-
improvements in code Show build errors production
organization and readability

8
 Squads will jointly work to deliver the digital native capabilities
For the digital transformation
journey squads will have to …
Rapidly prototype digital journey MVPs
Adopt a customer-centric view for the
development and end-products
Deliver quality end-products
autonomously
SQUAD
… which requires..
Fully automated dev/test/production
environment integrated with security
to streamline and secure the software
development processes
Continuous Integration and
continuous deployment for the
deployment of MVPs to the production
environment
Dedicated parts of business and IT
functions including IT security to co-
locate and work side-by-side as an
agile team to develop the required
products
… to be enabled by..
Decoupling the legacy code
architecture into smaller re-usable units
as microservices
Self-service provisioning of infra-
structure based on defined catalogues
that allows for instant provisioning,
monitoring and management
of infrastructure

Journey DevSecOps Micro-

Cloud
squads services

9
Agile key capabilities

Design principle Rationale

Closer collaboration between a. Business and IT co-locate to rapidly prototype digital journey MVPs
business and IT in an agile b. Agile squads formed from business and IT have joint responsibility and
environment autonomy to deliver end-products
c. Business brings customer-centricity into the design of journeys/products

Continuous development and a. Fully automated dev / test / production environment to streamline the
deployment software development processes
b. DevOps and continuous development enable rapid deployment of MVPs
to the production environment

Modular and standardized a. An architecture based on clear logic that forces developers to adopt
2-speed architecture that modular and configurable run-time environments/databases and a logic
allows for quick fully exposed in open API
release cycles b. Release cycles drastically reduced to allow for faster TTM

Self-provisioning of a. Self-service provisioning of Infrastructure based on defined catalogues

infrastructure through Cloud that allows for instant provisioning and management of infrastructure

10
10 core engineering practices and capabilities enable software development agility
1 2 3 4 5

Source code Peer code Environments Code Unit testing

management review refactoring

6 7 8 9 10

Continuous Automated Pair Test-driven Just enough

integration testing programming development documentation

11
Each practice and capability solves for pain points (1/2)
Description What pain points it solves

Version control - source code is version controlled in a master • No history of code changes • Integration phase needed to merge
Source code location and managed with source code management tools • Code is saved in multiple places changes long after they are made
management • No accountability for changes • Difficult to understand when or why
changes were made

Second set of eyes - review of code by peers intended to find • No coding standards across teams • Only Jane knows how module X
Peer code and fix mistakes. Leveraging co-development to solve complex • Inconsistent design and works
review problems and improve developer skills implementation • Lack of sharing and teaching in
teams

Avoid interference – a collection of environments for • Environments with different configurations

Environments development, testing, integration, and sign-off for an application • Slow, buggy deployments only happen in production
• Inconsistent testing results

Leaving it better than you found it - improve existing code that • Hard to maintain code and fix bugs
Code is time-consuming to maintain and enhance without changing its • Over time, it gets harder to make changes or add enhancements
refactoring external behavior or adding new functionality • Team fears making changes in specific modules

Guard against bugs – developers write automated tests as part • Large number of sprint defects • Making development changes takes
Unit testing of coding a feature to find and guard against bugs. Unit tests • Code is overly complex and hard to a long time
safeguards the functionality from being accidentally harmed debug • The design of the code is low-quality

12
Each practice and capability solves for pain points (2/2)
Description What pain points it solves

Fail fast, move faster - code is integrated frequently to detect • Failures happen significantly after • Long and tense integrations happen
Continuous errors quickly, locate them more efficiently, and facilitate a faster code is written infrequently
integration build cycle, e.g., every few hours or daily • It is hard to remember why changes • Much time spent on debugging
were made

Giving customers what they asked for - describe what we • Developers say “done”, then testers • Users find bugs before the team
Automated want to see the software do, then automate the verification that find lots of bugs
testing the software does it to facilitate a faster build cycle • Significant time spent doing
regression testing

Collective ownership - all code is produced by two people • No code ownership amongst the team • Hard to help when one person
Pair programming on one task on one workstation. Alternatively, code • Inconsistent coding standards becomes a bottleneck (or goes on
programming is co-owned by multiple people, e.g., no expertise siloes • Only Jane knows how module X works vacation)

Better design - tests are written first to ensure that what I’m • Code hard to reason about and overly • Over-architected systems
Test-driven about to write works once I write it, then minimal coding is done complex • Building for more than what’s needed
development to pass the test. This leads to a better / more straightforward • Design anticipates future complexity
design. that may not happen

Package your product - ensure that support onboarding, • Significant time is spent writing • Code is overly complex and not “self-
Just enough
architecture document, and all other just enough documentation supporting documentation that is documenting”
documentation is in place rarely used

13
Agenda

Introducing engineering practices in Agile

teams

Engineering principles and standards

Deep dive DevSecOps

14
A few rules to keep the code consistency high
Description

A Test-driven development • Tests are written and run first, then minimal coding is done to pass the test

• All code is integrated in a latest version as often as possible, e.g., every few hours
B Continuous integration or daily

• Keeping a small set of highly stable code branches, with local branches frequently
C Branch hygiene merged after undergoing robust (automated) testing (e.g., GitHub flow philosophy)

• Automate a high percentage of tests to allow for continuous regression and integration
D Testing automation testing: 80%+ of unit tests including happy path, 70%+ functional tests

• Ensure high code quality through automated quality checks on each build (e.g., code
Refactoring and high code
E complexity, duplication, etc.)
quality • Allow for time to improve the code without adding functionality

Collective code ownership • Code is co-owned by multiple people, e.g. no expertise siloes
F • Practice peer review of code (e.g., GitHub automated pull requests)
and code reviews

15
Incorporate the best engineering principles
Key principles Description
YAGNI - You ain’t Don’t write code for functionality you think might be needed in the future but do not
gonna need it need yet

DRY - do not repeat Avoid rewriting the same piece of code more than once. When the same piece of
yourself code is being written for the third time, extract it to a common place

SOLID Principles enable code extensibility and maintainability:

SOLID Single Responsibility Principle; Open/Closed Principle; Liskov Substitution Principle;
Interface Segregation Principle; Dependency Inversion Principle

Self-Documenting
Write code so that it is self documenting and does not require complex explanations
Code

KISS - Keep it simple

Write the simplest code possible to implement the functionality expected
stupid

TDD – Test Driven Ensure code is properly tested and documented. Important discipline to practice
Development ensuring highly maintainable code and correctly functioning code

16
Testing is everyone’s responsibility
Developer
• Develop story
Product Owner
• Write, executes unit tests
• Write user story and • Contributes to integration and
Write user story
acceptance criteria functional tests
Develop
and unit
test user
story QA/Developer
• Test story against acceptance criteria
• Writes integration and functional
Product owner Conduct automation tests supported by
• Review of story to ensure it acceptance Systems team
meets the acceptance criteria review Conduct
functional
Conduct testing
non- Run regression
Developer/System team and E2E tests
• Test for performance, security, stability, functional
testing
load, stress, resilience testing

Tester/System team/Developer
• Writes and executes suite of regression and
end-to-end tests supported by System team
17
Test coverage: Different levels of automated tests are required to ensure quality
in software product
Return on
automation Rationale

Lower • Most volatile to change, can be brittle,

Func- slower to execute
tional
(UI) tests

API / Integration/ • Closer to business rule/ function

smoke tests • Less volatile to change

• Closer to executable code

Unit tests • Very focused on a specific behavior of code

Higher
Number of tests

18
Agenda

Introducing engineering practices in Agile

teams

Engineering principles and standards

Deep dive DevSecOps

19
While Agile focuses on extending development teams towards the customer,
DevSecOps extends development teams into IT operations

Stakeholders

Customer Development team IT operations

Agile

DevSecOps

Description Agile improves collaboration between the customer and
the development teams by developing working solutions
frequently to iteratively align requirements with the
customer.
DevSecOps improves collaboration between
development teams and IT operations by extending
development teams to IT operations. It also includes IT
security to deliver faster and with higher security through
faster release cycles, using continuous delivery and
security by design.

Goal Be closer to the Business Deliver in weeks rather than years

20
DevOps is an extension of Agile that includes business, IT and operations

1. Plan 2. Design 3. Build 4. Test 5. Release 6. Operate

Agile development

Agile + Continuous
integration

Agile + Continuous delivery

Agile + DevSecOps

21
A new technology stack and toolchain is required to enable the methodology

1. Plan 2. Design 3. Build 4. Test 5. Release 6. Operate

Issue and project Integrated project Git code hosting and Continuous code Docker Container Logging and analytics
tracking software collaboration platform automated built quality platform orchestration and
pipelines dynamic scaling of
services
Application perfor-
Automated security mance management
Sketch Static Code Analysis testing and reporting platform
CI/CD pipelines

Collaboration
between designers
and developer

Design and proto-

typing platform

22
The tools must be complemented by new engineering and operational practices
“Learning to crawl” “Starting to walk” “Running” “Flying”

1 Unit and integration testing Test driven development 6 Self-service access to

4 Suite of automated 5 One-click continuous prod-like environments
… tests deploy to any
environment

Development
2
Streamlined Cloud
Perf. QA
code review Unit tests
tests

Version Functional Security Staging

control tests tests

3 Continuously integrate w.
“single source of truth” Production 7 Auto-scaling of infrastructure

8 Automated
performance management

Build status Code metrics Site monitoring Integrated logging

23
Key DevSecOps metrics link integration, delivery and deployment to
business growth
​DORA KPIs ​DORA supporting metrics
​Becoming a level for growth as elite DevSecOps org. ​Supportive KPIs ​Description

Low Mid High ​Sprint commitment  % of story points committed with DoD by
performers performers performers sprint end
​DORA KPIs1
​Code review  % of PRs reviewed according to best-practice
​Deployment frequency ​Between ​Between ​On-demand
coverage  Code review duration
​How often a team 1/month and 1/week and (multiple
successfully releases to 1/6 months 1/month deploys / day)
​Test coverage  Unit test coverage %
production
 Automated integration test coverage %
​Lead time for changes ​Between 1 to 6 ​Between one ​Between one  % testing automatization
​The amount of time it takes months week and one day and one
for committed code to get month week ​Static code quality  Maintainability rating
into production analysis scores  Code smells
 % Duplicated lines
​Mean time to restore ​Between one ​Between one ​Less than one
​How long it takes an week and one day and one day ​Security code  Vulnerabilities
organization to recover from month week analysis  Hotspots
a failure in production
​Manual steps to  # of steps in the release process that are
​Change failure rate ​46%-60% ​16%-30% ​0%-15% release manual
​Percentage of deployments
that result in a failure in ​Build and  % success build
production deployment  % success deployment

1. DORA (DevOps Research and Assessment 2022) research program acquired by Google in 2018

​ ource: https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance
S 24
To generate revenue more efficiently and release faster, traditional development
cycles can benefit from DevSecOps capabilities
SAMPLE DELIVERY CYCLE

1 Manual deployment
Deployment is complex and
lengthy, taking up the time
Release
review 1 Manual deployment and effort of multiple
“Pitfall” Sprint
Build Test
resources for both rollout and
start
rollback
delivery Week 1 Week 2 Week 3

cycle 2 Faster iteration

Full Release Developers can build, test,
and validate features on the
2 Faster iteration
same day. Delivery
Release (integrating new code into the
Build Test Build Test Build Test Build Test review Capturing
Optimal Sprint extra week of impact
staging/production-ready
delivery start environment) is automated
Week 1 Week 2 Week 3
cycle
3 Automated release
3 Automated Release Deployment (releasing new
code to the general customer
base) is automated to simplify
the task for release managers

25
A key part of the new technology stack is the automated CI/CD pipeline (1/2)
ILLUSTRATIVE
Automated Manual

A developer works on a branch that is separate
from the main codebase. When he is done
writing the automated tests and developing the
feature he initiates the testing and code review
process by creating a “pull request”
The tech lead on the team reviews the code and
provides feedback and coaching to the
developer. Once any changes are made and all
the tests pass the pull request can be approved

Tech Lead
Code is Tech Lead
Automated approves Code is
committed Pull request reviews the
tests are and merges merged into
to a feature is opened code and
run the pull master
branch test results
request

• Code quality analysis A series of automated tests are

• Unit tests run and the results are reported
• Integration tests back in the body of the pull
• Security vulnerability request. The pull request cannot
assessment be approved until the tests pass.

26
A key part of the new technology stack is the automated CI/CD pipeline (2/2)
ILLUSTRATIVE
Automated Manual

After a pull request is merged into master
and the CI process has completed
successfully, the code is automatically
deployed to the development environment
on the cloud
While a manual step is not required as all
tests are automated, the Product Owner
indicates he is ready to test the new
features by approving the promotion to the
test environment on the cloud
Once the Product Owner and Tech
Lead are satisfied that everything is
working, they can approve the
promotion of the feature to the
production environment

Deploy to Product
Product Deploy to Deploy to
Code is Develop- Manual Owner and
Automated Owner Test Automated Production Automated
merged into ment sanity Tech Lead
tests Approves Environ- tests Environ- tests
master Environ- testing Approves
Promotion ment ment
ment Promotion

• Sanity tests • Sanity tests • Sanity tests

• Integration tests
• End-to-end tests
If any of the automated tests
fail, the deployment is rolled
back to the last working version
• Integration tests
• End-to-end tests
• Security testing
• Performance testing
If any of the automated tests
fail, the deployment is rolled
back to the last working version
• End-to-end tests
• Security testing
• Performance testing (periodic)
• Error monitoring
If any of the automated tests fail,
the deployment is rolled back to
the last working version

27
CI/CD pipeline
demonstration
Thank you
Backup
Agenda ​Engineering principles & standards

​DevOps – DORA metrics

​Development – Developer workflow

​Development – Git Branching model (GitFlow1)

​Dev/Ops – CI/CD workflow

31
A: Test driven development

• Important discipline to practice ensuring highly

maintainable code and correctly functioning code
• Improves code design (outside-in instead
of inside-out)
• Much easier to change/refactor/maintain code
• Drives the code design
• There are three main steps:
– Decide on the desired functionality
– Create test for that functionality first. Test fails
– Write the code that implements the functionality.
Test Passes
– Look for opportunities to refactor. Repeat steps 1 to 3

32
A: Advantages of TDD
Early bug • Developers test their code but in the database world, this often consists of manual
notification tests or one-off scripts. Using TDD you build up, over time, a suite of automated
tests that you and any other developer can rerun at will.

Better Designed, • It helps to understand how the code will be used and how it interacts with
cleaner and more other modules.
extensible code • It results in better design decision and more maintainable code.
• TDD allows writing smaller code having single responsibility rather than monolithic
procedures with multiple responsibilities. This makes the code simpler to
understand.
• TDD also forces to write only production code to pass tests based on
user requirements.

Confidence to • If you refactor code, there can be possibilities of breaks in the code. So having a
Refactor set of automated tests you can fix those breaks before release. Proper warning will
be given if breaks found when automated tests are used.
• Using TDD, should results in faster, more extensible code with fewer bugs that can
be updated with minimal risks.

33
Our Typical Workflow Fast step < 5 mins Slow step > 5 mins

Notifications / Alerts / Analysis

Kubernetes

Store
artifacts
1 2 3 4 5 6 7 8
Run unit tests Deploy Run Run
Checkout Static code Build the Static Security
and coverage & tag integration Functional
code analysis code Checks
reports branch tests tests

Webhook
trigger

AZURE REPO
Java
Pipeline Purpose Trigger Steps

Commit code,
Pull Request Validate each PR raised by developers Pull Request 1,2,3,4
Raise Pull
Request Dev Build the Dev branch and deploy Dev backend Merge of PR 1,2,3,4,5,6,7
Peer Review
SIT Build the SIT branch and deploy the SIT backend Manual* 6,8
Developer Tech Lead / Developer
1. During sprint development. In BAU for one-click to production each pipeline is triggered after the success of the previous 34
DevOps Process for Web Apps and Microservices ​Gates:
 Code quality passes
 Peers and Tech Lead approves
Pull Request is created  Automated Tests pass

Developer writes code Automated unit and

integration tests are run Code Quality Analysis Peer code review Automated End-to-End
and tests Automated deployment to
tests are run
Development environment
​ ates (Microservice squad)
G
 Documentation review
 API conventions review
 Duplicate review

​Gates:
Developer publishes API API is published
to Service Registry  Automated end-to-end tests pass
to Service Registry and Gateways
 Tech Lead approves

Manual Testing by ​Gates: Automated security scans are run Automated End-to-End Automated deployment to Team requests promotion
Product Owner  Automated end-to-end tests pass (SAST and DAST) tests are run UAT environment to UAT
 Automated security tests pass
 Product Owner approves
 Change Management approves (first-time only)

Team requests Automated sanity tests are

Automated creation of Automated deployment to Automated closing of Change Optional: configuration of
production run
Change Request (CR) in Production environment Request (CR) in Remedy A/B tests and enabling of
deployment
Remedy Feature Flags

Optional: rollout “dark feature” or

A/B test to user group(s)
35
Short lived Feature branches will be used together with squad branches
Stable and production ready “master” branch

Stable digital squad branch

Developers Once ready, The code is

Tech Lead The code is
Incremental The code is merged into
creates a create a a pull merged into
changes reviewed squad branch
master
digital squad feature request1 is
are added online branch and
branch branch raised and deployed
deployed

36
Agenda ​Engineering principles & standards

​DevOps – DORA metrics

​Development – Developer workflow

​Development – Git Branching model (GitFlow1)

​Dev/Ops – CI/CD workflow

37
Measuring delivery speed impact - lead time to change metrics
SAMPLE TEMPLATE

Current LTTC Target LTTC LTTC Savings

XX days / release XX days / release XX days / release

The expected impact of product innovation in backlog: The delay in releasing features
The avg revenue uplift impact of new features per quarter Time lost due to current time to deploy

Expected feature and Expected unique Expected increase Current lead Benchmark led time
optimisation output user conversion rate in AOV time to deploy to deploy

~ X features ~XXk visitors ~ $X.XX X days X releases ~$XX.Xk / Qtr

(per release) (per day) Avg AOV (per release) (per quarter) Potential impact
to capture

Baseline of impactful Baseline for fully Avg AOV uplift Saved time on lead time We have 6 release
feature output converted user visits The current USD to change per Qt
The average amount of Avg of unique visitors average basket value The amount of time it For each release we will
significant features the that end up making a for the last year takes today for newly save ~ 5 days
core squads produce purchase committed code to be
introduced into production

38
Measuring revenue at risk, relating to bugs deployed – time to restore service and
change failure rate metrics
SAMPLE TEMPLATE

​Current TTRS ​Target TTRS ​TTRS Savings

​~XX hours ​XX hours ​XX hours

​Current CFR ​Target CFR ​CFR Savings

​~%XX% ​~XX% ​~XX%

The hourly avg amount of impact The number of significant failures introduced into the system and the
generated avg time to fix them

~XX hours(time XX bugs ~$XX.XM /Qtr

~ $ XX / h ~XX (hourly) ~X%
saved) (quarterly) Potential impact to
capture
AOV baseline Baseline for fully Mean time to Avg. defect Change failure rate
The current USD converted traffic recovery Number of bugs that The percentage of
average basket value The annualized The amount of time it “escape” testing and deployed changes
for the last year number of visitors takes the business to are released into causing a failure in
that end-up buying recover from a failure production production
in production

39
Let’s create DORA KPIs for our squads

Exercise description
​Round 1 ​Tech Lead takes each squad ​Each squad prepares DORA
through squad DORA KPI KPIs
framework ​20 min
​2min

​Each squad presents KPIs to ​Each squad shares feedback

other squads on the squad’s KPIs
​5min ​2min each

​Round 2 ​Each squad revises their KPIs

​5min

40
Agenda ​Engineering principles & standards

​DevOps – DORA metrics

​Development – Developer workflow

​Development – Git Branching model (GitFlow1)

​Dev/Ops – CI/CD workflow

41
Development – Developer Workflow Overview
Focus of today

Start
Develop
Assign JIRA
Feature (follow
Task
DoD)

Developer Create Git Submit Pull

Feature Branch Request

PR Check Pipeline Lint, Dev Pipeline End

Build
No Lint, Build
Pass Deploy
Unit Test Unit Test
Integration Tests Integration Tests
CI-CD Tool

Yes No

Review Code Pass

Tech Lead / yes

Peer Developer
Merge Code

42
Let’s review the current Developer workflow

Exercise description
​Round 1 ​Tech Lead takes each squad ​Each squad reviews and
through Developer workflow discusses current Developer
overview workflow
​2min ​20 min

​Each squad presents their ​Each squad shares feedback

views on Developer workflow on the squad's workflow
to other squads ​2min each
​5min

​Round 2 ​Each squad revises the workflows

​30min

43
Agenda ​Engineering principles & standards

​DevOps – DORA metrics

​Development – Developer workflow

​Development – Git Branching model (GitFlow1)

​Dev/Ops – CI/CD workflow

44
Development – Git Branching Model (GitFlow1)
​Git Branches are very cheap, so we following a branching strategy where branches are created for bugs and features.
Pull Requests are used for every merge. This is explained in this image. During development NO CODE
is committed directly to master or the development branch
​Feature Branch
Development branch Master/release branch
​When working on a new task in Jira follow these
steps:
 Assign the task to yourself
 Follow these steps to create a branch from
feature/123-login
Master
Bug/123-login
hotfix/123-login
 Branch name should be in the following format
feature/<Jira-task_id>-awesome-feature
Branch
Feature x

 Do your work and commit regularly to this

branch
Hotfix
branch, pushing code to the remote repository
 Once done (please review Definition of Done),
Branch
Bug x

create a Pull Request (PR) adding the relevant

Pull request
Pull request Work Item to the PR
Pull request
 The PR will get reviewed by a colleague,
Delete
branch address any comments or issues
Merge Hotfix
 The PR will then gets approved and merged into
Pull request Master. The feature branch can be deleted
​Bug Branch
​Bug branches follow the same process as feature
Promote release
Pull request branches, the only exception is that bug branches
Create should follow this naming convention bug/<Jira-
release task_id>-annoying-bug
branch

1. http://nvie.com/posts/a-successful-git-branching-model/ 45
Release – Promotion and Release Process

Development branch DEV branch Master branch Prod branch

Bug fix
1 Promote release
pull request
Bug fix

1 Promote release
pull request
​Normal Flow
 A developer clones the repository and creates a feature branch according to the
tasks assigned
Create release  After the feature is complete and ready, the developer creates a pull request and
branch
merges the code into the dev branch
Promote release Golive
2 pull request  After the feature has been tested and signed off by the PO/Testers, the developer
creates another pull request and merges/promotes the dev branch to the master
Create release
branch
notes based on  Upon successful testing of the Master branch/version, a release branch is then
Jira stories created for production release
 The release branch is then deployed to production with a new release version
Merge release
3 pull request number and the old release branch is deleted
Branch: release  Release notes are generated from the user stories associated with all the pull
v_1.2
requests merged
 Release notes are owned by the PO based on the user stories being released to
production

46
Support – Workflow for New Features development and BAU Support
e.g. production enhancements and bug fixes

Development Master/Prod
branch branch

1 Create a
support
Support
branch
branch
5

Feature/567-
6 login hotfix/123-
transfer
Bug/125-kyc
Branch
Feature x

Hotfix branch
2
​Patch / Hotfix Flow on Master
 If there is a feature that needs a fix or a bug that has occurred in the Master branch,
Branch

Pull
Bug x

request the support developer creates a new hotfix/patch branch from the Master/Release
Pull
branch
request
Pull
request Merge hotfix Delete  After the branch has been created, the developer applies a relevant fix to it
3 pull request branch  After the fix has been tested and verified, the hotfix branch is merged into the
Master branch through a pull request
Create
 The fix is also merged into the dev branch for future promotions
4 Merge hotfix release _x.x.x  Re-Tag the Master branch with the fix for fixes audit log and for reverting to the
pull request tagged version later.
 Release notes should also be generated for all the hotfixes performed by the PO.

47
Let’s discuss the team’s view on GIT branching for our squads

Exercise description
​Round 1 ​Tech Lead takes each squad ​Each squad discusses GIT
through GIT branching branching and what this could
framework look like in their context
20 min
2min

​Each squad presents their ​Each squad shares feedback

discussion’s results to other on the squad's results
squads ​2min each
​5min

​Round 2 ​Each squad revises their assumptions

​5min

48
Agenda ​Engineering principles & standards

​DevOps – DORA metrics

​Development – Developer workflow

​Development – Git Branching model (GitFlow1)

​Dev/Ops – CI/CD workflow

49
While Agile focuses on extending development teams towards the customer,
DevOps extends development teams into IT operations

Stakeholders

Customer Development team IT operations

Agile

DevOps

Description Agile improves collaboration between the customer and DevOps improves collaboration between development
the development teams by developing working solutions teams and IT operations by extending development
frequently in order to iteratively align requirements with the teams to IT operations in order to be able to deliver faster
customer through faster release cycles, using continuous delivery

Goal Be closer to the Business Deliver in weeks rather than years

50
10 core engineering practices and capabilities enable software development agility
​Detailed next

1 2 3 4 5

Source code Peer Code Environments Code Unit testing

management review Refactoring

6 7 8 9 10

Continuous Automated Pair Test-driven Just enough

integration testing (func, Programming development documentation
regression)

51
6: Continuous integration / deployment
​Fail Fast, Move Faster - Code is integrated frequently to
detect errors quickly, locate them more easily and facilitate
Continuous a faster build cycle e.g., every few hours or daily
integration
 Failures happen significantly after code was written
 Hard to remember why changes were made
 Long and tense integrations happen infrequently
 A lot of time spent on debugging

Ensure that you have your source code configured

Set-up continuous integration tools – CircleCI

Set-up automated tools for the build process (e.g., Maven for JAVA)
​Exercise
Manual nightly build process in development ​Where are you in
this process?
Automate nightly build in QA

Integrate unit testing framework into build process

Integrate code quality checks including code coverage and security checks into CI process

Integrate automated integration testing into build process

Build is failed if code coverage < 80%

One click deployment to production

Developers treat the Continuous Integration process as “Stop the work”

Automated build to trigger deployment and automated testing (unit, functional and integration)

52
6: Continuous integration/continuous deployment workflow
​Continuous Integration and Continuous Deployment (CICD) tasks performed by an Agile team

Developer creates a pull Developer makes Updates assembly

request for team changes based on Code merged file versions Create Execute
members review the code the feedback into main branch with build number build Unit Test
artifact

Execute
Static
Developer Team members Team members review CI build cycle Fetch external Code
completes review code and changes, approve and initiated libraries Analysis
developing a feature provide feedback merge pull request
Code Coverage and
static code analysis
published to SonarQube

Acceptance tests execute Application is deployed Release is created Application artifact

and deployment is in order of dependency and deployed to packaged and pushed to
complete if tests succeed (DB then API then UI) to DEV environment deployment repository

53
Let’s review the team’s view on CI/CD workflow for our squads

Exercise description
​Round 1 ​Tech Lead takes each squad ​Each squad reviews and
through CI/CD development discusses CI/CD workflow
framework and what it could look like in
their context
2min 20 min

​Each squad presents workflow ​Each squad shares feedback

to other squads on the squad's workflow
​5min ​2min each

​Round 2 ​Each squad revises their workflows

​5min

54