SKILLCERTPRO

1. Introduction

Introduction
DevOps

What

 Misconceptions
o It fits every organization
o It can be applied to any application lifecycle process
o Leads to failure when it comes to implementing DevOps
 As per Wikipedia:
o Set of software development practices

 So it's not a software or application

 It's not set of tools, even though tools are important
 It's a practice where an organization needs to be mature enough to
develop & follow the practice
o Combines software development (Dev), information technology operations
(Ops)

 💡 Do not separate them but combine them!

 Maturity: Departments must be mature enough to work together.
 Most common blocker.
o Shortens development lifecycle, while delivering features, fixes, updates
frequently in close alignment with business objectives.

Why

 Automation: that all organizations go towards

o Changes onto applications
o Delivery of releases
o Creation of the infrastructure
 Agility: How fast you deliver your changes to customer
 Customer satisfaction: Based on how fast you deliver
 Quality through more automation

pg. 1
 SKILLCERTPRO

 Delivery with more value

Other resources

Whitepapers

 Container security in Microsoft Azure

Free

 DevOps engineer - Microsoft Learn

 DevOps labs - Azure DevOps Labs
 Many refers to OpenEdx however it's not free anymore.
o Use 21cskills.africa instead.
 (Video) Ignite | Exam Prep
 Pluralsight Azure Devops Learning Track
 (Video) Microsoft Certification Exam Revision
 Azure DevOps AZ-400 Exam – Study Notes - Gregor Suttie
 Preperation slides

1.1. Design a DevOps Strategy

Design a DevOps Strategy

Greenfield & Brownfield
 Greenfield projects

o Brand new, lacks any constraints imposed by prior work.

 Brownfield projects

o Development and deployment of new software systems within the

immediate presence of existing (legacy) software applications/services

 Greenfield vs Brownfield Ops

pg. 2
 SKILLCERTPRO

Brownfield Greenfield

Fast change, innovative and more tolerant

Slow change, but reliable & scalable
of bugs

Maintaining workloads across on- Maintaining workloads across externally

premise data centers sourced infrastructures

Not cloud platform enabled Cloud platform enabled

Waterfall release cycles Agile development cycles

Legacy, monolithic tools New, composable tools

Legacy, serial processes New, collaborative processes

Project Metrics and KPIs

 Faster Outcomes
o Deployment Frequency
 Increasing the frequency of deployments is often a critical driver in
DevOps projects.
o Deployment Speed
 As well as increasing how often deployments happen, it's important
to decrease the time that they take.
o Deployment Size
 How many features, stories, and bug fixes are being deployed each
time?
o Lead Time
 How long does it take from starting on a work item, until it is
deployed?
 Efficiency
o Server to Admin Ratio
 Are the projects reducing the number of administrators required for
a given number of servers?
o Staff Member to Customers Ratio

pg. 3
 SKILLCERTPRO

 Is it possible for less staff members to serve a given number of

customers?
o Application Usage
 How busy is the application?
o Application Performance
 Is the application performance improving or dropping? (Based
upon application metrics)?
 Quality and Security
o Deployment Failure Rates
 How often do deployments (and/or applications) fail?
o Application Failure Rates
 How often do application failures occur, such as configuration
failures, performance timeouts, etc?
o Mean Time to Recover
 How quickly can you recover from a failure?
o Bug Report Rates
 You don't want customers finding bugs in your code.
 Is the amount they are finding increasing or decreasing?
o Test Pass Rates
 How well is your automated testing working?
o Defect Escape Rate
 What percentage of defects are being found in production?
o Availability
 What percentage of time is the application truly available for
customers?
o SLA Achievement
 Are you meeting your service level agreements (SLAs)?
o Mean Time to Detection
 If there is a failure, how long does it take for it to be detected?
 Culture
o Employee Morale
 Are employees happy with the transformation and where the
organization is heading?
 Are they still willing to respond to further changes?
o Retention Rates
 Is the organization losing staff?
 Lead time vs. cycle time

pg. 4
 SKILLCERTPRO

1.2. Common tools for DevOps

Common tools for DevOps

Project management
 💡 Recommended: Azure Boards in Azure DevOps services, JIRA
 Allows you to:
o Work with different work items.
o Track items on a Kanban board
o Create test cases from items.
o Work with sprints.

Team Collaboration
 💡 Recommended: Microsoft Teams, Slack
 Allows you to:
o Create multiple channels for communication
o Highly accessible as it's available in the browser.
o Collaborate with external suppliers and contractors
o You can integrate slack with Azure DevOps
 E.g. by installing Azure Pipeline app for Slack.
a. Log-in through app
b. Run /azpipelines subscribe [project url] inside a channel

Managing Technical Debt

 Technical Debt = Compromising quality over speed of delivery
 📝 SonarQube for variety of languages.

Build and Release Pipelines

 Jenkins: Continuous Integration & Continuous Delivery
 Azure Pipelines: Continuous Integration & Continuous Delivery

pg. 5
 SKILLCERTPRO

 Octopus: Continuous Delivery

 Bamboo: CI/CD tool from Atlassian
 Travis: CI/CD tool

2. Azure DevOps Overview

Azure DevOps Overview

Interacting with Azure DevOps

Azure CLI

 You can use Azure CLI with the Azure DevOps extension to work with Azure
DevOps
o Enables automate everything as you can script anything and hook it in a
build pipeline.
 Run on Linux, macOS or Windows
 Accessible via browser using Azure Cloud Shell
 It structures as group, sub group and commands
o az –help: display all groups, sub groups and commands
 e.g. vm is sub group of az so you can run az vm -help too
o Find command: az find -q MyCommand

Azure PowerShell

 PowerShell providing services like Shell and Command parsing

 Base PowerShell has 2 format:
o Windows PowerShell
o PowerShell core (Cross platform, Linux and macOS)
 Cmdlets are shipped in modules
o Module is a .DLL that include the code for Cmdlet
o Cmdlet is loaded by loading all its modules
o Get-Module to get list of all loaded modules.

pg. 6
 SKILLCERTPRO

 Azure PowerShell is a PowerShell module that can be added to Windows

PowerShell & PowerShell core
o Accessible via browser using Azure Cloud Shell
 💡 Can store templates or scripts on Azure Storage account and securing them
with SAS token

Azure DevOps tool landscape

 Azure Repos
o Source control for your code.
o You can use Git repositories or Team Foundation Version Control.
 Azure Pipelines
o Helps providing build & release services for continuous integration &
delivery
 Azure Boards
o Agile tools that support planning and tracking work items
 Azure Test Plans
o Tools for testing your applications
 Azure Artifacts
o Allows teams to work with maven, npm and NuGet packages
o Same purpose as JFrog Artifactory

Licensing & Azure DevOps packages

 Check on azure prices

Azure DevOps Server

 📝 Can install on-prem

 Free-trial available
 Formerly known as Team Foundation Server
 Pay month-to-month or 3-year commitment
o Bonus for using cloud services for migration.
o You'll also need Windows or Windows Server licenses for the servers

Azure DevOps Services

pg. 7
 SKILLCERTPRO

 On cloud
 Licensing
o Individual services:
 e.g. Azure Pipelines & Azure Artifacts
 Free tier available
o User licenses
 Basic Plan: First 5 users free then cheap price per user
 Basic + Test plans: More expensive per user
o Management:
 Automated: group-based licensing using AD groups, just add &
remove people from the group.
 Manually: Assign & remove users and assign access level (basic etc.)

Service hooks
 Trigger tasks in external services from Azure DevOps
o E.g. if code is pushed => create a new card in Trello or just call a webhook
 You can integrate with (see updated list in official docs)
o Build and release: AppVeyor, Bamboo, Jenkins, MyGet, Slack,
o Collaborate: FlowDock, HuBot, Office 365
o Customer support: UserVoice, ZenDesk
o Plan and track: Trello
o Integrate: Service Bus, Azure Storage, Grafana, Web Hooks, Zapier

Managing users and their access

 Access levels
o Basic features
 Accessed by first 5 free users.
 Includes almost all of the features.
o Stakeholder features
 Unlimited free users
 Only includes working with Azure boards but not with agile tools
such as Kanban boards, backlogs etc.
 Go to your "Organization Settings" -> Users
o List of users & their access levels (basic / stakeholder)

pg. 8
 SKILLCERTPRO

o You can add new users here:

 A user must have a valid Microsoft account.
 They get an e-mail to join.

Notifications
 You're notified when changes occur to work items, code reviews, pull requests,
source control files, and builds.
 You can be notified via email.
 Notifications are managed in 4 levels:
o Your own notifications in your organization menu, managed by you
o Team notifications, managed by a team administrator
o Project notifications, managed by a member of the Project Administrators
group
o Organization/collection-level notifications, managed by a member of the
Project Collection Administrators group

Extensions
 Install extensions
o Extensions are installed on organization level.
o Allows you to e.g.:
 Introduce new tasks in Azure Pipelines
 Integrate with e.g. JIRA
 Solve pull request conflicts

2.1. Migrations

Azure DevOps Migrations

Migrating code
 TFVC to Git

pg. 9
 SKILLCERTPRO

o Using web portal by clicking import repository (180 days history & less
complex)
o Using git-tfs command line tool (more than 180 and complex)
 git tfs clone https://tfs:8080/tfs/DefaultCollection $/Project1
 Git to Git
o Using web portal by clicking import repository
o Using git mirror

Migrating from on-prem TFS/Azure DevOps to Azure

DevOps Services
 You can migrate easily to Azure DevOps services from on-prem TFS / Azure
DevOps Server
o Using the data migration tool (formerly Database Import Service for
Visual Studio Team Services).
o 💡 Team Foundation Server (TFS) became Azure DevOps Server with the
2019 release of the on-premises product
 Summary of Azure DevOps Server to Azure DevOps Services Migration Guide and
tool.
o Prerequisites
 Ensure your team has active Azure Active Directory
 You can implement Azure Active Directory to synchronize with your
on-premises Active Directory environment.
 Use Azure AD Connect
 Good to enable MFA for access from unknown places
using Conditional Access
o Steps
a. Upgrade TFS
a. Upgrade your Azure DevOps Server or Team Foundation
Server
 It's to get DB scheme as close to the current in Azure
DevOps Service
b. Run "Configuration Features" to enable new features
b. Validate Your TFS Server
. Run validations with migration tool
a. Review logs and fix errors
b. Repeat validation checks

pg. 10
 SKILLCERTPRO

c. Get Ready for Import

. Assign, activate, and map Azure DevOps Services
subscriptions
a. Generate import settings using Migrator prepare command
b. Provide the configurable settings
c. Review the Identity Map log file
d. Create an Azure Storage Container in the same datacenter as
the final Azure DevOps Services organization.
d. Import
. Dry run of end-to-end import
a. Detach the team project collection
b. Create portable backup
c. Upload SQL database backup
d. Generate SAS key
e. Delete previous dry run organizations
f. Rename imported organization
g. Set up billing
h. Reconnect to new organization

3. Agile work management

Agile work management

Benefits
 Allows for faster delivery of product features to your customer
o Leads to increased customer satisfaction.
 Reduced risks since you have small feature releases carried out frequently
 Predictable costs and schedule
 Easily allows for change

Traditional Waterfall Model

 Stages

pg. 11
 SKILLCERTPRO

i. Requirements: Business analysts gets all requirements from customers.

ii. Design stage: Peers & architects design application
iii. Code: Dev teams work with application
iv. System testing
v. User Acceptance Testing
vi. Software release
 Problems
o Release date can be far into the future.
 Can be redundant even before release as business can change
o Bugs & issues detected during the testing phase, it can delay the release
as you repeat stages.
o Software may not comply with the requirements
 E.g. during coding stage design requirements can change which will
not be reflected.
 As result, user may not get what he/she wants

Scrum
1. Have a vision / goal
2. User stories: describes what customer / end user wants
3. Product backlog
o Start taking tasks from user stories
4. Pick tasks from product backlog to sprint backlog
5. Work with them during a sprint
o Sprint = 1-2 weeks
o Sprint results in working functionality
6. Retrospective & review meetings

Reporting (Project metrics)

 Important to avoid frustrations such as late deliveries
 Understand how your work items are progressing in terms of development,
testing, release
o Are work items being tracked to completion?
o Are feature requests being tracked?
o Time remaining for key work items

pg. 12
 SKILLCERTPRO

o Time spend on work items.

 Normally use cumulative flow diagrams to monitor the flow of work.
 📝 Primary metrics are:

o
o Cycle time
 How long it takes to complete one production cycle
 Calculated by work completion time - start of doing work


o Lead time
 Measures work completion time - work requested time

pg. 13
 SKILLCERTPRO


o Burndown: Shows remaining work within a specific time period.
 Burnup is exactly like burndown, except that it plots work
completed, rather than work remaining.


o Velocity
 Indication of how much work a team can complete during a sprint
based.

pg. 14
 SKILLCERTPRO


o Cumulative Flow Diagram
 See the count of work items over time of a Kanban board.

pg. 15
 SKILLCERTPRO

3.1. Azure Boards

Azure Boards
 Allows teams to follow an agile project management approach.
 Has native support for Scrum & Kanban type projects
 Has customizable dashboards
 Has integrating reporting

Terminology
 Work item
o Track your project features & requirements

pg. 16
 SKILLCERTPRO

o Track your code defects or bugs

 User stories
o Helps define the application requirements
o Product owners who will define & rank user stories.
 Boards
o Collaborate with others
o Kanban board: Add, update & review the work items as cards.
 Sprints
o Used working with scrum
o Schedule work items & update them as required.
 Queries
o Helps you list or search for work items based on specific criteria.

📝 Choose a process
Separate
Name When to choose Hierarchy
items

Epic (in Portfolio backlog)

Basic Need for the simplest
� issue (in Product ␀
Process model
backlog) � task

Epic (in Portfolio backlog)

Need for an agile
process e.g. Scrum, can
Agile
track user stories, bugs,
Process
development, test
activities
� feature 〖� user story
(in Backlog) � task (in
issue
Backlog)〗 OR 〖bug (in
Backlog) � task (in
Backlog)〗

Same as agile but impediment

Scrum Need to align with
product backlog item for issue &
process Scrum
instead of user story bug tracking

change
CMMI Need to follow more Same as agile but feature
request, issue,
Process formal project process instead of user story
review, risk

pg. 17
 SKILLCERTPRO

 � Capability Maturity Model Integration (CMMI) is framework to move towards

an more agile approach.
o defines the following maturity levels for processes: Initial, Managed,
Defined, Quantitatively Managed, and Optimizing.

Flow
 Log in to dev.azure.com with your Microsoft account
 You create an organization or use default organization for your user name.
 Create a project
 You have
o Boards
 Boards: Create work items
 Backlogs: See all items from backlog
 Sprints: you see also tasks inside work items
 Can create new sprints with start & end date
 You assign work-items as part of sprints
 Queries
o Repos, Pipelines, Test plans, Artifacts
 You can create work items
o Can be issue, issue or task
 In boards you have columns such as to-do, doing, done
o They are customizable
o You can move work items between them
 You can create tasks inside a work item.

Connecting to GitHub
 Enables linking between
o GitHub commits, pull requests, and issues to work items
 Steps
i. Add connection
 Project settings => Boards => GitHub connections
 Add a new connection
 To authenticate you can use
 Username + Password

pg. 18
 SKILLCERTPRO

 or PAT (Personal Access token)

 or OAuth (only for GitHub Enterprise Server)
 Add GitHub repositories to use with Azure Boards once the
connection is established
ii. Install Azure Boards app for GitHub

4.1. Continuous Testing - Choosing Test and Work Management Tools

Test and Work Management Tools

 Decide tools that works best for you
 Ask questions:
o What sort of testing do you perform?
 E.g. unit testing, system testing, volume testing, system testing
o Do you perform static code analysis?
 Do developers use tools where errors are highlighted?
o Do you perform dynamic code analysis?
 Tests on runtime
o Do you posses any test frameworks?
o Do you test your code against security vulnerabilities?
 Most common by OWASP (Open Web Application Security Project)
o What languages do your company use? E.g. .NET, java, python
 Tools will change based on the support for the underlying
programming language
o Do you use any performance testing tools?
o Do you use any work management tools?
 E.g. change management, configuration management and release
management

Test Tools
 Load Testing
o Load Runner
o Apache JMeter
 UI Testing

pg. 19
 SKILLCERTPRO

o Selenium for web applications

o Xamarin.UITest for running NUnit on android & IOS applications
 Unit testing
o NUnit for .NET
 Static code analysis
o Microsoft.CodeAnalysis (Rosyln APIs)
o PMD, CheckStyle, FindBugs

Test Coverage tools

Name Language Format supported by Azure Pipelines

Cobertura Java ✔️

JaCoCo Java ✔️

BullseyeCoverage C++ ❌

MSTests .NET ❌

NCover .NET ❌

Coverlet .NET ❌

Coverage.py Python ❌

4.2. Continuous Testing - Azure Test Plans

Azure DevOps - Test Plans

 View test cases, runs & results.
 Test suites: Groups test cases together for e.g. different scenarios.
 Test plans: Groups test suites and test cases.

Test cases
 Each work item in Azure Boards can have multiple test cases.
o Create by clicking on Add test
 Each test case consists of multiple steps

pg. 20
 SKILLCERTPRO

o Each step has an Action, Expected result, and Attachments.

 You can assign the test cases to individual testers
 Run test cases manually
o In test plan view you can run your tests one by one.
 You can comment & add screenshots/recording/user actions and
create issues & bugs easily.
o Or you can as passed or failed on directly on Azure Boards
 Run automated test cases
o You can couple test-cases with pipelines and run them automatically
through Test

Load Tests
 See how well your application can behave under certain types of load or stress.
 Types
o You can create URL based load tests
o Import tests from tools such as Visual Studio or Apache JMeter.
o Run HTTP-archive based tests
 Record HTTP sessions.
 Tests from Fiddler can be important this way
 You can set:
o Load pattern that can be:
 Constant: Same amount of users
 Step: Set amount of initial users x, after period of y seconds,
increment number of users by z.
o Set time duration, maximum amount of users, initial user count, warmup
duration, and which browsers to mimic.
o Select user agents:
 Automatically provisioned: You can select the geo-location
 Your own provisioned agents
 After execution you get summary, charts (performance, throughput, errors, tests),
diagnostics and logs.

5. Continuous Feedback

pg. 21
 SKILLCERTPRO

Continuous Feedback
 You can integrate Azure DevOps with Slack & Microsoft teams
 You can then integrate Azure Boards & Pipelines to e.g. :
o Create issues, monitor pipeline results, approve release requests
 Continuous Experimentation mindset
o Design practices to measure end-user satisfaction
o Design processes to capture and analyze user feedback
o Design process to automate application analytics
 Analytics
o Developers need to be able to look for patterns in log messages to
identify if there is a problem in the code.
o Operations need to do root cause analysis across multiple log files to
identify the source of the problem in complex application and systems.

Test & Feedback extension

 Helps teams perform exploratory testing and provide feedback
 Users can install the "Test & Feedback" extension in their browserAzure DevOps
 You can then:
o Capture notes, screenshots with annotations, user actions, page load data
& system information
o Create sessions in the browser itself
o Log sessions against work items in Azure Boards
 E.g. create test cases & bugs
 Access levels:
o Basic
 Can use the extension to perform exploratory testing
 Can request feedback
o Stakeholder
 Can use the extension to respond to feedback requests or to
provide feedback voluntarily.
o Both can use extension to respond feedback requests sent by the team by
choosing the Provide feedback link in the email.

pg. 22
 SKILLCERTPRO

Static Code Analysis

 Reports feedback about the code

 Can be embedded in IDE e.g. Microsoft.CodeAnalysis

 PMD, CheckStyle and FindBugs support for Maven and Gradle is currently
available in Azure DevOps Services (official docs)

o PMD
 PMD is a source code analyzer.
 It finds common programming flaws like unused variables, empty
catch blocks, unnecessary object creation, and so forth
o CheckStyle
 Help programmers write Java code that adheres to a coding
standard
o FindBugs
 Program which uses static analysis to look for bugs in Java code

 Example Apache Maven task (offical docs):

 - task: Maven@3
 inputs:
 #mavenPomFile: 'pom.xml'
 #publishJUnitResults: true
 #testResultsFiles: '**/surefire-reports/TEST-*.xml' # Required when
publishJUnitResults == True
 #codeCoverageToolOption: 'None' # Optional. Options: none, cobertura,
jaCoCo. Enabling code coverage inserts the `clean` goal into the Maven goals
list when Maven runs.
 #codeCoverageClassFilter: # Optional. Comma-separated list of filters to
include or exclude classes from collecting code coverage. For example:
+:com.*,+:org.*,-:my.app*.*
 #codeCoverageClassFilesDirectories: # Optional
 #codeCoverageSourceDirectories: # Optional
 #codeCoverageFailIfEmpty: false # Optional
 #...
 #sonarQubeRunAnalysis: false
 #sqMavenPluginVersionChoice: 'latest' # Required when
sonarQubeRunAnalysis == True# Options: latest, pom
 #checkStyleRunAnalysis: false # Optional
 #pmdRunAnalysis: false # Optional
#findBugsRunAnalysis: false # Optional

pg. 23
 SKILLCERTPRO

5.1. Azure Monitor

Azure Monitor
 Allows continuous monitoring.
 Collect & analyze & act on telemetry data from cloud & on-prem environment
 It collects Metrics (e.g. CPU usage) and Logs
 Tools:
o Insights
 Application Insights
 Containers
 Virtual Machines
 Diagnostics (Microsoft docs)

pg. 24
 SKILLCERTPRO

 You can use the Diagnostics extension of the virtual

machine to collect data.
 E.g. Performance counters, Windows Events logs in
Table Storage, or IIS logs in Blob storage


 Service Map
 Graphical representation of a service, its dependencies
and its settings
 Automatically discovers application components on
Windows and Linux systems and maps the
communication between services


 Monitoring Solutions (easy to go monitoring setups)
o Visualizations
 Dashboards
 Views (from log queries)
 Power BI
 Workbooks (interactive reports, dashboards on steroids)
o Optimizations
 Analyze
 Metric Analytics to query metrics
 Log Analytics to query logs
 Uses ad-hoc query language Kusto

pg. 25
 SKILLCERTPRO

 Respond
 Alerts
 The IT Service Management Connector (ITSMC)
 Provides a bi-directional connection between
Azure and ITSM tools to help you resolve issues
faster.
 E.g. ServiceNow, System Center Service
Manager, Provance, Cherwell
 Allows you to
 Create work items in ITSM tool, based
on your Azure alerts
 metric alerts, Activity Log alerts
and Log Analytics alert).
 Sync your incident and change request
data from your ITSM tool to an Azure
Log Analytics workspace.
 Autoscale
 Integrate
 Logic Apps
 Export APIs

Application Insights
 Monitor and diagnose availability, usage & performance of web apps
 Availability tests: Alerts if your application isn't responding, or if it responds too
slowly.
o URL tests test URL for status code or ping.
o Multi-step tests: Test recorded sequence of URLs and interactions
o Performance tests: Set user load & duration
o or can run custom Azure Functions

pg. 26
 SKILLCERTPRO

o
 Profiler captures data & provides performance traces.

pg. 27
 SKILLCERTPRO

o
 Application Map
o Helps you spot performance bottlenecks or failure hotspots
o KPIs such as load, performance and failures, availability test failures

o
 Smart Detection

pg. 28
 SKILLCERTPRO

o Failure Anomalies: ML based

o Performance Anomalies
o General degradations and issues, like Trace degradation, Memory leak,
Abnormal rise in Exception volume and Security anti-patterns.
 Usage Analysis
o Track users & user behavior.
o what pages they're most interested in, where your users are located, what
browsers and operating systems they use.
o Users: numbers of unique users that access your pages within your chosen
time periods
o Sessions: the number of user sessions that access your site
o Retention
 how many users come back & how often they perform particular
tasks or achieve goals
 show user who used any event or page view and returned to use
any event or page view over x period.
o Events
 Send custom business events & track feature usages
o Allows you to do A | B Testing & measure the success of each, and then
move to a unified version.
o Funnels
 The progression through a series of steps in a web application is
known as a funnel
 E.g. how many users are viewing the home page, viewing a
customer profile, and creating a ticket
o Cohorts
 A cohort is a set of users, sessions, events, or operations that have
something in common
 Defined by an analytics query
 Difference from filters: more adaptable, complex, other team
members in your team can reuse them.
 You can use template gallery e.g.:
 Engaged Users -- by Days Used
o User flow
 Visualizes how users navigate between the pages and features of
your site
 Starts from an initial page view, custom event, or exception

pg. 29
 SKILLCERTPRO

 User Flows shows the events that happened before and afterwards
during user sessions
 Lines of varying thickness show how many times each path was
followed by users
o Impact
 It discovers how any dimension of a page view, custom event,
or request affects the usage of a different page view or custom
event.
 e.g. how load times influence conversation rates

6. Package Management

Package management
 It's possible to manage all aspects of software such as installation, configuration,
upgrade, and uninstall.
 Benefits:
o Reusability: reuse same package in multiple solutions
o Download a package into your solution whenever required.
o Leads to faster development
 Issues with public package managers
o Maintaining governance and control
 E.g. people can use different versions of packages
o Security
 Does it have loopholes? Concerns?
 Developer can use any package to ensure application works.
 Need for managing dependencies
o Applications can just get swarmed into using application dependencies
o There can be no control over the packages being used in application
o Security can also be concern when you are looking at working with public
packages

Some package managers

 apt for Debian Linux environments

pg. 30
 SKILLCERTPRO

 yum for CentOS Linux environments.

 Chocolatey: software management solution built on PowerShell for Windows
operating systems.
 nuget for .NET applications
 npm for JavaScript packages
o package.json
 resides in project root folder
 lists the packages your project depends on
 specifies versions of a package that your project can use
using semantic versioning rules
 makes your build reproducible, and therefore easier to share with
other developers
o .npmrc
 npm gets its config settings from the command line, environment
variables, and npmrc files.
 Can be defined as
 per-user: in home directory of the user ($HOME/.npmrc)
 per-project: project root ($PREFIX/etc/npmrc)
 global: $PREFIX/etc/npmrc
 built-in: unchangeable, in path/to/npm/itself/npmrc
 Azure DevOps Services recommends using two .npmrc files:
a. One .npmrc should live at the root of your git repo adjacent
to your project's package.json.
 Should define registries
b. On the development machine, .npmrc in $home for Linux or
Mac systems or $env.HOME for win systems
 Should contain credentials for all of the registries that
you need to connect to.
 💡 The NPM client will look at your project's .npmrc,
discover the registry, and fetch matching credentials
from $home/.npmrc or $env.HOME/.npmrc
 I[n build task, you give credentials in npm
Authenticate task
 maven: most popular build and dependency resolution tool for Java
o gradle is a Java build tool that can use Maven or Ivy repositories for
dependency resolution.

Security and Compliance

pg. 31
 SKILLCERTPRO

 Developers make use of publicly available packages on the Internet

 Security concerns: Are all security vulnerabilities addressed?
 Licensing problems: e.g. some licenses if you change the code in the package you
must make the package publicly available.
 Some tools
o BlackDuck by Synopsys
 Scan all open source dependencies in your application
 Get issues reported on all possible security vulnerabilities
o WhiteSource Bolt
 tool for scanning open-source dependencies for vulnerabilities and
licensing
o You can install them as extensions for those services for your organization
in Azure DevOps
 You must also install them in a server & buy a license
 Create a service connection that points to your server

6.1. Azure Artifacts

Azure Artifacts
 Service that allows you to organize and control access to packages

 Upstream sources

o Stores your produced packages and proxies & caches packages form
remote feeds
o Remote feeds can be one of the official public sources or a private source.

 Package Graph

o Ensure that any dependencies of your package are also available in your
feed
o You can
 republish them directly (not recommended)
 or consume them from an upstream source.

 ❗ Packages are immutable: You cannot replace / update existing version.

pg. 32
 SKILLCERTPRO

 Permissions

Permission Reader Collaborator Contributor Owner

List and restore/install

packages ✓ ✓ ✓ ✓

Save packages from

upstream sources ✓ ✓ ✓

Push packages ✓ ✓

Unlist/deprecate packages ✓ ✓

Delete/unpublish package ✓

Edit feed permissions ✓

Feeds
 Developers download & use packages from feeds itself
 You can create multiple feeds
 Each feed can have its own set of packages
 Public feeds (project-scoped)
o If the project is private, the feed will be private;
 If the project is public, the feed will be public e.g. accessible by
everyone on internet.
 Private feeds (organization-scoped or project-scoped)
o Can be accessed by whole organization or specific selected people in the
organization.
o Consumers need Personal Access Token with read access to packaging to
download packages.
 Feeds can proxy public sources such as NuGet, npm, Maven and Python.
 You need to create Personal Access Token with write access to packaging to push
packages.

pg. 33
 SKILLCERTPRO

 Feed permissions: Levels of access: Owners, Contributors, Collaborators,

and Readers.

Feed views

 Default: @local, @prerelease, @release, you can add more & delete (except @local)
o The default URI of the feed points to @local that contains:
 all packages published directly to the feed e.g. by npm publish
 packages saved from upstream resources
 You can promote packages to them
 They get URL like ...feed@view/nuget/...

Best practices
 Creating packages as part of a build
o Each repository should only reference one feed
o On package creation, automatically publish packages back to the feed.
o Enable retention policies to automatically cleanup old package versions
o Promote your package to the correct view (have good quality
in @release view)
o If external teams are consuming your package, ensure that
your @release view and @prerelease view are visible across the organization
and/or organization
 Consuming packages from public and internal sources as part of a build
o Each repository should have a unique feed
o Configure upstream sources for public and internal sources
o Sources not in your organization but in the same AAD tenant should be
added using the feed locator
o Ensure that the order of the sources matches your desired package
resolution order
 The feed will check each upstream in order, returning the package
from the first source that has it.
o To avoid confusion, place any public upstreams FIRST in your resolution
order

7. Continuous Integration & Continuous Delivery

pg. 34
 SKILLCERTPRO

Continuous Integration & Continuous

Delivery
Continuous Integration
 Automation for entire application lifecycle.
 Allows you to detect issues & bugs early on in development lifecycle
o It takes more time for issues to resolved when they are detected too late
o Re-testing needs to be carried out.
o Solution:
 Run tests as soon as developer makes a commit to repository
 Based on a schedule that runs e.g. every day
 E.g.
o Commit -> Version control --triggers--> build ---triggers--> deployed to a
test environment --triggers--> test cases are automated --triggered--
> final results -> build is marked as success or failure
 Tools are important e.g. Jenkins, Atlassian Bamboo, TeamCity, Azure Pipelines
 Multi-configuration builds
o e.g. build app for both debug and release configurations on both x86 and
x64 platforms.

Continuous Delivery
 Compliments your continuous integration process.
 Automates deployment of your changes after build.
 Track of your release process quality
o Visualizations about the quality of all the releases pipeline. e.g. adding a
dashboard widget which shows the status of every release.
 Release Notes, functional and technical documentation
o Generate Release Notes Build Task (VSTS)
o WIKI Updater Tasks (VSTS)
o 💡 Treat release documentation & manuals as source-code
 When the product changes, the documentation needs to change as
well
 Multi-configuration deployments

pg. 35
 SKILLCERTPRO

o e.g. for different geographic regions.

Feature Flags

 Allows you to separate your functional release from your technical release
 Decide to have a feature on runtime; enable/disable a feature based on a
boolean

Deployment rings

 Gradually deploying and validating changes in production

 Impact
o Also called blast radius
o evaluated through observation, testing, analysis of telemetry, and user
feedback
 E.g.:
o Canaries* who voluntarily test bleeding edge features as soon as they are
available.
o Early adopter* who voluntarily preview releases, considered more refined
than the canary bits.
o Users who consume the products, after passing through canaries and early
adopters.

Web App Deployment

 Deployment slots
o Allows you to create a new deployment for the web app.
o ❗ Requires Standard or higher plan to be able to use deployment slots.
o App content and configurations elements can be swapped between two
deployment slots, including the production slot.
o Use-cases:
 Create staging environment easily in Web Apps
 Validate in staging before swapping to production
 You can apply Blue Green deployments
 Zero downtime deployment with a auto swap
 Allows you to ensure that all instances of the slot are
warmed up before being swapped into production

pg. 36
 SKILLCERTPRO

 Click on slot => App settings => Auto swap: on

7.1. Deployment Patterns

Deployment Patterns
Feature toggles
 Feature toggles are booleans in code that activates or deactivates a feature in
run-time
 You can deploy first
o Measure soundness of your release in backwards compatibility/bug
perspective
o Release new functionality gradually to different users, or vice versa (scale
down or even rollback functionality and/or binaries).
o Allows for splitting availability of functionality from deployment of
binaries, and gives much more fine-grained decision making then only
"deploy/rollback"
 💡 Always using it a good way to increase your confidence in a new version, since
the new version functions exactly like the old until someone flips a feature toggle.

Blue Green deployments

 The essence of blue-green is deploying all at once
 Easy rollbacks in case of failure.
 Completely automated deployment process
 Zero downtime deployment
 Concept
o Blue version = Current version, users use it
o Green version = New version on production, not yet available
o You redirect users to Green release and at the end it becomes your Blue
release.
 Azure Traffic Manager allows it with its weighted round-robin routing method

pg. 37
 SKILLCERTPRO

Canary Deployments
 The essence of canary deployment is deploying incrementally
 Deploys in small, incremental steps, and only to a small group of people
 It is about to get an idea of how new version will perform (integrate with other
apps, CPU, memory, disk usage, etc).

Rolling deployment
 Slowly replaces currently running instances of the application with newer ones.
 Noting that the old one is removed only when the new is has passed health
checks is important

7.2. Azure Pipelines

Azure Pipelines
 Continuous integration & delivery service
 You can group pipeline steps into jobs, and jobs into stages.
o Building, testing, and release can be automated using stages
 Before Azure had 2 different type of pipelines:
o Build pipeline that results in artifact (.yaml file)
o Release pipeline that takes the artifact and pushes it (UI only)
 It's now the "old way" and new way only uses .yaml same pipeline
for both
 A job is a series of tasks that run sequentially on the same agent
o It has tasks, e.g. install packages & build solution
 Manual Intervention task
 Pauses an active workflow to do a manual task, can have
instructions
 Can continue or reject deployment on time out
o Build is done on an agent (VM or a container), you can design your own
agents.
o Timeouts before job cancelled:
 Forever on self-hosted agents
 360 minutes (6 hours) on Microsoft-hosted agents with public
project or repository

pg. 38
 SKILLCERTPRO

 60 minutes on Microsoft-hosted agents with a private project or

repository.
 Each pipeline has a source version control system
o Can be GitHub, GitHub Enterprise, Azure Repos Git & TFVC, Bitbucket
Cloud, Subversion
 💡 GitHub App is the recommended authentication method with
GitHub.
o You also have option called "Other Git" to e.g. use an on-prem git server.
 Has native support for Python, Java, JavaScript, PHP, Ruby, C#, C++ and Go
 Using secrets:
o ❗ Don't set secret variables in your YAML file.
o Use the script's environment in order to pass secrets to the script.
o Operating systems often log commands for the processes that they run,
and you wouldn't want the log to include a secret that you passed in as an
input.
 Supports visualization of Cobertura & JaCoCo test coverage tools.
o ❗ Only those 2!
 Use the secure files library to store files such as signing certificates, SSH keys on
the server.
 You can use runtime parameters to prompt for input when running manually.
 Retention policy
o Used to configure how long runs and releases are to be retained by the
system
o 💡 You can use the Copy Files task to save your build and artifact data for
longer than what is set in the retention policies

Variables

Variable
 Store values used in the pipeline

 Can be scoped to specific stages.

 There are some predefined build variables

o E.g. System.HostType, Build.ArtifactStagingDirectory , Agent.HomeDirectory .

pg. 39
 SKILLCERTPRO

 Usage

 variables:
 configuration: debug
 steps:
 - task: MSBuild@1 # Use them once
 inputs:
 configuration: $(configuration) # Use the variable
 - task: MSBuild@1 # Use them again
 inputs:
configuration: $(configuration) # Use the variable

Variable Groups

 to store values that available across multiple pipelines.

 also to store secrets and other values that might need to be passed into a YAML
pipeline.

o Usage:

o variables:
o - group: my-variable-group
o steps:
o - script: echo $(myhello) # uses macro syntax
- script: echo $[variables.myhello] # uses runtime expression

Triggers
 Pipelines are triggered by triggers e.g. commits to repository, webhooks etc.
 Defined in .yaml file but ❗ can override YAML continuous integration trigger in
pipeline settings.
 Batch changes

o When a build is running, the system stacks other changes and build them
all at once.

o In yaml:
o trigger:
batch: false

Filters

 You can filter triggers by path, branch or tag.

pg. 40
 SKILLCERTPRO

o E.g.:

o trigger:
o branches:
o include: [ 'master', 'release/*' ]
o paths:
o exclude: [ '/Code/Previous' ]
o tags:
include: [ '*' ]

Pipeline as code
 Triggers, agent, all tasks are defined as a yaml.

o It's version-controlled in the repository.

o Schema:

o name: string # build numbering format

o resources:
o pipelines: [ pipelineResource ]
o containers: [ containerResource ]
o repositories: [ repositoryResource ]
o variables: # several syntaxes, see specific section
o trigger: trigger
o pr: pr
stages: [ stage | templateReference ]

 You can organize steps in jobs and organize jobs as stages.

 If you have single stage you can define jobs jobs: [ job |
templateReference]
 If you have single job you can define steps: steps: [ script |
bash | pwsh | powershell | checkout | task |
templateReference ]

o Example:

o name: $(Date:yyyyMMdd)$(Rev:.r)
o variables:
o var1: value1
o jobs:
o - job: One
o steps:
- script: echo First step!

Agents
 You can choose Microsoft agents.

pg. 41
 SKILLCERTPRO

o Maintenance & upgrades are taken care by Microsoft

o On each run it's a new VM/container & discarded after one use
 You can also build your own agent as self-hosted agent.
o You control the VM & container
o Can run in Azure or on-prem
o You need to create Personal Access Token with Agent Pool scope to
configure your agent.
o 💡 Good way to integrate on-prem & cloud systems

Agent pools
 Organize agents into agent pools for easier management.
 Pools are scoped to & visible in the entire organization
 Default agent pools
o Default pool: Use it to register self-hosted agents that you've set up.
o Azure Pipelines hosted pool with various Windows, Linux, and macOS
images.
 Examples 📝:
 Ubuntu 1604: Run jobs on a Linux based VM
 macOS: Build & release on Mojave macOS
 ❗ For builds and releases running on Microsoft-
provided macOS agents, your data will be transferred
to third party data centers in US or EU (offical docs).
 Windows:
 Windows 2019 with VS2019
 VS2017
 Hosted: Older versions of Visual Studio installed on
Windows Server 2012
 Hosted Windows Container
 Removed after March 23, 2020
 The list here can be outdated, check the latest from the docs
 📝 Permissions:
o Organization- or collection-level:
 Reader: can view agent pool & its agents to e.g. monitor health
 Service Accounts: Can create an agent pool in a project

pg. 42
 SKILLCERTPRO

 Administrator: Can also register & unregister agents from an

organization agent pool
o Project-level:
 Reader: can view agent pool & its agents to e.g. monitor health
 User: Can use the pool when authoring pipelines.
 Creator: Can use the pool when authoring build or release
pipelines.
 Administrator: Can manage membership for all roles of the pool,
as well as view and use the pools

Static Code Analyzer tools in Azure Pipelines

 You can run source code analyzer tools as part of your build pipeline.
 For Java based projects you have tools such as PMD, CheckStyle and FindBugs
 In Azure Pipelines yaml file, use maven task template and simply give three
parameters:
o checkStyleRunAnalysis: true, pmdRunAnalysis: true, findBugsRunAnalysis:
true
o In build summary you can then see information in Code Analysis Report
section
 Analysis results are also published as an artifact.

Environment
 Collection of resources that can be targeted by deployments from a pipeline.

o E.g. VMs in secondary and primary site

 You run a script in each VM to register them in an environment

 Can include Kubernetes clusters, Azure web apps, virtual machines, databases.

 E.g. Dev, Test, QA, Staging, and Production.

 E.g. following runs for each resource in the environment:

 - stage: deploy
 jobs:
 - deployment: DeployWeb
 displayName: deploy Web App
 pool:
 vmImage: 'Ubuntu-latest'

pg. 43
 SKILLCERTPRO

 # creates an environment if it doesn't exist

 environment: 'smarthotel-dev'
 strategy:
 runOnce:
 deploy:
 steps:
- script: echo Hello world

Approvals

 Manually control when a stage should run using approval from user(s)
 Can be pre-deployment, or post-deployment approvals
 Defined in environment level or (legacy) release pipeline UI
 💡 Commonly used to control deployments to production environments.
 You can assign a timeout for the approval for auto-rejection after time is out.

Parallel Jobs
 Parallel job = One job at a time in a pipeline
o Pipeline = collection of jobs
o Each job consumes a parallel job that runs on an agent
 When there aren't enough parallel jobs available for your organization
o the jobs are queued up and run one after the other.
 Free tier
o Public project:
 10 free Microsoft-hosted parallel jobs that can run up to 360
minutes each time
 10 free self-hosted parallel jobs (unlimited parallel jobs)
o Private project:
 1 free Microsoft hosted parallel job, can run up to 60 minutes each
time
 Limit of 30 hours per month
 1 free self-hosted parallel job (unlimited parallel jobs)

Service connections
 Allows you to connect to external and remote services to execute tasks in a job.
 Service connections are created at project scope

pg. 44
 SKILLCERTPRO

o So a service connection created in one project is not visible in another

project.
 Common examples:
o ARM, Bitbucket Cloud, Chef, Docker Registry, GitHub, External Git, Jenkins,
Kubernetes, npm, SSH, Subversion, Visual Studio App Center...
o Generic service connection to any other type of service or application
 Consists of "Connection Name", "Server URL", "User name" and
"Password/Token Key" parameters
 You can secure a service connection in three different levels of permissions:
i. User: Creator, Reader, User, Administrator
ii. Pipeline: control which YAML pipelines are authorized to use this service
connection
iii. Project: allows cross project sharing of service connections
 You can create custom service connections using service endpoints
o Consists of "Service name", "Description", "Server URL", "Certificates or
tokens", "User names" and "passwords"
o See official docs for more information.

Integrate with third party

 Integrate with ServiceNow
o Steps:
 Install ServiceNow extension on Azure DevOps
 Install Azure DevOps application on ServiceNow
o Allows you to:
 Use ServiceNow gates
 Monitor change management process from releases
 Keep ServiceNow change requests updated with the deployment
result
 Microsoft Teams & Slack: Install Azure Pipelines app
 Jenkins through Service Connection, PAT from Jenkins and Team Foundation
Server plugin

7.2.1. Azure Pipelines - Container Agents

pg. 45
 SKILLCERTPRO

Container agents
 The agent will first fetch and start the container.
o Then, each step of the job will run inside the container
 or you can set agent on task level

Linux agents
 E.g.:

 pool:
 vmImage: 'ubuntu-16.04'
 container: ubuntu:16.04
 steps:
- script: printenv

 On your agent host, ensure:

o Docker is installed
o Agent has permission to access the Docker daemon

 Container requirements:

o Bash
o glibc-based
o Can run Node.js (which the agent provides)
o Does not define an ENTRYPOINT
o USER has access to groupadd and other privileges commands without sudo

Windows agents
 E.g.

 pool:
 vmImage: 'windows-2019'
 container: mcr.microsoft.com/windows/servercore:ltsc2019
 steps:
- script: set

Service containers

pg. 46
 SKILLCERTPRO

 Spin up multiple containers

 Automatically create, network, and manage the lifecycle
 Read more: Microsoft documentation

Service container example

resources:
containers:
- container: my_container
image: ubuntu:16.04
- container: nginx
image: nginx
- container: redis
image: redis

pool:
vmImage: 'ubuntu-16.04'

container: my_container

services:
nginx: nginx
redis: redis

steps:
- script: |
apt install -y curl
curl nginx
apt install redis-tools
redis-cli -h redis ping

 Fetches the latest nginx and redis containers from Docker Hub and then starts
the containers
 The containers are networked together so that they can reach each other by
their services name.
 Pipeline then runs the apt, curl and redis-cli commands inside
the ubuntu:16.04 container.
 From inside this job container, the nginx and redis host names resolve to the
correct services using Docker networking
 All containers on the network automatically expose all ports to each other

7.2.2. Azure Pipelines - Release Pipelines

pg. 47
 SKILLCERTPRO

Release pipelines (legacy)

 GUI only
 It's legacy and will be replaced by multi-stage pipelines.
 A release pipeline consists of different stages per environment e.g. QA / Staging /
Production
 You can:
o Edit a particular release only (not entire pipeline)
o Specify retention period for the releases
 How many days + minimum number of releases
 Global release retention policy
 For on-premises Team Foundation Server
 ❗ In Azure Pipelines, you can view but not change
these settings for your project.
 Maximum retention policy sets the upper limit for how
long releases can be retained for all release pipelines
 Default retention policy sets the default retention values
for all the release pipelines
 Destruction policy helps you keep the releases for a certain
period of time after they are deleted
o Minimum number of releases to retain for each pipeline

Deployment pools & groups

 Deployment pools are environments in a release pipeline.
o exists at the account level & contains agents (targets)
 can be shared in different projects
 Permissions
 Reader: can view
 Creator: can view & creator
 User: can view & use but cannot manage or create
 Administrator: Can administer roles, manage, view and use
deployment groups
 Deployment groups
o Layer over pools which makes these targets available to release definitions
in a project

pg. 48
 SKILLCERTPRO

 e.g. "Dev", "Test", "UAT", and "Production".

o Project-scoped: Exists in a single project
o Permissions:
 Reader: Can only view deployment pools.
 Service account: Can view agents, create sessions, and listen for
jobs from the agent pool.
 User: Can view and use the deployment pool for creating
deployment groups.
 Administrator: Can administer, manage, view and use deployment
pools.
 You can deploy gradually to ensure application is available to the customers at all
time:
o Configure Maximum number of targets in parallel

Gates
 Not yet available for multi-stage pipelines, see GitHub issue
 Collect information from external services
o then decide if a stage should run or not.
 Use-cases:
o Incident and issues management, e.g. :
 Ensure deployment occurs only if no priority zero bugs exist
 Validate that there are no active incidents takes place after
deployment
o Seek approvals outside Azure Pipelines, e.g.:
 Notify legal approval departments, auditors, or IT managers about a
deployment by integrating with approval collaboration systems
such as Microsoft Teams or Slack
o Quality validation, e.g. :
 Allow release only if code coverage >= 90
o Security scan on artifacts, e.g.:
 Anti-virus checking, code signing, and policy checking..
o User experience relative to baseline, e.g.:
 Ensure the user experience hasn't regressed from the baseline state
o Change management, e.g.:
 Wait for change management procedures in ServiceNow before
deployment

pg. 49
 SKILLCERTPRO

o Infrastructure health e.g.

 Execute monitoring and validate the infrastructure against
compliance rules after deployment
 Gate can be:
o An Azure function that returns success or fail
o Based on Azure Monitor alerts
o Invoking an external REST APi to check success or fail
o Query work items in Azure boards
 ❗ Queries must be in "Shared Queries" folder.
o Query compliance from within Azure Policies
 You can have multiple gateways between stages.
o Pre-deployment gates: runs before the deployment stage
o Post-deployment gates: runs after the deployment stage
 Can have delay before evaluation
o E.g. wait for application reach a steady operational state before running
penetration tests
 Can have evaluation options that applies to all gates:
o timeout after which gates fail
o time between re-evaluation of gates: sampling interval
o minimum duration for steady results after a successful gates evaluation

7.2.3. Azure Pipelines - DevTest Labs

Azure DevTest Labs

 VMs that can have lab policies:
o to automatically shut down & start up VMs
o to have caps e.g. max VMs per user or max costs
 To be used for e.g.:
o Conduct compatibility and automated testing with reusable environment
templates
o Provide virtual machines for hackathons that automatically expire after the
event.

pg. 50
 SKILLCERTPRO

Use DevTest Labs in Azure Pipelines

 Two use-cases with Azure Pipelines:
o Cheap way to create continuous test environments
 E.g. for development and test environments
o Create a VM with golden image to execute a specific task e.g. build
Erlang/Hack.
 During the build/test phase
o you can add ARM templates & supporting files to the build sources
o so that during the release phase the exact configuration used to test with
is deployed to production.
 You can use the Azure DevTest Labs Tasks extension with tasks such as:
o Create Azure DevTest Labs Environment
 💡 You can instead use an ARM template to deploy the
environment instead of this task.
o Deploy ARM template to existing Azure DevTest Labs Environment
 You can provision both Azure PaaS resources & IaaS VMs
 Read more: Microsoft documentation

7.3. Jenkins

Jenkins
 Tool for continuous integration & delivery, see jenkins.io
 Multi-OS & open-source
 Supports many languages with rich set of plugins
 You can use webhooks for auto-trigger from GitHub

Jenkins & Azure Repos

 To enable Jenkins to fetch from Azure Repos (see lab):
o Steps:
 Create a personal access token in Azure DevOps with read access
& add it in Jenkins

pg. 51
 SKILLCERTPRO

 Install TFS plugin (yet to be renamed Azure DevOps!) to Jenkins to

allow:
 TFVC: Poll & read & label
 Git: Push trigger, build information fetching
o For auto trigger from Azure Repos:
 Create service hook in Azure repos to trigger Jenkins build (official
docs)
o To nest a Jenkins Job with Azure Pipelines
 Add a service connection to Jenkins.
 You can use Jenkin tasks such as Queue Jenkins Job, Download
artifacts ...
 💡 Recommended as you can have end-to-end traceability from
work items to source code to build and release pipelines.

7.4. SonarQube

SonarQube
 Open-source code Analysis tool, sonarqube.org
 Helps you to see your projects technical debt
 Detect bugs, vulnerabilities, code smells, coverage...

SonarQube & Azure Repos

 See labs
 📝 Steps:
o (If you don't have SonarQube) Create VM with container & SonarQube
image
 Ensure port 8080 is open on VM/container to be able to
comunicate with Azure DevOps
ii. Create a project in SonarQube
 It'll give you authentication token you'll need (you can also use an
existing token)
 Also gives you scripts to run for different languages/frameworks
 You'll use name of this project in service connection.

pg. 52
 SKILLCERTPRO

iii. You create service connection for SonarQube.

 You can use token from SonarQube project or generate a new
token in security section of SonarQube
iv. In organization settings add SonarQube extension
 Gives you tasks to execute in following order:
b. Prepare Analysis Configuration
 Before executing the build
c. Run Code Analysis
 Not required for Maven or Gradle projects, because scanner
will be run as part of the Maven/Gradle build.
d. Publish Quality Gate Result
 Optional to display the Quality Gate status in the build
summary
v. You can analyze results in SonarQube server
 Set-up a pull-request integration:
. Create a Personal Access Token in Azure DevOps
i. Configure SonarCloud to analyze pull requests
 In Pull Requests tab set provider to Azure DevOps Services
ii. Configure the branch policy for the project in Azure DevOps
 Set SonarQube pipeline as build definition
iii. Block pull requests if the Code Quality check failed
 Branch Policy => Add status policy => SonarCloud/quality gate and
mark requirement as Required
 Tasks to run:
o Prepare Analysis Configuration
o Run Code Analysis (not required)

8. DevSecOps

DevSecOps (Security)
 Philosophy of integrating security practices within the DevOps process.
 Makes security a responsibility of everyone on the team

Best Practices

pg. 53
 SKILLCERTPRO

 Provide training
 Define requirements
o Minimum-security baseline that takes account of both security and compliance
controls
o Ensure these are baked into the DevOps process and pipeline
o Check at least for OWASP Top 10, SANS Top 25
 Top 5 from the Top 10
a. Injection
 Never trust any user input
b. Broken authentication
 Use custom or own authentication system rather than well
known authentication systems
c. Sensitive data exposure
d. XML external entities
e. Broken access control
 Roles and permission poorly implemented or not
implemented at all
 Define metrics and compliance reporting
 Use software composition analysis (SCA) and governance
o Evaluate third party components for security & licensing
 Perform threat modeling
o Helps you to
 More effectively and less expensively identify security vulnerabilities
 Determine risks from those threats
 Make security feature selections and establish appropriate mitigations
o At the very least, threat modeling should be used in environments where there is
meaningful security risk.
 Use tools and automation
o Integrate Static Application Security Testing (SAST) into your IDE
o Integrate Dynamic Analysis Security Testing (DAST) tools into CI/CD
o Choosing tools:
 Tools must be integrated into the CI/CD pipeline.
 Tools must not require security expertise.
 Tools must avoid a high false-positive rate of reporting issues.
 Keep credentials safe
o Do not store credentials in code
 Consider using a bring-your-own-key (BYOK) solution that generates keys
using a hardware security module (HSM).
 Use continuous learning and monitoring

pg. 54
 SKILLCERTPRO

Threat Modeling
 Do it as soon as and often as possible
o Not only when releasing a new feature because single line of code can change
everything
 Microsoft Threat Modeling tools is a tool for threat modeling.
 Steps:
i. Defining security requirements.
ii. Creating an application diagram.
iii. Identifying threats.
iv. Mitigating threats.
v. Validating that threats have been mitigated.

Security tools

Open source library vulnerabilities and license

 Whitesource – Open source library vulnerabilities and license
 Black Duck – Open source library vulnerabilities and license

Static security testing

 Checkmarx – Static Application Security Testing (SAST) tool
 BinSkim – A binary static analysis tool that provides security and correctness results for
Windows portable executables

Penetration testing tools

OWASP ZAP

 Full name: OWASP Zed Attack Proxy (ZAP)

 Passive tests with CI and active tests with nightly build
 💡 Better to use the OWASP Zap/Weekly docker container within Azure Container
Services to get latest updates.

Secure DevOps Kit for Azure (AzSK)

 ❗️ It's replaced by AzTS as of 2021.

pg. 55
 SKILLCERTPRO

 PowerShell cmdlets and guidance for security best practices

 Scans Azure resources (run PowerShell command) and provide CSV file with the scan
result and recommendations as fix script.
 Allows running the recommendation fix with command to automatically fix issues
 On Azure DevOps you can use Secure DevOps Kit (AzSK) CICD Extensions for Azure
 Allows for scanning whitelisted/blacklisted ports.
 👀 Official page

Azure Tenant Security Solution (AzTS)

 Cloud security and compliance solution using native security capabilities in Azure
 Scans Azure subscriptions and resource configurations across multiple subscriptions
 Next level AzSK, similar to AzSK Continuous Assurance (CA) in central-scan mode.
 Integrates with Azure/Azure Security Center policies
 Requires installation on Azure with a management identity with reader permissions
 Stores results in Log Analytics Workspace that can be sent to Power BI

Secret management
 Azure Key Vault

Governance & compliance

 Azure policy
o Natively create policies, e.g, you create a policy from Azure (portal/CLI/REST API)
to prevent creation of VM with high cost or specific region
 Azure Blueprints (Polices, RBAC, ARM)
o Container above resource group / subscriptions.
o Helps to apply roles & ARM templates in a repeatable configuration manner
 Azure Artifacts to stay in control of the dependencies that are consumed

Azure Security Center

 Centralizes security policy management
 Continuous security assessment
 Actionable recommendations
 Prioritized alerts and incidents

pg. 56
 SKILLCERTPRO

Penetration testing
 To find out weaknesses in the system
 Passive Tests or Passive Scan
o Scan the target site as is but don't try to manipulate the requests to expose
additional vulnerabilities.
o Runs fast and good candidate for CI as they complete in minutes
 Active Tests or Active Scan
o Also called as dynamic or fuzz tests because
o Used to simulate many techniques that hackers commonly use to attack websites
o Tries a large number of different combinations to see how the site reacts to verify
that it doesn't reveal any information
o CI/CD pipeline should run within a few minutes, so you don't want to include any
long-running processes.
 Nightly tests are a good idea.
 OWASP ZAP is a free penetration testing tool for beginners to professionals
o OWASP Zap/Weekly docker container within Azure Container Services ensures
the image is always updated.
 Example pipelines:
o Application CI/CD:
a. Build
b. Static Security Scan
c. Package
d. Deploy
e. Passive Pentest: Pull OWASP Zap Weekly => Start Container => Run
Baseline (1 to 2 min) => Report Results => High alerts - Fail, Release, All
Alerts - Create bugs
o Nightly OWASP ZAP Pipeline
 Nightly Schedule
 Application Full Active Scan: Pull OWASP Zap Weekly => Start
Container => Spider Site => Run Active Scan => Report Results => High
Alerts - Fail, Release, All Alerts - Create Bugs

Continuous Security Validation

 Should be done at every point in application lifecycle:

pg. 57
 SKILLCERTPRO

Stage Application CI / CD Feedback Nightly Test Runs

• Static Code
IDE / • Code Review
Analysis • Code
Pull Comments • Static Code -
Review • Work Item
request Rule Warnings
Linking

• Static Code • OSS Library

Analysis • OSS Vulnerabilities • OSS
CI Vulnerability Scan • License Violations • -
Unit Tests • Code Failed Unit Tests • Static
Metrics Code Rule warnings

• Load and
• Passive Pen Test • • Pen Test Issues • SSL Performance Testing •
Dev SSL Scanner • Issues • Performance Automated Regression
Infrastructure Scan Issues • Regression Bugs Testing •
Infrastructure Scan

• Pen Test Issues • • Active Pen Test •

Test • Infrastructure Scan
Infrastructure Issues Infrastructure Scan

 CI/CD steps:

i. When interacting with master branch.

 Can be pull request / code commits / code merges...
 PR to master should require a code review
 Possible with a branch policy in Azure Repos
 Commits should be linked to work items for auditing why the code
change was made and require a CI.
 Should have a PR-CI (Pull Request Continuous Integration)
 💡 You can use Static Code Analysis Code Review tools
 Highlights any security / vulnerability issues in code
ii. Continues integration through e.g. Jenkins
 Difference from PR-CI => it does packaging also.
 Should do Static Code Analysis with e.g.
 Visual Studio Code Analysis and the Roslyn Security Analyzers
 Checkmarx - A Static Application Security Testing (SAST) tool
 BinSkim - Binary static analysis tool that provides security and
correctness results for Windows portable executables

pg. 58
 SKILLCERTPRO

 See other tools

 Many tools integrate into the Azure Pipelines build process.
 Scan for 3rd party packages for vulnerabilities & OSS (open-source)
license usage
 See WhiteSource, Black Duck
 Also run unit tests & report code metrics
iii. Promoting to development & test (QA) stage
 Verify that there are not any security vulnerabilities in the running
application
 = Automated Penetration testing
 Infrastructure scanning
 Is your hardware up-to-date? Are all tools installed on
development machine? Are all tools patched with latest security
patches?
 E.g. Azure Security Center to report and Azure Policies to prevent.
 E.g. Both immediate & nightly runs of port scanning to check
against blacklisted/whitelisted ports
 Load and Performance testing, Automated Regression Testing

Container Security
 Container security best-practices: (whitepaper)
o Scan for vulnerabilities before pushing images to registry.
o Continue scanning in the registry because new vulnerabilities are discovered all
the time.

Continuous Assurance
 Tracks configuration drift
o Checks for "drift" from what is considered a secure snapshot of a system
 Treat security truly as a 'state' as opposed to a 'point in time' achievement
o important in today's context when 'continuous change' has become a norm.
 2 types of drift:
o Drift involving 'baseline' configuration
 Often pre-defined/statically determined ones
 E.g. SQL DB can have TDE encryption turned ON or OFF
o Drift involving 'stateful' configuration
 Cannot be constrained within a finite set of well-known states.
 E.g. the IP addresses configured to have access to a SQL DB

pg. 59
 SKILLCERTPRO

 For more info & scripts, see Secure DevOps Kit for Azure

8.1. Azure Key Vault

Azure Key Vault

 Secrets
o Securely store secrets and passwords
o Storing in code is not secure.
 Keys
o Create and control encryption keys
o E.g. for an encrypted storage integrated with e.g. Azure Storage
o Each key can have activation & expiration date
o Creating a key
 Can generate & import or restore from a backup
 Certificates: Provision, manage and deploy public and private SSL/TLS
certificates
 Controlled through:
o Management plane
 Manage Key Vault itself, e.g. create & delete vaults
 Uses Azure Resource Manager role-based access control (RBAC)
o Data plane
 Working with key stored in the vault
 Uses Key Vault access policy
 Using CLI:
o Create: az keyvault create --name "Contoso-Vault2" --resource-group
"ContosoResourceGroup" --location eastus
o Add a secret: az keyvault secret set --vault-name "Contoso-Vault2" --
name "ExamplePassword" --value "hVFkk965BuUv"
o Show a secret: az keyvault secret show --name "ExamplePassword" --
vault-name "Contoso-Vault2"

Access Policies
 On vault level

pg. 60
 SKILLCERTPRO

 Each policy has:

o A principle: Can be a user or an application

o Key permissions
 Key Management Operations: Get, List, Update, Create, Import,
Delete, Recover, Backup, Restore
 Cryptographic Operations: Decrypt, encrypt, unwrap key, wrap
key, verify, sign
 Privileged Key Operations: Purge
o Secret permissions
 Secret Management Operations: Get, List, Update, Create, Import,
Delete, Recover, Backup, Restore
 Privileged Secret Operations: Purge
o Certificate permissions
 Certificate Management Operations: Get, List, Update, Create,
Import, Delete, Recover, Backup, Restore, Manage Contacts,
Manage Certificate Authorities, Get Certificate Authorities, List
Certificate Authorities, Set Certificate Authorities, Delete Certificate
Authorities
 Privileged Certificate Operations: Purge

 📝 Retrieve using PowerShell:

o Key: $keyUrl = (Get-AzureKeyVaultKey -VaultName "testvault" -Name

"keyname").Key.Kid
o Secret: $secretText = (Get-AzureKeyVaultSecret -VaultName "testvault" -
Name "secretname").SecretValueText

 📝Reference it in an ARM template:

 {
 "$schema": "https://schema.management.azure.com/schemas/2015-01-
01/deploymentParameters.json#",
 "contentVersion": "1.0.0.0",
 "parameters": {
 "adminLogin": {
 "value": "exampleadmin"
 },
 "adminPassword": {
 "reference": {
 "keyVault": {
 "id": "/subscriptions/<subscription-id>/resourceGroups/<rg-
name>/providers/Microsoft.KeyVault/vaults/<vault-name>"
 },

pg. 61
 SKILLCERTPRO

 "secretName": "ExamplePassword"
 }
 },
 "sqlServerName": {
 "value": "<your-server-name>"
 }
 }
}

Authorize access
 Using RBAC
i. Ensure entity to give access to user/application exists:
 For application
 Create service principal az ad sp create-for-rbac -n
"http://mySP"
 For users, you can add following to RBAC list of the vault:
 Add individual users (not recommended)
 💡 Add AD groups (recommended)
ii. Then give your principal access to the vault:
 az keyvault set-policy -n <your-unique-keyvault-name> --spn
<ApplicationID-of-your-service-principal> --secret-permissions
get list set delete --key-permissions create decrypt delete
encrypt get list unwrapKey wrapKey
 Using managed identity
i. In application => add system-assigned identity
 E.g. for web-app: az webapp identity assign --name myApp --
resource-group myResourceGroup
 Or Identity section on portal.
ii. In key vault => Allow access:
 az keyvault set-policy --name myKeyVault --object-id
<PrincipalId> --secret-permissions get list

9. Source code version control

Source code version control

Introduction

pg. 62
 SKILLCERTPRO

 Helps developers create & maintain versions for their source code.
 Developers can collaborate on code & track changes
 Required for any development project.
 git is most commonly used.
 Azure DevOps has Azure Repos that support git and Team Foundation Version
Control.
 Flow
o Each developer has their own copy
o They make changes
o Push their changes into central repository
o Other developers pull changes into their repositories

Conflicts

 If developer makes changes to same line of code it results in conflict.

 Other developers must resolve the conflict.
 Other solution: Locking
o Single developer locks a part of the code.
o Other developers cannot work on the locked part of the code.
o Git does not provide any locking functionality, since it is decentralized
 git lfs that's supported by some git providers like GitHub
supports File Locking
o Centralized solutions such as SVN & ClearCase supports it.

Centralized vs Decentralized repositories

 In a centralized repository like SVN everyone commits to a single repository.
o If you lose server without backups, then everything is gone
 In a decentralized repository like git everyone commits to their own local copy
(repository) of the remote repository and then pushes their changes to the
remote repository.
o You can work offline, as you don't need network to e.g. diff, check history,
commit ,merge, switching branches, obtaining other revisions of a file

Version Control Systems

pg. 63
 SKILLCERTPRO

Azure
descriptio Repos Terminolog
name type license
n suppor y
t

most
distribute open-
git popular, a ✔️ pull & push
d source
lot tooling

branches
= paths,
centralize proprietar granular check-in &
TFVC ✔️
d y permission check-out
s down to
a file level

used by
.eg.
distribute open- Facebook
mercurial ❌ pull & push
d source & Mozilla,
simpler
than GIT

branches
= paths,
svn (apache centralize open- granular commit &
❌
subversion) d source permission update
s down to
a file level

branches
= paths,
Helix
centralize proprietar granular check-in &
Core (Perforc ❌
d y permission check-out
e)
s down to
a file level

pg. 64
 SKILLCERTPRO

Azure
descriptio Repos Terminolog
name type license
n suppor y
t

file-based
architectur
e (where
everything
centralize proprietar
ClearCase happens ❌
d y
e.g.
tagging at
the file
level)

Central source code repository

 Git is distributed where each developer has own copy of the repository
o so who has the master?
 Many teams uses centralized Git model
o Possible to setup a Git server to host the repositories, but then
maintenance is needed
o or there are e.g. GitHub, Atlassian, Azure Repos on internet.

Team Foundation Version Control

 💡 In Azure DevOps: Choose TFVC for granular permissions on branch (=path)

level, otherwise use GIT.
 Types
o Main Only - commit your changes to the main branch and optionally
indicate development and release milestones with tags


 ❗ The mutability and lack of history with TFVC labels can add risk of
change control.
o Development isolation

pg. 65
 SKILLCERTPRO

 Have stable main branch, you can branch one or

more dev branches from main
 Enables isolation and concurrent development


o Feature isolation
 Special derivation of the development isolation
 Branch one or more feature branches from main, as shown, or from
your dev branches


o Release isolation
 Introduces one or more release branches from main
 Never forward integrate (FI) from main.
 Patches and hot fixes made to the release can be reverse
integrated (RI) back to the main.


o Servicing and Release isolation
 Allows e.g. service packs
 release branch should never be modified
 Never forward integrate from main to servicing,
and servicing to release.
 Although not recommended, you can continue to evolve by
introducing e.g. hotfix branches to releases=> Servicing, Hotfix,
Release isolation

pg. 66
 SKILLCERTPRO

9.1. Git

Using git
 You can use git CLI
 You can also use other tools such as Visual Studio (through Team Explorer)
o Connect directly to Azure DevOps with your Microsoft account to work with
Azure Repos.

Git CLI
 Create a repository
i. Download git
ii. run git init on a folder where you'll want to have your repository
 You get master branch ready
 HEAD shows the current branch in start it's master
iii. Create a file called filename.txt
iv. Run git add filename.txt or git add . (for all files in the folder)
v. Commit your changes with a message git commit -m "first commit"
 Useful commands
o git log: to show all commits
 Each commit has SHA hash you can see here.
 You can revert a commit using git revert commit-sha (or first 3 chars of
the sha)
o git fetch: to download remote-tracking branches
o git pull: does git fetch followed by a git merge FETCH_HEAD to update your
files to latest in remote.

Branching

pg. 67
 SKILLCERTPRO

 Working with multiple developers -> Changing master branch is not good.
o Some changes may not be complete or working as it should.
o Making changes to master branch itself can make other developers get incorrect
or non-working code.
 Instead create another branch from master and each developer work on their own
branches
o Main master branch remains intact with working code.
o You can create multiple branches
o Branches are lightweight
o Copies of your code are not made.
o E.g. separate branch for each bug fix / feature.
 In git:
o Run git status to see the base you're currently working from
o Create new branch using git branch <branch-name>
o Switch to the new branch using git checkout <branch-name>
o At any point, when you want to merge your changes, you can run git merge
 First you need to be on master git checkout master
 Then you run git merge feature-branch

Branching Workflows
 Long-Running Branches
o e.g entirely stable in their master branch
 Topic branches
o Short-lived branches for particular feature / work
o There are usually multiple topic branches
 Progressive-stability branching
o Long-running stable master
o Another parallel branch named develop or next to test stability
o It isn't necessarily always stable, but whenever it gets to a stable state, it can be
merged into master
o Used to pull in topic branches & test so they don't introduce bugs.

pg. 68
 SKILLCERTPRO

Centralized Workflow

 There's only one branch master and everyone commits to master

 Good for teams transitioning from SVN.
 See also trunk-based development

Feature Branch Workflow

 All feature development should take place in a dedicated branch instead of

the master branch.
 Logical extension of Centralized Workflow.
 Cons:
o Easy for multiple developers to work on a particular feature without disturbing
the main codebase.
o master branch should never contain broken code, which is a huge advantage for
continuous integration environments.
o Enables PR's

Gitflow Workflow

 Strict branching model designed around the project release

 Based on Feature Branch Workflow
 Development:
o Feature branches are branched off of the develop branch
o Finished features and fixes are merged back into the develop branch
 Releasing:
o When it is time to make a release, a release branch is created off of develop.
o When the release is finished, the release branch is merged into master and
into develop too

pg. 69
 SKILLCERTPRO

 � See Gitflow animation

 � The Gitflow Workflow was first published in a highly regarded 2010 blog post
from Vincent Driessen at nvie

Release-flow

 Based on feature branch workflow

 Enforced in Microsoft
 Development:
i. Dev creates a new branch off of our main integration
 💡 Use feature flags to have short-lived branches
ii. Devs push their local branch to a branch on the server, and open a pull request
iii. When tests are passed, changes are merged into master
 Releases:
o Releases always result in a tag of the repository
o A release tag should use a three segment number as defined in Semantic
Versioning 2.0.0: MAJOR.MINOR.PATCH
o If a release branch is necessary release/1.1.x is created.

Forking Workflow

 Each contributor two Git repositories

o a private local one and a public server-side one.
 E.g. GitHub

Cloning
 Once the code is in a shared repository, a developer can clone the repository
 They can then create a branch out of the current master branch
 They then make changes on their branch

Forking
 Copy of the entire repository
 Commits do not go against the original repository
 Use-cases:
o Main repository might have issues, so clean new repository is forked for making
changes
o Building application for multiple clients and each client has client-specific
features

pg. 70
 SKILLCERTPRO

Pull requests
 A dev has cloned a repo > created a new branch > made changes > commited to branch
> pushed branch to remote repo > wants to merge the branch into the branch.
 Developer initiates a pull-requests
o If there are conflicts between main & uploaded branch
o They need to be resolved
o Approval needs to be put into place for the pull request

9.2. Pull request strategies

Pull request strategies

 In Azure repo's you can have limit merge types branch policy
o Standardizes a strategy for the whole team

Merge (no fast-forward)

 Standard strategy in Azure repos & most other Git providers
 It emulates running git merge pr from the master branch
 All the individual commits in the pull request branch are preserved as-is,
o and a new merge commit is created to unite the master branch and the
pull request branch.


 Trade-off:
o Pros
 It gives the most insight into how a branch evolves
 Illustrates exactly how a developer (or developers) worked on a pull
request
o Cons: since it preserves every commit is may be very verbose.

pg. 71
 SKILLCERTPRO

Squash commit
 Creates a single new commit
o leads to a just a simple, straight, linear history
 Emulates running git merge pr --squash from the master branch.
 The resulting commit is not a merge commit; those individual commits that made
up the pull request are discarded.


 💡 As individual commits are lost, it's best for teams that use "fix up" commits or
do not carefully craft individual commits for review before pushing them.

Rebase
 Takes each individual commit in the pull request and cherry-pick them onto the
master branch.
 Emulates running
i. git rebase master on the pull request branch
ii. git merge pr --ff-only on the master branch.
 History is straight and linear, like it is with the "squash" option but each individual
commit is retaine.
 💡 Useful for teams that practice careful commit hygiene, where each individual
commit stands on its own.

Semi-linear merge
 Also known as "rebase and merge"
o The commits in the pull request are rebased on top of the master branch
o Then rebased pull requests are merged into master branch
 Emulates running
i. git rebase master on the pull request branch
ii. git merge pr --no-ff on the master branch

pg. 72
 SKILLCERTPRO

 💡 Some see it as best of both worlds

o individual commits are retained, so that you can see how the work evolved
o but instead of just being rebased, a "merge bubble" is shown so that you
can immediately see the work in each individual pull request.

9.3. Azure Repos

Azure Repos
 Supports git, TFVC
o TFVC = Team Foundation Version Control
 branches = paths
 more granular permissions down to file level
 In Azure DevOps we have Azure Repos
o It comes with default repository for the project
 You can import repositories form GitHub, TFVC..
 You can create PRs in either direction: from fork to upstream, or upstream to fork.
o Most common = From fork to upstream
o The destination repository's permissions, policies, builds, and work items will
apply to the PR.

Service connections
 Helps you to connect to external and remote services to execute tasks in a job.
o e.g. connect to Microsoft Azure subscription, to services you install on remote
computers.
 Created at project scope
 Permissions
o User: Creator, Reader, User, Administrator roles
o Pipeline: Which pipelines can access
o Project permissions: which other projects can access the project (only its project
scope by default)

Permissions
 Azure Repos supports Git and Team Foundation
o In Team Foundation Version Control, you can set permissions at the file level.

pg. 73
 SKILLCERTPRO

o Git is more popular and have better compatibility with IDE's

 You can grant permissions for Azure repositories at both the repository an the branch
level.

Groups
 Reader: Clone, fetch, explore contents of a repository. Can also create, comment on,
vote and contribute to pull requests.
 Contributor & Build Admins: Contribute to a repository, create branches, create tags,
manage notes.
 Project Admins: Create, delete and rename repositories.
 You can modify permissions in each group.

Branch policies

 Require minimum number of reviewers for pull requests

 Check for linked work items: Enforce traceability to check for linked work items on PR's
 Check for comment resolution: Check that all comments have been resolved on PR's
 Enforce a merge strategy
o No fast-forward merge: merges the commit history of the source branch when
the pull request closes and creates a merge commit in the target branch.
o Squash merge: Complete all pull requests with a squash merge
 Build validation through build policies
o It's a pre-merging policy, only if build success then PR will be able to merged
o You link to a build pipeline that'll be triggered
 Require approval from additional services using PR Status API
 Automatically include code reviewers for specific directories and files in your repo
 Bypass policies: You can assign following to users / groups when
o completing pull requests
o or pushing

Authenticating with Git

 You can authenticate using:
o SSH
o Git Credential Managers
 💡 Recommended
 Supports MFA
o Personal Access Token (PAT)

pg. 74
 SKILLCERTPRO

 Clone using git clone

https://anything:{yourPAT}@dev.azure.com/yourOrgName/yourProjectN
ame/_git/yourRepoName
 You can also save your PAT in Git Credential Managers

10. Containers

Containers
 Has everything packaged in it to allow software to run (e.g. which OS, packages)
 Lightweight
 Code, runtime, system tools, and libraries
 Architecture: Infrastructure -> Host Operating System -> Container Service
(Docker) -> Container Apps

Docker
 Tool to create container based applications.
 Includes code + any other dependencies to run your code.
 Makes easier to run your container on different computing environments
 Architecture
o Daemon => Rest API => CLI
o Docker daemon
 Only runs on Linux because it depends on a number of Linux kernel
features
 There's ways to run it in Windows / MacOS
o The Docker daemon itself exposes a REST API
o Command line tool that lets you talk to the Docker daemon
 Dockerfile is a text file that defines the environment
o Each line in file creates a layer
o FROM ubuntu
 The first statement in the Dockerfile. It refers to the parent image
that this new image will be based upon
 An image that doesn't have a parent is called a base image and FROM
scratch can be used instead.
 Commands:

pg. 75
 SKILLCERTPRO

o Build: docker build -t container-name . in folder where Dockerfile exists

o Run: docker run -d -p 8080:80 --name app-name container-name
o List containers: docker ps
 ❗ You cannot build Windows containers in Linux
o But you can build Linux containers in Windows

Multi-stage builds

 Problem
o It was popular to maintain two Dockerfiles one for development one for
production (slimmed)
 E.g. production merged different bash commands with && to avoid
creating additional layers.
 Because each RUN instruction creates a new layer in the
container image
 E.g. with using \ to wrap lines:
 RUN powershell.exe -Command \
 $ErrorActionPreference = 'Stop'; \
 Invoke-WebRequest
https://www.python.org/ftp/python/3.5.1/python-3.5.1.exe -
OutFile c:\python-3.5.1.exe ; \
 Start-Process c:\python-3.5.1.exe -ArgumentList '/quiet
InstallAllUsers=1 PrependPath=1' -Wait ; \
Remove-Item c:\python-3.5.1.exe -Force

 However this increases the build speed as each layer is

cached separately when they are separate.

o Another And then a build.sh was used to copy contents from one docker
image to another.
 Solution

o Single Dockerfile resulting in tiny production image, no scripts using multi-

stage builds

o FROM mcr.microsoft.com/dotnet/core/sdk:2.2 AS build-env # from docker

hub, good to name with AS for flexible re-ordering
o WORKDIR /app # create directory
o
o # Copy csproj and restore as distinct layers
o COPY *.csproj ./
o RUN dotnet restore
o
o # Copy everything else and build
o COPY . ./
o RUN dotnet publish -c Release -o out

pg. 76
 SKILLCERTPRO

o
o # Build runtime image
o FROM mcr.mcrosoft.com/dotnet/core/aspnet:2.2 # for running application
o WORKDIR /app
o COPY --from=build-env /app/out .
ENTRYPOINT [ "dotnet", "serverappname.dll" ]

o A multi-stage data file consisting of two stages.

a. Build application
b. Run the application
 Read more at docker.com

o Visual Studio Docker support

 Right click on the project → Add → Docker support

 This will add Dockerfile as multiple stage build

Docker compose

 A tool to define and run multi-container docker applications.

 Configure all containers in a single YAML file, then spin up all of them with a
single command.
o E.g. can run following with single command: • DB • Cache service • Web
service

Kubernetes
 Kubernetes helps to:
o Orchestrate containers
o Deploy containers
o Maintain and monitor containers
o Scale applications
 Architecture:
o Master: Deploy the Docker and Kubernetes tools
o Nodes: Hosts the actual containers.

Azure Service Fabric

pg. 77
 SKILLCERTPRO

 Solution making it simple to package, deploy, and manage scalable and reliable
containers and microservices.
 Assists with the developing and managing cloud native applications.
 Intended for container-based applications running on enterprise-class, cloud-
scale environments.
 To be able to deploy using Azure Pipelines:
o Add new Service Fabric Connection
 Requires a cluster endpoint
 Authentication options:
 Azure Active Directory
 Add Server certificate thumbprint of the server cert.
used to create the cluster.
 Also the cluster credentials
in Username and Password fields.
 Certificate Based
 Requires:
 Server certificate thumbprint of the server
cert. used to create the cluster.
 Client certificate
 or cluster/server certificate
 Password for the certificate

10.1. Azure Container Registry

Azure Container Registry

 Managed private Docker registry service
 Store your private Docker container images
 Security (❗ below features are only available in Premium SKU)
o Consent trust for image tag signing
o Firewalls and virtual networks to restrict access to the registry.
 If you want your CI/CD tool to be able to work with ACR =>
o Create a service principal e.g. for Jenkins.
 az ad sp create-for-rbac --skip-assignment
o Assign it to the ACR:

pg. 78
 SKILLCERTPRO

 az role assignment create --assignee 626dd8ea-042d-4043-a8df-

4ef56273670f --role Contributor --scope $ACR_ID
o You can now use appId (username) and password of your service principal
to push & update images.
 To upload image
o Install azure CLI: apt-get azure-cli
o Sign in: az login
o Create a container registry: az acr create --resource-group registry-rg --
name registry --sku Standard --location eastus
o Build image & tag & push it to Azure
 Using CLI:
 az acr build --registry registry --image namewithtag:v1 .
 Using Azure Pipelines:
 You can create a new pipeline & configure it to be docker,
it'll then create azure-pipelines.yml file.

 Or you can create the yaml file yourself using Docker task:

 - stage: Build
 displayName: Build and push stage
 jobs:
 - job: Build
 displayName: Build job
 pool:
 vmImage: $(vmImageName)
 steps:
 - task: Docker@2
 displayName: Build and push an image to container
registry
 inputs:
 command: buildAndPush
 repository: $(imageRepository)
 dockerfile: $(dockerfilePath)
 containerRegistry:
$(dockerRegistryServiceConnection)
 tags: |
$(tag)

10.2. Azure Kubernetes Service

Azure Kubernetes Service

 Managed Kubernetes platform

pg. 79
 SKILLCERTPRO

 You can manually scale nodes using az aks scale --resource-group

myResourceGroup --name myAKSCluster --node-count 3
 To scale out run: kubectl scale --replicas=3 deployment/test-app

Creating a cluster
 Creates all infrastructure NSG, route table, VNET, VMs, NICs., Availability set,
storage account..
o See Microsoft walkthrough for more information
 Configurations
o Select a DNS name
o Choose node VM size
o Select amount of initial nodes
o Authorization
 Create / select a service principal
 Service principals are used to ensure that the cluster can
work with other Azure services
 RBAC: Use Azure AD to limit access to cluster resources based a
user's identity or group membership.
o Network: Use existing VNet or create new
o Monitoring: Collected from containers to Log Analytics Workspace
through Azure Monitor.

Using CLI
 Create a resource group az group create --name test-rg --location eastus
 Create cluster: az aks create --resource-group test-rg --name testcluster --
node-count 1 --enable-addons monitoring --generate-ssh-keys
 Install kubectl CLI: to manage the cluster: az aks install-cli --install-
location=./kubectl
 Connect: To configure kubectl to connect to your Kubernetes cluster, use az aks
get-credentials --resource-group test-rg --name testcluster
 To verify connection use kubectl get nodes

 To deploy apache server you need a deployment yaml:

 apiVersion: apps/v1
 kind: Deployment
 metadata:
 name: test-apache-app
 spec:

pg. 80
 SKILLCERTPRO

 replicas: 1
 selector:
 matchLabels:
 app: test-app
 template:
 metadata:
 labels:
 apsps: test-app
 spec:
 containers:
 - name: testappserver
 image: httpd
 ports:
 - containerPort: 80
 ---
 apiVersion: v1
 kind: Service # To expose the container to outside world
 metadata:
 name: test-app
 spec:
 type: LoadBalancer
 ports:
 - port: 80
 selector:
app: test-app

 Two ways to deploy the specs:

i. Declarative management: kubectl apply -f app.yaml

ii. Imperative management: kubectl create
 Run kubectl get all to see status of containers, services, replicas & deployment
o Can also run kubectl get deployments, kubectl get pods
o Pod = group of containers that are deployed together on the same host.
 If you deploy single containers, pod = container
 Get external IP: kubectl get service test-app --watch (--watch flag to wait until
it's exposed to internet)

Using ARM

 Create a key ssh-keygen -t rsa -b 2048

o Copy your public key
 Create a service principal for AKS so it can work with other Azure services
o az ad sp create-for-rbac --skip--asignment
 Copy appId and password
 Select AKS template, e.g. quickstart 101-aks

pg. 81
 SKILLCERTPRO

o Paste RSA public key

o Past appId and password as service principal client id and service principal
client secret
 See Microsoft walkthrough for more information

Interacting with Azure Services

 Needed to dynamically create and manage other Azure resources such as an
Azure load balancer or container registry (ACR).
 To interact with Azure APIs, an AKS cluster requires either:
o Azure Active Directory (AD) service principal
o managed identity
 az create creates a service principal automatically.
o Credentials will be saved in $HOME/.azure/aksServicePrincipal.json on
machine that run the command
 Or you can create it manually using az ad sp create-for-rbac --skip-assignment
--name myAKSClusterServicePrincipal
o You can then assign it to AKS when creating it:
o az aks create --resource-group myResourceGroup --name myAKSCluster --
service-principal <appId> --client-secret <password>
o Delegate access to other resources as Azure Container Registry,
Networking, Storage, Azure Container Instances.
 az role assignment create --assignee <appId> --scope
<resourceScope> --role Contributor

Running serverless containers in AKS

 Virtual Kubelet allows to add virtual serverless nodes (e.g. AWS Fargate, Azure
Container Instances) to a Kubernetes cluster.
o You can add e.g. Azure Container Instance using az aks install-
connector command
 Creates service principal
 ❗ Depends on helm (requires helm init)
 Will be deprecated shortly
 Ensure trust between service principle of AKS and Azure resources with az role
assignment create
o E.g. allow AKS to connect to Azure Container Registry
 Create a yaml file for deployment
 Run kubectl command to deploy the file.

pg. 82
 SKILLCERTPRO

Deploy helm to cluster

 Helm
o Open-source package manager for Kubernetes clusters (like APT or Yum for
Linux)
o Applications are deployed with the help of helm charts
 Helm charts = packages of preconfigured Kubernetes clusters
 It's used for a Tiller service
o Tiller is a server service that resides in Kubernetes
o It's used along with helm for deploying the application
 Using v3
i. helm repo add to add a repo
ii. helm repo update to update charts
iii. helm install to run charts
 📝Using v2
. kubectl apply
 Deploy a yaml file to create a service account using kubectl apply -
f helm-rbac.yaml:
i. apiVersion: v1
ii. kind: ServiceAccount
iii. metadata:
iv. name: tiller
v. namespace: kube-system
vi. ---
vii. apiVersion: rbac.authorization.k8s.io/v1
viii. kind: ClusterRoleBinding
ix. metadata:
x. name: tiller
xi. roleRef:
xii. apiGroup: rbac.authorization.k8s.io
xiii. kind: ClusterRole
xiv. name: cluster-admin
xv. subjects:
xvi. - kind: ServiceAccount
xvii. name: tiller
namespace: kube-system

xviii. helm init

 Initialize service account: helm init --service-account tiller --
node-selectors "beta.kubernetes.io/os=linux"
xix. helm install
 E.g. install an nginx container with helm:
 helm install stable/nginx-ingress --set
controller.nodeSelector."beta\.kubernetes\.io/os"=linux --set
defaultBackend.nodeSelector."beta\.kubernetes\.io/os"=linux"

pg. 83
 SKILLCERTPRO

Access and identity options

 Accounts in Kubernetes
o Service account: one of the primary user types in Kubernetes.
 credentials for service accounts are stored as Kubernetes secrets
 allows to be authorized by pods to communicate with API Server
o Normal user accounts: human administrators/developers.
 No identity solution in Kubernetes, in AKS integrated identity
solution is Azure Active Directory
o Credentials for service accounts are stored as Kubernetes secrets
o To obtain a kubectl configuration context, a user can run the az aks get-
credentials command
 Azure RBAC can be used to access Azure resources.
 Kubernetes RBAC
o A role is defined as set of granted permissions.
 Two types:
 Role: within a namespace
 ClusterRoles: within
 an entire set of cluster
 or cluster resources outside a given namespace
o Once roles are defined, you assign those using:
 RoleBinding to assign roles for a given namespace
 ClusterRoleBindings to assign across entire cluster or to cluster
resources outside a given namespace.
 You use bindings to grant Azure AD users to perform actions within
the cluster.

11. Mobile DevOps (Visual Studio App Center)

Visual Studio App Center

 appcenter.ms
o Build, test, distribute, diagnostics, analytics, auth, data push mobile
applications.

pg. 84
 SKILLCERTPRO

 Automate & manage the lifecycle of iOS; Android, Windows and macOS
applications.
o Connect to your repositories & automate your builds
o Test builds on real devices in the cloud
o Distribute apps to beta testers
o Monitor real-world usage with crash and analytics data
o Enable get feedback from users on the new features
 📝 It's used to:
o Manage mobile target device sets and distribution groups
o Managed target UI test device sets
o Provision tester devices for deployment
o Create public and private distribution group

Distribution groups
 Controls access to releases
 Set of users e.g. QA Team, Canary users etc. releases, such as Staging.
 Release the application to users via distribution groups
 Types 📝
o Private: Invited by e-mail to test application
o Public: Unauthenticated users, download application with a link.
o Shared: Shared across multiple applications in a single organization.
 Created at organization level, not application level.
 Device registration - example for iOS application
o Devices have to be specified in the provisioning profile for the application
o App Center will help register the tester device IDs into the Apple
Development account
o You will need the .p12 certificate which was used to sign the application at
build time.

Releasing an application
 Android
o Ensure you have updated the manifest and have a correctly configured
Gradle build.

pg. 85
 SKILLCERTPRO

o In Android Studio, choose Build > Generate Signed Bundle / APK and
follow the steps in the wizard to build the app bundle or APK.
 iOS / macOS
o ❗ Register each testers devices on Apple Developer portal as test devices.
o In Xcode, go to Product > Archive to archive your app.
o Export the archive using the proper provisioning profile.
 Windows: .appx, .appxbundle, .appxupload, .msi, .msix, .msixbundle, .msixupload,
or .zip
 Other OS: .zip

12. Infrastructure as code

Infrastructure as code
 DevOps + Agile => Needs faster techniques to provision infrastructure
o E.g. create test environments & terminate quickly
 Good for disaster recovery.
 There are many tools to automate the underlying infrastructure

12.1. ARM templates

Azure Resource Manager Templates

 Azure Resource Manager (ARM) templates are infrastructure as code (JSON) in Azure.
 You can define dependencies to define their deploy order.

{
"$schema": "https://schema.management.azure.com/schemas/2019-04-
01/deploymentTemplate.json#",
"contentVersion": "", // (Required) Your own version to ensure right template is
deployed
"apiProfile": "", // API versions for resource types.
"parameters": { }, // prompted when deployment is executed.
"variables": { },
"functions": [ ],
"resources": [ ], // (Required) Resource types that are deployed or updated
"outputs": { } // Values that you want to return after deployment.
}

pg. 86
 SKILLCERTPRO

 You can use visual editor in Azure Portal

o Template Deployment => Custom Deployment => Edit Template
 You can deploy your template on the portal
o or use Azure CLI: az group deployment create --name testdeployment --
resource-group test-rg --template-file test-template.json

Deployment using Azure Pipelines

 Two ways to deploy templates with Azure Pipelines:
i. Add task that runs an Azure PowerShell script
 AzurePowerShell task to execute Deploy-AzureResourceGroup.ps1
 Consistency throughout the development life cycle because you use the
same script that is included in the Visual Studio project
ii. Add tasks to copy and deploy tasks
 AzureFileCopy task to copy templates to blob storage
 AzureResourceGroupDeployment to create or update resource group using
URL of the template

Nested templates
 💡 Recommended way to deploy multiple ARM templates.
 Reasons
o Template can grow long and unmanageable
o Difficult to deploy the template for customized environments
 E.g. an environment needs everything template except one component.
o Automation becomes difficult to accomplish
 Solution
o Modularize your templates for the minimum e.g. per resource
o You then create a main template and add you can
 nest other templates into the main template by defining them there
 or add URL's to child templates as linked template
 Pros: Child templates are reusable!
 Cons: All templates must exist in the remote.
 To create multiple instances of a resource with a nested template
o you can add a copy element to copy an existing child template.

Managing secrets
 E.g. username & password for a VM.

pg. 87
 SKILLCERTPRO

 Use Azure Key Vault service

 Flow
i. Create a vault
 In Access Policy, enable "Azure Resource Manager for template
deployment"
ii. Create a secret e.g. examplesecret
iii. Reference it in ARM template

Referencing in ARM template

With static resource ID

 You can reference it in ARM template such as:

"adminPassword": {
"reference": {
"keyVault": {
"id": "/subscriptions/<subscription-
id>/resourceGroups/examplegroup/providers/Microsoft.KeyVault/vaults/<vault-name>"
},
"secretName": "examplesecret"
}
}

 Pros
o Existing ARM templates are not changed.
o Only parameter files are changed to include Azure Key Vault references.
 Cons
o Within the parameter file, Azure Key Vault resource ID must be hard-coded.
o The hard-coded resource ID includes the subscription ID, which might be
considered as a sensitive data.

With dynamic resource ID

 Use this when you do not want to hardcode the Key Vault ID with e.g. subscription id.
 Dynamically construction ID does not work in ARM template, neither in parameters file.
 Nested templates are the key to using this dynamic id.

Using linked template

 Notice templateLink
o It links to another ARM file that will use secret value as string
o Template must exist in the remote location

pg. 88
 SKILLCERTPRO

{
"apiVersion": "2015-01-01",
"name": "nestedTemplate",
"type": "Microsoft.Resources/deployments",
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "[concat(parameters('templateBaseUri'), 'my-nested-
template.json')]",
"contentVersion": "1.0.0.0"
},
"parameters": {
"resourcegroup": {
"value": "[parameters('resourcegroup')]"
},
"vaultName": {
"value": "[parameters('vaultName')]"
},
"secretToPass": { // here vault ID & secret name is dynamically
generated
"reference": {
"keyVault": {
"id": "[resourceId(subscription().subscriptionId,
parameters('resourcegroup'), 'Microsoft.KeyVault/vaults', parameters('vaultName'))]"
},
"secretName": "examplesecret"
}
}
}
}

 Pros
o There is no hard-coded value required.
 Cons
o Additional linked templates should be written increasing maintenance effort.

Using nested template

 Another option is to have a nested template

{
"$schema": "https://schema.management.azure.com/schemas/2015-01-
01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"vaultName": {
"type": "string",
"metadata": {
"description": "The name of the keyvault that contains the secret."
}
},
"secretName": {

pg. 89
 SKILLCERTPRO

"type": "string",
"metadata": {
"description": "The name of the secret."
}
},
"vaultResourceGroupName": {
"type": "string",
"metadata": {
"description": "The name of the resource group that contains the keyvault."
}
},
"vaultSubscription": {
"type": "string",
"defaultValue": "[subscription().subscriptionId]",
"metadata": {
"description": "The name of the subscription that contains the keyvault."
}
}
},
"resources": [
{
"type": "Microsoft.Resources/deployments",
"apiVersion": "2018-05-01",
"name": "dynamicSecret",
"properties": {
"mode": "Incremental",
"expressionEvaluationOptions": {
"scope": "inner"
},
"template": { // nested child template
"$schema": "https://schema.management.azure.com/schemas/2015-01-
01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"adminPassword": { // gets from the parent
"type": "securestring"
}
},
// ... stripped rest of the template
},
"parameters": {
"adminPassword": { // here vault ID & secret name is dynamically generated
"reference": {
"keyVault": {
"id": "[resourceId(parameters('vaultSubscription'),
parameters('vaultResourceGroupName'), 'Microsoft.KeyVault/vaults',
parameters('vaultName'))]"
},
"secretName": "[parameters('secretName')]"
}
}
}
}
}
],

pg. 90
 SKILLCERTPRO

"outputs": {
}
}

13. Configuration as Code (PowerShell DSC & Azure Automation & Custom Script)

Configuration as Code
PowerShell Desired State Configuration
 Management platform in PowerShell
 Manage IT infrastructure with configuration as code
 PowerShell DSC consists of Configurations.
o Declarative PowerShell scripts
o Used to define the configuration of the underlying resources they are
attached to.
 Resources
o Contain the code that keep the target of a configuration in a specified
state.
 E.g. if VM changes, ensure IIS (server) will be in its configured state

o Configuration would look like:

o configuration IISInstall {
o node "localhost" { # Ensure applied to this node
o WindowsFeature IIS {
o Ensure = "Present"
o Name = "Web-Server"
o }
o }
}

o In Extensions section of an VM, you need to install "PowerShell Desired

State Configuration"

 Zip the configuration upload it as Configuration Modules/Scripts

 Type name of the configuration e.g. config.ps1\IISInstall

Azure Automation

pg. 91
 SKILLCERTPRO

 Cloud-based automation and configuration service.

 Allows you to do:
o Configuration Management:
 Collect inventory for your resources
 Track changes
 📝 Implement Desired State Configuration
o Update Management
 Manage OS updates in Azure, on-prem or in other cloud providers
 Access compliance and for scheduled update installations
 Azure Automation account
o Similar to Azure Storage accounts in that they serve as a container to store
automation artifacts
 Runbook
o Set of tasks that perform some automated process in Azure Automation
o Webhooks allows to start a particular runbook in Azure Automation
through a single HTTPS request.
 Automation Shared resources
o Resources that allow to be used in, or associated with a runbook e.g
Schedule
 Process Automation
o Orchestrate processes using PowerShell and Python.
 Source control integration allows you to
o push code from Azure Automation to source control
o pull your runbooks from source control to Azure Automation.

Implement Desired State Configuration

 Official walkthrough

pg. 92
 SKILLCERTPRO


 📝 Steps
i. Create an automation account (New-AzureRmAutomationAccount)
 Enable option to create an Azure Run As account
 It's an AD service principal Azure will create & assign
Contributor RBAC role to it.
 Allows you to authenticate with Azure when managing
resources
 Automate the use of global runbooks configured in
Azure alerts
ii. Upload the Desired State configuration (Import-
AzureRmAutomationDscConfiguration)
 Create a DSC configuration script & upload it
iii. Compile the Configuration (Start-AzureRmAutomationDscCompilationJob )
 A DSC configuration must be compiled into a node configuration
before it can be assigned to a node.
 � Behind the scenes it compiles the PowerShell script to
a .moc (Managed Object Format) file that has C++-like syntax.
iv. Register a VM as node to the automation account (Register-
AzureRmAutomationDscNode)
 Add VM in DSC → Nodes.
 Configuration settings:
 The Local Configuration Manager (LCM) is the engine of
Desired State Configuration (DSC)
 Configuring LCM:

pg. 93
 SKILLCERTPRO

 Refresh frequency: E.g. every 30 min automation

account will check configuration on VM
 Configuration mode:
 ApplyAndMonitor
 Applies any new configurations.
 If target node drifts from the desired
state later on, DSC reports it.
 ApplyOnly
 Applies initial configurations.
 DSC does not check for drift from a
previously configured state
 ApplyAndAutoCorrect
 If target node drifts from the desired
state later on, DSC reports & re-applies
current configuration.
v. Assign a node configuration to a managed node (Set-
AzureRmAutomationDscNode -NodeId $node.Id)
vi. (Optionally) Check the compliance status of a managed node (Get-
AzureRmAutomationDscNodeReport)
 Node status can be "Compliant", "Failed", or "Not Compliant"

Custom Script Extension

 Allows for the download & execution of scripts on Azure VMs

 Deploy any post deployment configuration & install any software

 E.g. script to install IIS on a Windows machine:

o In portal go to VM 0 => Extensions => Custom Script Extension

o Add using:

o Import-Module ServerManager
Install-WindowsFeature Web-Server -IncludeAllSubFeature

 Example for deploying extension using ARM template:

 {
 "name": "MyCustomScriptExtension",
 "type": "extensions",
 // ...

pg. 94
 SKILLCERTPRO

 "properties": {
 "publisher": "Microsoft.Compute",
 "type": "CustomScriptExtension",
 // ...
 "settings": {
 "fileUris": [
 "[concat('https://', variables('storageName'),
 '.blob.core.windows.net/customscripts/start.ps1')]"
 ],
 "commandToExecute": "powershell.exe -ExecutionPolicy Unrestricted -
File start.ps1"
 }
}

pg. 95