You are on page 1of 36

Reference architectures for

enterprise-grade Internal Developer


Platforms built on Azure Cloud,
GitHub Actions, Backstage,
Humanitec, Flux CD, Terraform,

and more

e
Backstag

API

log / Portal
e Cata
Servicg Developer
Catalo

IaC
specs
n cont
rol Workload orm
Versio Terraf
Score
e
ne
Github ce Pla
Resour

Compute

line bernetes

CD Pipe Azure Ku
Service

Registry
Data
line
CI Pipe
FluxCD Azure SQ
L

Platfor tor
ra
Azure
r
Orchest ng
Containe Networki
Registry
Github

Actions Azure DN
S

Services
s
rvice Bu
Azure Se

nitor
Azure Mo

y
abilit
Observ

y Vault
Azure Ke
ment
Manage
entity
s & Id
Secret
02 Introduction

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Introduction
Organizations need to be agile and innovative to stay competitive in today's software
development era, which has led to changes in how applications are built, deployed, and managed.
This necessitates the transformation of static CI/CD setups into modern Internal Developer
Platforms (IDPs) that provide developers with the tools needed to innovate and move quickly.
43% of DevOps professionals recognize this and, as a result, have built an IDP to improve
developer experience (DevEx) and enable developer self-service.

As an industry, we need to move beyond buzzwords and provide real-life examples of modern
IDPs. A blog post by McKinsey (soon to be released) makes a major contribution to this. 

While every platform looks different, certain common patterns emerge. To help simplify things,
McKinsey consolidated the platform designs of hundreds of setups into standard patterns based
on real-world experiences, which have been proven to work effectively. By adopting these
patterns, organizations can create IDPs that keep them ahead of the competition and deliver
innovative applications faster than ever before.

This whitepaper is inspired by McKinsey’s blog post and provides an overview of one reference
architecture for a dynamic IDP using Azure Cloud, GitHub Actions, Backstage, Humanitec, Flux
CD, Terraform, and several other technologies. Please note this architectural design tries to use
the most common combinations, but it doesn’t restrict the use of these technologies; every one
of them is interchangeable with others.

Internal Developer Platform on Azure Cloud

IDE Service Catalog / API Catalog

Developer Portal

Developer
 Version Control


Control Workload Specs IaC
Visual

Plane Studio Code


Github Score Terraform

CI Pipeline Registry CD Pipeline Resource Plane


Humanitec

Deploy
Compute

Azure Kubernetes

Integration
Azure
Service
& Delivery Plane Github
Container
Platform
FluxCD
Actions Registry Orchestrator
Data

Azure SQL

Networking

Observability Azure DNS


Monitoring & Logging Plane Azure Monitor

Services

Azure Service Bus


Secrets & Identity Management
Security Plane Azure Key Vault
03 Introduction

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

There will be a replicable open-source version of this architecture soon, so you can set this up
yourself. In the meantime, please get in touch with a Humanitec solution architect
(info@humanitec.com), who will be happy to provide a test setup.

Please note that definitions and wording in this whitepaper references


internaldeveloperplatform.org, which usually follows official Gartner definitions, and that this
reference architecture is for enterprises with more than 50 full-time application developers
focused on cloud-native technologies.
04 Table of contents

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Table of contents

06
Problems this IDP design aims to solve

07
Design principles

08
Architectural components

Developer Control Plane level 08

Integration and Delivery Plane level 09

Resource Plane level 10

Security Plane 10

Logging and Monitoring Plane 10

11
The end-to-end architecture result

Integrating the Developer Control Plane 12

Integrating the Integration and Delivery Plane 12

Integrating the Resource Plane 12

Integrating the Monitoring and Logging Plane 12

Integrating the Security Plane 13

How platform engineers or Ops teams operate, build, and


maintain a platform 14

Wire up existing resources and define when they should be matched 16

Define “how to create new resources” at a specific request 17

Set baseline app configs and policies 17

Configure a base set of automation for the platform 18

Roles and Role-Based Access Control (RBAC) 18

19
How developers use such a platform
05 Table of contents

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Zooming in on a “golden path” to understand the


interplay of all components 22

Golden path 1: Simple deployment to dev 22

Golden path 2: Create a new environment 24

Golden path 3: Create a new resource based on the developers’ request 25

Golden path 4: A platform engineer updates the dev Postgres resource to

the latest Postgres version 26

Benefits of this architecture 28

Key impact for platform or Ops teams 28

Key impact for developers 28

Key impact for executives 29

Conclusion 30

Ready to build your dynamic Internal Developer Platform? Next steps 31

Appendix 32

Capabilities of this architecture 32

Dynamic Configuration Management 32

Environment management 32

Deployment Management 33

Infrastructure Orchestration/Resource Management 33

Observability 33

Administration 33

Cost management 34

Integration 34
06 Problems this IDP design aims to solve

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Problems this IDP design aims to solve


Modern Internal Developer Platforms (IDPs) can help solve a number of problems faced by
organizations today, including:

Low levels of productivity and innovation

When developers spend their most productive hours dealing with tedious
infrastructure and configuration management tooling, creativity and velocity suffer.
A burned-out engineer does not write code as quickly and efficiently; inspiration
often dies when too many steps are needed to test and deploy an idea. On average,
developers with inefficient platform setups have longer waiting times as they are
stuck in a loop while other teams manually resolve things for them. Mediocre setups
also make onboarding difficult.

Missing self-service increases Ops pressure and slows down devs

Ops teams and developers often find themselves waiting for each other to complete
tasks which can result in delays, frustration, and decreased productivity. Modern
IDPs can help solve this problem by providing code such as UI, CLI or API-based
interfaces that allow developers to quickly and easily provision resources without
having to wait for Ops to do it for them.

High cognitive load on developers caused by disparate interfaces,



tools and formats

Developers find themselves spending a lot of time switching between interfaces,


tools, and formats. This can cause frustration and additional cognitive load, which
can lead to decreased productivity and a lack of innovation. Dynamic IDPs can help
solve this problem by providing developers with unified interfaces (golden paths) and
formats, freeing them up to focus on coding. Contrary to common belief, adding a UI
on top to create a new resource is not the solution; it’s primarily about building code-
based paths across the application's entire life cycle. Taking a dynamic approach to
config management (creating configurations at deployment time based on rules set
by the platform team) allows users to specify their requirements in a simplified,
abstract format (a workload specification, UI, API or CLI). The platform then
dynamically creates the necessary configs in the target format.
07 Design principles

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Lack of standardization leads to an explosion of complexity

In current static CI/CD setups, there are often dozens of ways to materially reach the
same goal, such as spinning up a new Postgres Database or deploying to production,
describing the state of a cluster or other conventions. These scripts often vary only
slightly and sometimes just by environment. Their sheer number and unstructured
nature make them hard to maintain. Good platform design reduces the number of
variances through Dynamic Configuration Management (DCM) by up to 95%.

Design principles
According to McKinsey, there are eight proven design principles:

01 Focus on the user. The most important customers of a developer platform


are developers. Developers need to be heavily involved in the design,
prioritization of features, and testing to ensure the platform is fit for purpose
and fully self-service. Focusing on creating a usable design that addresses the
real needs of developers will make all the difference.
02 Run your platform team like a start-up. Establish a small central team that
owns the platform and is responsible for marketing it (and has the resources to
do so), ensuring it is easily consumable and fulfills developers’ needs.
03 Build golden paths vs cages. Developers should be free to choose their
abstraction level. While your IDP should provide a set of golden paths for
developers to follow, it should never force their use.
04 Drive standardization y design. Enabling self-service means platform
b

engineers must define how to vend resources and configuration. This ensures
every resource is built securely, compliant, and well-architected.
0 5 Implement Dynamic on guration anagement. Dynamic
C fi M

Configuration Management significantly reduces config comple ity and x

enforces standardization by continuously generating app and infrastructure


configs with every single deployment. This allows you to enforce policies and
standards with every git-push.
08 Architectural components

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

06 Let developers decide on their platform interface. The developer


platform should never break your developer's workflow or force them to use a
specific interface. To support this, a code-based workflow by default works
best, with the option to use a UI, CLI, or API.

07 Keep code as the single source of truth. This ensures everyone is working
from the same version, reducing the risk of errors.

08 Assume a brownfield scenario. Use tools that have already been


productized and adopted by the organization (such as backlog management,
CI/CD toolchain, and container platform) and let the existing teams pursue
integration through plug-ins into the platform. Where applicable, organizations
should use open-source tooling and a cloud-native approach.

Architectural components
According to McKinsey's blog post, “plane levels” are different areas of the platform
architecture that cluster certain functionalities. Let’s zoom in on the plane levels we have to
take care of and see what technologies fulfil each function in all of those levels.

Developer Control Plane level


Under the term “Developer Control Plane”, we’re clustering the primary “interfaces” developers
can choose to use when interacting with the platform or applying any change. As discussed in
the “design principles” section, we leave interface change to the developer on a workload-by-
workload basis. We also, by default, recommend not to break the current workflow of the
developer. Which is why we default to code wherever possible.

Components used in this architecture


Visual Studio Code - a source code editor
GitHub - a Version Control System (VCS) to store and structure our code
Score  - a workload specification that allows developers to describe the workload and
their dependencies as code in an environment-agnostic way. 95% of developer
interactions with the platform will usually happen via Score
09 Architectural components

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Terraform - an Infrastructure as Code (IaC) tool written in HCL to describe the state of
the infrastructure resources in a declarative way
Backstage - a developer portal/service catalog to provide an interface to consolidate
documentation, structure service templates, and catalog existing services.
Service Catalog / API Catalog

Developer Portal

Developer
 Version Control


Control Workload Specs IaC
Plane
Github Score Terraform

Integration and Delivery Plane level


The integration and delivery layer contains the tools that build, store, configure and deploy any
request coming from the Developer Control Plane. It includes continuous integration (CI),
registries, platform orchestration, and continuous delivery (CD). 
Components used in this architecture
Github Actions - a CI pipeline that builds and tests the code of our workload.
Azure Container Registry - an image registry that helps us store and manage

container images.
Humanitec - a Platform Orchestrator that dynamically generates app and

infrastructure configs. 
FluxCD - in this example, we’re using Azure Pipelines, a continuous delivery

tool for Kubernetes

CI Pipeline Registry CD Pipeline

Integration
Github
Azure

& Delivery Plane Actions Container


Platform
FluxCD
Registry Orchestrator
10 Architectural components

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Resource Plane level


The Resource Plane level includes all resource components necessary to run the app. Since
this reference architecture is primarily Azure-based, we mostly feature Azure products here.
The resources are configured as code using Terraform (see dev control planes).

Components used in this architecture


Azure Kubernetes Services - a fully managed Kubernetes service on Azure.
SQL Server - a managed relational database service.
Cloud DNS - a highly available and scalable DNS service. 

Security Plane 
On the Security Plane, we’re managing secrets and identity to protect sensitive data. We’re
storing, managing, and securely retrieving API keys and passwords.

Components used in this architecture


Azure Key Vault - a secrets manager. 

Logging and Monitoring Plane 


A plane that provides real-time metrics and logs for application and infrastructure that
developers can utilize to observe, monitor, and make data-driven decisions.

Components used in this architecture


Azure Key Vault - a monitoring and observability service. 

Observability
Monitoring & Logging Plane Azure Monitor

Secrets & Identity Management


Security Plane Azure Key Vault
11 The end-to-end architecture result

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

The end-to-end architecture result


Now that we’ve defined the different plane layers, let’s put all of it together into an end-to-end
architecture. The interplay of some layers is intuitive, others need slightly more deep-dive.
Ultimately, our assembled architecture will look as follows:

Internal Developer Platform on Azure Cloud

IDE Service Catalog / API Catalog

Developer Portal

Developer
 Version Control


Control Workload Specs IaC
Visual

Plane Studio Code


Github Score Terraform

CI Pipeline Registry CD Pipeline Resource Plane


Humanitec

Deploy
Compute

Azure Kubernetes

Integration
Azure
Service
& Delivery Plane Github
Container
Platform
FluxCD
Actions Registry Orchestrator
Data

Azure SQL

Networking

Observability Azure DNS


Monitoring & Logging Plane Azure Monitor

Services

Azure Service Bus


Secrets & Identity Management
Security Plane Azure Key Vault

Integrating the Developer Control Plane


The IDE simply allows developers to work on the files stored in the VCS. On this system, they
store the workload source code, the docker files, CI pipeline YAML, workload specifications,
workload profiles and IaC files. These files get pushed to or pulled from the Integration and
Delivery Plane. 

The developer portal component adds an aggregator of information pulling data from the
Observability Plane, Resource Plane, Platform Orchestrator, CI pipeline and VCS. If used for
service creation and as a templating engine, the portal might call the templating API of

the VCS.
12 The end-to-end architecture result

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Integrating the Integration and Delivery Plane


Most of the “configurations” for this plane will be stored as code on the Developer Control
Plane level. This holds true for the CI pipelines and the matching and automation criteria of the
Platform Orchestrator using the respective Terraform provider. We’ll explain the end-to-end
flow of the platform in the next section, but for now, let’s focus on how the elements of this
plane connect to each other and other planes. 

The Continuous Integration (CI) pipeline receives a notification to build and test from the git-
push of the terminal, pushing the latest changes to the Version Control System (VCS) and
indicating the branch. The CI pipeline will store its image in the registry and—at the last step of
the build pipeline—inform the Platform Orchestrator that a new image is available. It will also
send the metadata and eventual “orders” of new resources from the workload specification
(we’ll get to this in more detail later.) The Orchestrator will hand over the deployment-ready
app and infrastructure configs to the Continuous Delivery (CD) part. Note that the Orchestrator
is also taking over the CD functionality in our example, which doesn’t necessarily have to be the
case. The CD pipeline will go ahead and update the Resource Plane. 

All parts of this plane can output workflow performance data and other metrics to the
Developer Control Plane, such as CI build time, DORA metrics etc. The Orchestrator can
register new services and their dependent resources to the portal layer.

Integrating the Resource Plane


The Resource Plane is mainly on the receiving side of the other planes. The resources are
declaratively represented by Infrastructure as Code (IaC) in the VCS, and the Platform
Orchestrator dynamically generates the final app and infrastructure configs. The Resource
Plane supplies the Integration, the Monitoring and Logging Plane with data and pulls secrets
from the Security Plane.

Integrating the Monitoring and Logging Plane


This plane primarily pulls information from the Resource Plane by running agents and side-
cars next to the running containers and pulling intel from the respective cloud APIs. The
interesting part of this connection is how this gets effectively surfaced to the developer. The
answer is either a) through the interface of the plane or b) through notifications and logs that
get pushed to parts of the Developer Control Plane layer. This would either be surfaced in the
UI of the Orchestrator, which developers often use on a day-to-day basis. Or into the developer
portal/service catalog to enrich existing data.
13 The end-to-end architecture result

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

An important integration point for the Monitoring and Logging Plane also happens on the
Integration and Delivery Plane, where the Platform Orchestrator ensures the necessary
sidecars and agents are launched and running next to the cluster.

Integrating the Security Plane


This plane “receives” credentials, passwords etc, from the Resource Plane. It then receives 

the reference to where to inject those secrets at run-time from the configs created by the
Platform Orchestrator. It injects those secrets into the container and deals with the rotation

of the secrets.
14 How platform engineers or Ops teams operate, build, and maintain a platform

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

How platform engineers or Ops teams operate,



build, and maintain a platform
As the name suggests, platform engineers build the platform. This means they are ultimately
responsible for the design of the individual layers and the interplay of those planes. But
depending on the organisation's team structure, the responsibility for the different planes
varies. Here’s a list of the most common patterns we’ve observed in the enterprise; note that
often the distinction is pure naming convention:

Plane 60% of orgs 20% of orgs 20% of orgs

Developer
 Platform Developer

Control Plane engineering Ops experience

Integration and Platform


Delivery Plane engineering Ops CI/CD

Resource Plane Platform SRE/Infrastructure Ops


engineering

Security Plane Security Platform Ops


engineering

Platform Ops Developers


Observability Plane engineering

In the vast majority of cases teams already have an existing setup when building their IDP, so
it’s often about remodeling to ensure their setup matches the design principles. Here’s how:

01 Design the individual planes. Start with the Resource Plane because it
dictates other design decisions on other layers. We usually propose the
following order:
a. Resource Plane (you probably already have resources, in this case, decide
which ones are supported by your platform as a default).
b . Integration and Delivery Plane: Pipeline design, configs of the Orchestrator etc.
c . Security Plane.
d . M onitoring and Logging Plane.
e . Developer Control Plane: this heavily depends on the design choices of the other
planes and should always come last after thorough testing by developers.
15 How platform engineers or Ops teams operate, build, and maintain a platform

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

02 Wire the individual components of the planes to each other as well as one
plane to the other and test the raw end-to-end flows.

03 Set baseline configs for app and infrastructure configs (more details below).

04 Set deployment and environment progression automation.

05 Design and implement Roles Based Access Control (RBAC).

06 Onboard your first application.

We’ve covered the planes and their design; let’s next zoom in on the baseline configs

and automations.

Before the platform is ready, the platform engineering team still needs to set a number of
defaults and baseline configs. The entire idea of the presented reference architecture is to
enable developer self-service, lower cognitive load, drive standardization, and reduce ticket
ops. This requires the use of Dynamic Configuration Management (DCM), which in turn requires
a Platform Orchestrator that functions as a rules engine. It matches the request from the
developers with the config defaults provided by the platform team.

This means the next “job to be done” for the platform engineering team is to set those app and
infrastructure config defaults.

16 How platform engineers or Ops teams operate, build, and maintain a platform

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Wire up existing resources and define when they should



be matched
In a Dynamic Internal Developer Platform, the Platform Orchestrator resolves the abstract
request from a workload specification (like Score) “I need a Postgres”. They match the context
and wire up to the correct resources (in this case, a database) by pulling the credentials and
injecting them through secrets into the container at deployment time. To do this, the platform
team needs to “explain” to the Platform Orchestrator when to use what resource by wiring up
this resource. This can be done through the Platform Orchestrator UI (the Resource
Management) or the equivalent Terraform provider:

resource "humanitec_resource_definition" "postgres" {

id = "db-dev"

name = "db-dev"

type = "postgres"

driver_type = "humanitec/postgres-cloudsql-static"

driver_inputs = {

values = {

"instance" = "test:test:test"

"name" = "db-dev"

"host" = "127.0.0.1"

"port" = "5432"

secrets = {

"username" = "test"

"password" = "test"

criteria = [

app_id = "test-app"

}
17 How platform engineers or Ops teams operate, build, and maintain a platform

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Define “how to create new resources” at a specific request


A big part of the job of a platform engineering team (in tandem with the infrastructure and
security group) is to build reusable defaults for resources. For example, if a developer requests
an S3 bucket for an ephemeral environment and that developer is in the US west development
team, what’s the default config you want to use to spin up that S3? This “infrastructure
resource catalog” should be code based. Alternatively, but not necessarily, one could consider
the use of an infrastructure control plane such as Hashicorp Cloud. When you define “how” the
resource gets created, you need to tell the Platform Orchestrator “when” to use it. Essentially
the Orchestrator would act like a “rules engine” to define what to serve at what request. You
could again use the Terraform provider or the Platform Orchestrator UI.

Set baseline app configs and policies


To dynamically create app configs at deployment time, the Platform Orchestrator needs to
know what baseline configs (called ‘workload profiles’) to use. They contain information such
as CPU minimum allocation, labels, and annotations for Datadog or the fact that Hashicorp
Vault should start up as a sidecar. You can have one or many workload profiles, but in order to
drive standardization you want to keep the number as low as possible.
18 How platform engineers or Ops teams operate, build, and maintain a platform

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Configure a base set of automation for the platform

Now all our planes are in place, and we can technically already get from code to running

application. Fundamentally we would now be able to use our platform. However, as a developer,

you may not want to have to trigger the different elements of the process. For instance, if we

do a git-push, we expect the platform to deploy all the way through automatically. In more

advanced setups, we might even want to enable automated progression from one environment

to the other, theoretically to production or at least pre-production. So as a next step, we’ll need

to configure those automations on an environment level in the Platform Orchestrator, whose

rules engine can execute those requests.

Roles and Role-Based Access Control (RBAC)

The RBAC of the Version Control System (VCS) and the Platform Orchestrator allows your

organization to define the right levels and control for your developers across roles. How these

RBACs are set up depends on the security posture of your organization. Are developers

allowed to change infrastructure resources and or templates? Who’s allowed to deploy to

production? Who’s permitted to change the baseline templates? We usually think about a


RBAC setup before onboarding applications at scale.


19 How developers use such a platform

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

How developers use such a platform

Holding true to our design principle of leaving interface choice and opting for code first,


the answer is: It depends! The proposed architecture leaves that choice to a workload-by-

workload basis. 

The primary interaction method (by far the most used) is the code-based one. Developers

prefer to stay in their usual workflow, in the version control system (VCS), and within their

integrated development environment in order to “indicate” what their workloads require, spin

up new services, add resources etc. This is where a workload specification like Score comes

into play. It provides a code-based “specification” to describe how the workload relates to

other workloads and their dependent resources. Adding a Resource Definition to the Score file

will tell the Orchestrator to automatically create a new resource or wire an existing one. We’ll

explain how this works in the next section.

This is a simple example of how a Score file looks:

score.yaml

apiVersion: score.dev/v1b1

metadata:
name: python-service

containers:
python-service:
image: python
variables:
CONNECTION_STRING: postgresql://${resources.db.user}:${resources.db.password}@${resources.db.host}:${resources

resources:

db:

type: postgres

storage:

type: s3

dns:

type: dns

We can see that the developer requires a database type Postgres, a storage of type S3 and a

DNS of type DNS. For the vast majority of use cases, this code-based format should be entirely

sufficient and is the preference for most developers. 


20 How developers use such a platform

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

For specific situations (like running diffs, rolling back, spinning up new environments) they
might prefer to use the Orchestrator UI, CLI or API.

Portals and Service Catalogs are primarily used for consolidation and or product managers/
engineering managers, as well as onboarding and orienting new developers. 
21 How developers use such a platform

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Here’s a list of some activities a developer performs using an Internal Developer Platform and
what interface they usually choose: 

Activity type Predominant interface choice

Deploy Terminal/IDE

Change configuration Code: Workload specification (Score)

Add/remove resource Code: Workload specification (Score)

Roll back/Diff Orchestrator UI

Configure resource in detail Code: IaC

Spin up a new environment Orchestrator UI

See logs/error messages Orchestrator UI

Search service catalog Portal/Service catalog

Inner source use case Portal/Service catalog

Service create case Portal/Service catalog or templating in VCS


22 Zooming in on a “golden path” to understand the interplay of all components

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Zooming in on a “golden path” to understand


the interplay of all components

Now we understand the integration points for the different planes, let’s look at how a

developer deployment request would flow through the platform step-by-step. We call these

standard flows “golden paths”. Below are four (of many) standard flow examples the IDP

provides golden paths for.

Golden path 1: Simple deployment to dev

Let’s start with a very simple example. A developer has changed something on their workload

and now deploys to a dev environment. As we discussed earlier, the primary interaction

method for devs would be code. They would git-push their change, and the CI pipeline would

pick it up and run. It would then push the built image to the image registry. At this point we

have the service built, but we don’t have the configs yet (remember, we’re opting for Dynamic

Configuration Management). 

The workload source code contains the workload specification (Score), which in this case

might look like this:

score.yaml

apiVersion: score.dev/v1b1

metadata:
name: python-service

containers:
python-service:
image: python
variables:
CONNECTION_STRING: postgresql://${resources.db.user}:${resources.db.password}@${resources.db.host}:${resources

resources:

db:

type: postgres

storage:

type: s3

dns:

type: dns

We can see that the developer requires a database type Postgres, a storage of type S 3 and a
DNS of type DNS.
23 Zooming in on a “golden path” to understand the interplay of all components

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

So after the CI has been built, the Platform Orchestrator realizes the context and looks up
what resources are matched against this context (in our case, it's maybe the CI tag
"environment = development"). It checks whether the resources are already created (which is
likely in this case because it’s just a deployment to an existing dev environment) and reaches
out to the Azure API to retrieve the resource credentials. It then creates the application configs
in the form of manifests because our target compute in this architecture is Azure Kubernetes
Service (AKS). Once this is done, the Orchestrator deploys the configs and injects the secrets
at runtime into the container (utilizing Vault). 

score.yaml

apiVersion: score.dev/v1b1

metadata:
name: python-service

containers: CONTEXT:
python-service:
image: python
env = development
Dev request variables:
CONNECTION_STRING: postgresql://${resources.db.user}:${resources.db.password}@${resources.db.host}:${resources.db.port}/${resources.db.name}
Platform 

Orchestrator
resources:
db:
type: postgres
storage:
type: s3
dns:
type: dns

AKS cluster configured

Read workload specification


SQL credentials injected
Platform Match resource definitions
response Create app configs, configure resources
S3 credentials injected
Deploy
Azure DNS configured
24 Zooming in on a “golden path” to understand the interplay of all components

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Golden path 2: Create a new environment


Let’s repeat the same procedure but spin up a new environment. The developer experience

is as simple as it gets; they push the same repository and the same workload specification
(because it works across all environments). By setting the context to “ephemeral” (through the
tag), the Platform Orchestrator will again now interpret the workload specification. It will
realize that Postgres doesn’t exist yet and that it should create one using a specific Driver.

The Platform Orchestrator will then create the configs, inject the dependencies, and serve.

score.yaml

apiVersion: score.dev/v1b1
metadata:
name: python-service
containers: CONTEXT:
python-service:
image: python
Dev request variables: env = development
CONNECTION_STRING: postgresql://${resources.db.user}:${resources.db.password}@${resources.db.host}:${resources.db.port}/${resources.db.name}
Platform 

Orchestrator
resources:
db:
type: postgres
storage:
type: s3
dns:
type: dns

Create new namespace

Read workload specification


Platform Match resource definitions Create SQL

response Create app configs, configure resources Create S3


Deploy
Create DNS entry
25 Zooming in on a “golden path” to understand the interplay of all components

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Golden path 3: Create a new resource based on the


developers’ request
An interesting case is when the developer sends a request the system doesn’t know yet. This is
where many approaches fail, except when applying Dynamic Configuration Management
(DCM). Because everything is repository-based, the approach allows developers to extend the
set of available resources or customize them to their liking. Let’s play through the scenario
where a developer needs an ArangoDB, but this isn’t known to the setup so far. By adding a
Resource Definition to the general baselines of the organization, the developer can easily
extend the setup in a way that can be reused by the next developer. 

I need ArangoDB for my workload


Dev request I add a resource definition Platform 

but there is no default Orchestrator

Read workload specification


ArangoDB is avaialable
Platform Match resource definitions for reuseby the next
response Create app configs, configure resources user. Standardization

by design!
Deploy
26 Zooming in on a “golden path” to understand the interplay of all components

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Golden path 4: A platform engineer updates the dev Postgres


resource to the latest Postgres version
This is a great example of how platform engineers utilize the platform to maintain a high
degree of standardization. Let’s explore how one would go about updating the dev Postgres
resource to the latest Postgres version. Given our architectural choice, let's assume all
resources use Terraform for Infrastructure as Code. To make the example more compelling,
let's also assume that development environments all have their own instance of a Postgres.
This approach is so expensive that in reality, we would likely share instances across several dev
environments. So how would we go about updating all our Postgres resources across the
workloads that use them? 

01 The “thing” we need to update is the Resource Definition of dev Postgres


resources. Assuming we have our Postgres configured in Terraform, we’ll simply
update the Terraform module. Otherwise, we would actually adopt the driver.

02 Should we have to apply changes to the in and outputs, we would need



to update the Resource Definition itself in the Terraform provider of the
Orchestrator.

resource "humanitec_resource_definition" "postgres" {

id = "db-dev"

name = "db-dev"

type = "postgres"

driver_type = "humanitec/postgres-cloudsql-static"

driver_inputs = {

values = {

"instance" = "test:test:test"

"name" = "db-dev"

"host" = "127.0.0.1"

"port" = "5432"

secrets = {

"username" = "test"

"password" = "test"

criteria = [

app_id = "test-app"

}
27 Zooming in on a “golden path” to understand the interplay of all components

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

03 We then need to find out what workloads are currently depending on our
Resource Definition of “dev Postgres”. The answer can be found in our “rules
engine”, the Platform Orchestrator. Simply because this is where the “decision
is made” regarding what resources to use to wire the workload up, and in what
context. We can do this by pinging the Orchestrator API or looking at the user
interface in the Resource Definition section: “Usage”.

04 Once identified, we can auto-enforce a deployment across all workloads that


depend on the resource type “Postgres dev”, and the new version is rolled out
across all workloads and applications.
28 Benefits of this Architecture

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Benefits of this architecture


The presented architecture has a significant impact on reducing lead time and thus
decreasing time to market. On average, fully rolled-out platforms show a decrease of 30% in
lead time. The impact differs from team to team:

Key impact for platform or Ops teams


The impact of Internal Developer Platforms (IDPs) on Platform and Ops Teams can be
significant in a number of ways. First, by enabling developer self-service, manual work can be
reduced, which in turn allows teams to focus more on improvements. Second, Dynamic
Configuration Management (DCM) can drive standardization by design, which creates a more
efficient and effective process. Third, standardization enables the rapid onboarding of new
developers, which can be a significant advantage for companies looking to grow their teams
quickly. Fourth, golden paths can streamline your setup, reduce maintenance and allow team
members to work more effectively. All of these factors combine to improve the development
process, which can lead to better results and happier team members overall.

Key impact for developers


The impact of IDPs on application Developers can be significant in many ways. The first
advantage is that self-service reduces dependencies and waiting times. Developers can
easily access the resources they need and work on their projects without having to wait for the
IT department to assist them. This can lead to faster development and a higher deployment
frequency, which is a crucial aspect of modern software development.

Another benefit of IDPs is streamlined config management, which reduces cognitive load.
Developers can focus on writing code instead of worrying about infrastructure, which can be

a complex and time-consuming task. With an IDP, developers can simply select the resources
they need and configure them as required, freeing up more time for coding.

IDPs also offer new superpowers that can boost productivity. For example, developers

can use Score as a workload spec, which allows them to specify the desired performance
characteristics of their application. They can also spin up PR environments, which can be

used to test and debug code changes before merging them into the main codebase.
Furthermore, the diff functionality for debugging allows developers to quickly identify and

fix issues, while secure infrastructure self-service ensures that the entire development
process remains secure.
29 Benefits of this Architecture

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

In conclusion, IDPs can have a significant impact on the productivity and efficiency of
application developers. By reducing dependencies and waiting times, streamlining config
management, and offering new superpowers, developers can focus on delivering

high-quality applications.

Key impact for executives


While the most obvious impact IDPs have on executives and the wider business is the
significant reduction in time to market (TTM), the benefits are far more multifaceted and far-
reaching. One of the biggest advantages is the high degree of automation they provide, which
in turn reduces failure rates and increases security. This is achieved through the use of
advanced tools and technologies that are constantly improving and evolving.

In addition to the automation benefits, IDPs also enable developer self-service, which reduces
waiting times and skyrockets productivity. This allows for faster innovation cycles and
enables organizations to stay ahead of the competition. Moreover, dynamic IDPs require fewer
full-time Ops employees per every application developer, which helps organizations
streamline overall operations and reduce costs.

Another important benefit of IDPs is cost control. By reducing cloud bills and optimizing
resource allocation, organizations can invest saved money in other business areas. This is
especially important in today's highly competitive landscape, where every dollar counts.

Overall, IDPs have the potential to revolutionize the way organizations develop and deploy
software. By leveraging automation, developer self-service, and other advanced technologies,
IDPs can help organizations to stay ahead of the curve and achieve their goals more quickly
and efficiently than ever before.
30 Conclusion

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Conclusion

In conclusion, adopting modern Internal Developer Platforms (IDPs) using on Azure Cloud, SQL

Server, Backstage, Humanitec, GitHub, Azure Pipelines, Terraform, and several other

technologies can help organizations improve their developer experience, increase productivity

and innovation, and reduce cognitive load for developers. By implementing this architecture,

organizations can also deliver applications faster and more efficiently. However, it is important

to remember that the implementation of an IDP varies widely by organization, and our

reference architecture is just a starting point for building an effective dynamic platform.
31 Ready to build your dynamic Internal Developer Platform? Next steps

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Ready to build your dynamic Internal Developer



Platform? Next steps
If you're interested in using this reference architecture, please reach out to Humanitec's
Platform Architects. We have codified versions of this architecture that we are happy to

share upon request.
32 Appendix

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Appendix
Capabilities of this architecture
Dynamic Configuration Management
Use the environment-agnostic workload specification to describe infrastructure
dependencies once and for all environments

Workloads can depend on types of resources with resource-specific parameters


injected at deployment time

Multiple workloads can depend on the same resource (e.g. a shared database or

DNS name)

Analyze material differences between deployments and environments using diffs

Create workload profiles that standardize workload deployments across environments

Export generated Kubernetes manifests without access to cluster

Manage environment specific values and secrets separately from workload


configuration

Store secrets in your own secret management service

Developers define workload configuration using Score

A full history of all workload configuration, environment specific values and secrets

can be retrieved

Environment Management
New environments can be created on demand by cloning existing environments

Deleting an environment will automatically delete all associated resources provisioned


by Humanitec

Environments can be paused/unpaused, resulting in workloads being scaled down and


up for cost savings

Status of workloads in the environment can be determined without user access



to the cluster

A subset of workloads can be promoted between environments


33 Appendix

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Deployment Management
Deployments in an environment can be rolled back to a previous deployment
Webhooks for key events (creation/deletion of environments, deployments etc.)

are available
Deployments can be triggered based on criteria from source control, such as tag
format or branch name
Pipelines including additional pre and post-deployment steps can be defined (in beta)
Promotion of workloads between environments based on criteria such as tests passing
or manual approval can be automated

Infrastructure Orchestration/Resource Management


The life cycle of a resource (such as a DNS name or database) can be fully managed
Existing resources not managed can be used and connected via static definitions
Resource provisioning can depend on other resources, e.g. a database can depend on a
database instance which depends on the VPC that the environment is created in
A suite of drivers can be used to create, update and destroy infrastructure and

other resources
Driver for Terraform and other IaC formats to integrate brownfield setups
A particular Resource Definition is chosen by matching to the context being

deployed into, allowing the reuse of definitions and generalization across

ephemeral environments

Observability
Can be used to standardize the integration of APM products
Container logs are surfaced without the user needing access to the cluster
Monitor environment health via workload, pod and container statuses. Errors are
displayed in real-time
Services can be catalogs, and metadata can be aggregated
34 Appendix

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

Administration

Integration with SSO (SAML 1.1 & 2.0)

Manage access with RBAC

Run a self-hosted instance (still managed by Humanitec, but running in your network

and on your infrastructure)

Cost management

Resource limits

Pausing of environments

Pausing of environments on schedule

Deleting of environments and resources

Integration

All functionality is available via the API

Long-lived API tokens can be issued to support integration with 3rd party systems

Manifests can be exported to a git repository instead of deploying directly to a cluster

to enable GitOps workflows

Arbitrary Manifests can be configured to be included as part of a deployment

supporting the use of custom CRDs and specialist Kubernetes features

Connection to infrastructure can be controlled and secured via:

IP Whitelisting

Bastion hosts

VPN (IPSec)

Agent/Operator running in network

Custom Resource Drivers can be written to support infrastructure yet to be supported

and that do not have Terraform providers.


35 Imprint

Reference architecture for an enterprise-grade IDP built with Humanitec on Azure

© Copyright 2023 Humanitec GmbH

Humanitec GmbH
Wöhlertstraße 12-13, 10115 Berlin, Germany

Phone: +49 30 6293-8516

Humanitec Inc
228 East 45th Street, Suite 9E,

New York, NY 10017

Humanitec Ltd
3rd Floor, 1 Ashley Road

Altrincham, Cheshire WA14 2DT

United Kingdom

E-mail: info@humanitec.com

Website: https://www.humanitec.com

CEO: Kaspar von GrünbergRegistered at Amtsgericht Charlottenburg, Berlin: HRB 196818 B

VAT-ID according to §27a UStG: DE318212407

Responsible for the content of humanitec.com ref. § 55 II RStV: Kaspar von Grünberg
IDE IaC
specs
control Workload m
Version Terrafor
Score
loper
Visual
 de
Studio Co Github e Plane
rol
Resourc
e
Compute
line ernetes

CD Pipe Azure Kub


Service

Registry
Data
line
CI Pipe
FluxCD Azure SQL

Platforram
tor
Azure
r
Orchest ng
Containe Networki
Github
s Registry
Action Azure DN
S
tion

Integraery Plane
& Deliv
Services

rvice Bus
Azure Se

nitor
Azure Mo

ility
Observab

ing &

Monitor Plane Vault


Logging Azure Key
gement
ty Mana
& Identi
Secrets

y Plane
Securit

Reference architectures for


enterprise-grade Internal Developer
Platforms built on Azure Cloud,
GitHub Actions, Backstage,
Humanitec, Flux CD, Terraform,

and more

humanitec.com

You might also like