iMpLemENtiNg eVoLuTiONaRy

ARcHitECtuREs
Day 2:
The Evolution of Penultimate Widgets!
@neal4d
nealford.com @rebeccaparsons @patkua

2020-06-03 2021-08-17
 Agenda Part 1
Incremental Change
Definition

Guided Change via Fitness Functions

Structuring architecture for evolution

 Evolutionary Architecture

An evolutionary architecture supports

guided,
incremental change
across multiple dimensions.
 guided

architectural fitness function:

An architectural fitness function provides an

objective integrity assessment of some
architectural characteristic(s).
 Fitness Functions
fitness functions . . .
monitors

chaos
unit engineering
tests metrics
 Exercise:
What type of fitness function?

evolutionary architecture governance

process

enterprise architecture
Agenda Part 2

The Evolution of
Penul&ma e
 Agenda Part 2
fitness functions architecture migration

The Evolution of
Penul&ma e
experimentation

automating
governance hypothesis/data driven
development
Penul&ma e
Penul&ma e
 Penul&ma e

— many outdated systems

 Penul&ma e

— many outdated systems

— outsourced opera&ons
 Penul&ma e

— many outdated systems

— outsourced operations
— impending competitors
 Penul&ma e

— many outdated systems

— outsourced opera&ons
— impending compe&tors
— second to last in market!
Penul&ma e
 Penul&ma e
1.

3.
evolu&onary architecture
2.
automated
machine provisioning
Penul&ma e
Penul&ma e
Penul&ma e
Penul&ma e
Penul&ma e
Penul&ma e
Penul&ma e
demonstration
defeats
discussion
Penul&ma e

microservices for
new development
migrate critical
existing systems
 component coupling
the extent to which components know
about each other

component component

component
 component coupling

tight loose
coupling coupling

pathological external control data

coupling coupling coupling coupling
 component coupling
afferent coupling
the degree to which other components are
dependent on the target component

component b

component c component a

component d
 component coupling
efferent coupling
the degree to which the target component
is dependent on other components

component b

component a component c

component d
Abstractness

where:
ma => abstract elements
mc => concrete elements
Instability

where:
ce => efferent coupling
ca => afferent coupling
Distance from the Main Sequence
Distance from the Main Sequence
Distance from the Main Sequence
distance from the main sequence
Dependency Metrics
Connascence
h"ps://connascence.io/name.html
Origin
Two components are connascent if a
change in one would require the
other to be modified in order to
maintain the overall correctness of
the system.

sta$c dynamic
sta$c
Connascence of Name
mul$ple components must agree on the name of an en$ty
sta$c
Connascence of Type
mul$ple components must agree on the type of an en$ty
sta$c
Connascence of Type
mul$ple components must agree on the type of an en$ty

selec$ve typing via

clojure-spec
h7ps://clojure.org/guides/spec
sta$c
Connascence of Meaning
mul$ple components must agree on the meaning of par$cular values
sta$c
Connascence of Meaning
mul$ple components must agree on the meaning of par$cular values

constants!

…move from connascence of meaning to

connascence of name
sta$c
Connascence of Position
mul$ple en$$es must agree on the order of values

placeOrder(name, address, amount, discount)

“Neal” “Seat 10C”
“Seat 10C” “Neal”
sta$c
Connascence of Algorithm
mul$ple components must agree on a par$cular algorithm
Origin
Two components are connascent if a
change in one would require the
other to be modified in order to
maintain the overall correctness of
the system.

sta$c dynamic
dynamic

Connascence of Execution
the order of execu,on of mul,ple components is important
dynamic

Connascence of Timing
the %ming of the execu%on of mul%ple components is important

race

condition
dynamic

Connascence of Value
occurs when several values relate on one another
and must change together
dynamic

Connascence of Identity
mul$ple components must reference the same en$ty
dynamic

Connascence of Identity
mul$ple components must reference the same en$ty

ACID
transaction
Connascence
Two components are connascent if a
change in one would require the
other to be modified in order to
maintain the overall correctness of
the system.

sta$c dynamic
Connascence Properties

sta$c dynamic

Strength Locality Degree

 Connascence Properties

]
Strength name
type
static
meaning
algorithm

]
position
refactor
execution order
this way
timing dynamic
value
identity
Connascence Properties

sta$c dynamic

Strength Locality Degree

 Connascence Properties

]
Locality

static

]
Proximal code (in the same module, class, or
function) should have more & higher forms
of connascence than less proximal code (in
separate modules, or even codebases).
dynamic
 Coupling

seman&c syntac&c
 Connascence Properties
Locality dynamic

locality matters!

dynamic
Coupling

syntac'c
Connascence Properties

sta$c dynamic

Strength Locality Degree

Degree Connascence Properties
Connascence Properties

sta$c dynamic

Strength Locality Degree

 Component Cohesion
the degree and manner to which the opera0ons of a component are
related to one another
Chidamber & Kemerer Metrics
LCOM (Lack of Cohesion in Methods)
Chidamber & Kemerer Metrics
LCOM (Lack of Cohesion in Methods)

The LCOM is a count of the number of method pairs whose similarity is 0

(i.e. σ() is a null set) minus the count of method pairs whose similarity is
not zero.
Chidamber & Kemerer Metrics
LCOM (Lack of Cohesion in Methods)

The LCOM is a count of the number of method pairs whose similarity is 0

(i.e. σ() is a null set) minus the count of method pairs whose similarity is
not zero.

The sum of sets of methods not shared via sharing fields.

Chidamber & Kemerer Metrics
LCOM (Lack of Cohesion in Methods)
 Chidamber & Kemerer Metrics
LCOM (Lack of Cohesion in Methods)
ive
es
h
co
 Chidamber & Kemerer Metrics
LCOM (Lack of Cohesion in Methods)

ive
ive

h es
es

co
n-
h

no
co
 Chidamber & Kemerer Metrics
LCOM (Lack of Cohesion in Methods)

n
sio
ive
ive

he
es

co
h
es

co

d
ixe
n-
h

no
co

m
Unifying Coupling/Connascence
 When Migra+ng Architecture…

1. improve modularity
 When Migra+ng Architecture…

2. create service-based
1. improve modularity architecture
 When Migra+ng Architecture…

2. create service-based
1. improve modularity architecture 3. isolate quanta
 When Migra+ng Architecture…

2. create service-based
1. improve modularity architecture 3. isolate quanta

4. evolve via microservices

Migration Targets
Microservices?
Service-based Architecture
Service-based Architecture

service database
granularity scope
Electronics Recycling Application
public requests receiving department recycling and accounting

kiosk

receiving accounting
quote

assessment recycling
item status

reporting
Electronics Recycling Application
kiosk public ui receiving ui recycling ui
Electronics Recycling Application
kiosk public ui receiving ui recycling ui
Service-based Architecture
adding microservices
client requests client requests client requests

user interface layer

Service-based Architecture
adding microservices
client requests client requests client requests

user interface layer

Shared-database Integration
Refactoring Databases
Move Column Refactoring
Transition Period
 Decouple DB Updates:
the Expand/contract Pattern
Expand/Contract Pattern
Change ‘name’ to
‘firstname’ &
’lastname’
Change ‘name’ to
‘firstname’ &
’lastname’
Change ‘name’ to
‘firstname’ &
’lastname’
#1: integration points, legacy data
#2: integration points, legacy data
#3: integration points, legacy data
Penul&ma e

microservices for
new development
migrate critical
existing systems
 e
Deployment Pipeline
Penul&ma
cycle time
 Penul&ma e
fitness function:
CycleTimeGuard
Penul&mateWidgets has entered an extremely compe&&ve
market with a startup running popular flash promo&ons. The
marke&ng department commi?ed to keeping up with
Penul&mateWidgets own aggressive marke&ng campaign, which
requires development teams to be able to deliver changes to the
promo&on pages within an hour. Thus, they want a guarantee
that changes can always take place within an hour.
 Penul&ma e
fitness function:
CycleTimeGuard
• u&lize the good modularity of the monolith to build a
deployment pipeline that automa&cally incorporates CMS
changes
• con&nuously deploy new content for marke&ng pages as
long as they pass smoke test
• measure cycle &me from deployment pipeline for
deployments for marke&ng sites
• save cycle &me results to aggregate a weekly chart
Incidentally Coupled Teams
user interface

server-side

DBA
 Conway’s Law
organizations which design
systems ... are constrained to
produce designs which are copies
of the communication structures of
these organizations

Melvin Conway, 1968

en.wikipedia.org/wiki/Conway%27s_law
Low Efferent
Coupling
between
Teams
Inverse Conway Maneuver
 Experimentation is the only
way to discover if something
works within a particular
organization.
Retrofi'ng Governance
Penul&ma e
 Penul&ma e
fitness function:
Configure Some of the Things All of the Time

Penul&mateWidgets has been burned in the past by having hard-coded

constants in several places in their code base, governing both business
cases (thresholds, maximums, etc.) and opera&onal concerns. Architects
need a way to prevent developers from accidentally hard-coding cri&cal
values.
 Penul&ma e
fitness function:
Configure Some of the Things All of the Time
•Upon check-in:
•scan the list of dynamic values in the code base and see if any of them are
assigned to (or their seCer is called) or a new instance is instan&ated via a
constructor that provides values
•ensure that all assignments for dynamic values are assigned via the API that
handles configura&on, not with hard-coded values
• scan the defini&ons of infrastructure to look for dynamic values and ensure
they are assigned properly before building the environment
•scan the configura&on file and ensure that all defined dynamic values are
present and have values
What to Port?
 Experimentation!
•add logging to the exis/ng applica/on
to aggregate what menu items users
choose and the user group
•publish the log to a known URL at
Penul/mateWidgets

•aggregate the logs to get usage and user

coverage
 Penul&ma e
fitness function:
Replace the Cru1y Core
Penul&mateWidgets built a network scanning tool many, many
years ago to look for suspicious ac&vity at the network packet
level. To achieve the proper performance, the code was wriAen
and has been maintained in increasingly complex C code.
Penul&mateWidgets needs to replace the code but is nervous
about introducing bugs and/or harming performance or
throughput.
 Penul&ma e
fitness function:
Replace the Cru1y Core
• Built fitness func&ons around the exis&ng C code to ensure
performance and throughput values.
• Build a spike solu&on in both Java and Go (and perhaps others)
and run them against the fitness func&ons to see if they can
meet the threshold
• Use Scien&st (hGps://github.com/github/scien&st) port
Scien&st4J (hGps://github.com/rawls238/Scien&st4J) to verify
the new code against the old code in produc&on
While We’re at it…
Penul&ma e
fitness function:
A11y All the Things

Penul&mateWidgets has decided that all new

development of user interfaces must support
accessibility, both web and desktop.
 Penul&ma e
fitness function:
A11y All the Things
• add a fitness func&on to the deployment pipeline for web
projects that runs pa11y (h?p://pa11y.org/) to ensure
compliance
• for Java, use PMD to check code within the user interface and
look for control instan&a&on
• check to make sure every control has the AccessibleName
property set
Mi#ga#ng the Unknown
reuse
 John D. Cook
h"p://www.johndcook.com/blog/2011/02/03/lego-blocks-and-organ-transplants/
 custome custome
auto and r r
life
homeowners
insurance
insurance
division
division

custome custome
r r
commercial disability
customer
insurance insurance
service
division division

custome custome
r r
casualty travel
insurance insurance
division division
 auto and
life
homeowners
insurance
insurance
division
division

commercial disability
customer
insurance insurance
service
division division

casualty travel
insurance insurance
division division
eusable code is, the less u
hexagonal pattern
operational reuse
the sidecar pattern
the sidecar pattern
service mesh
 two reuse solutions

service mesh
SOA
 Data Mesh
h"ps://mar*nfowler.com/ar*cles/data-monolith-to-mesh.html
data warehouse
ETL

Data
Warehouse
Data Lake

Data
Data
LakeLake |
| Hub
Hub
data lake on the cloud
Data Mesh
DATA MESH
ARCHITECTURE
Federated Ecosystem Governance
(enable interoperability)

Decentralized
Data Product Domain Data Products as Architecture
ownership Quanta

Data Infra as a Platform Abstrac(ng tech complexity

Data infra engineers (storage, pipeline, discoverability, Access control, etc.)
 Data Mesh
h"ps://mar*nfowler.com/ar*cles/data-monolith-to-mesh.html
Penul&ma e
fitness function:
Elas&city of Microservices

Penul&mateWidgets u&lizes a microservices

architecture and want to ensure that as new
requests appear that the architecture
elas&cally scales.
 Penul&ma e
fitness function:
Elas&city of Microservices

icroservices architecture and want to ensure that as new requests appear that th
 Penul&ma e
fitness function:
Elas&city of Microservices

icroservices architecture and want to ensure that as new requests appear that th
 Penul&ma e
fitness function:
Elas&city of Microservices

icroservices architecture and want to ensure that as new requests appear that th
 Penul&ma e
fitness function:
Elas&city of Microservices

icroservices architecture and want to ensure that as new requests appear that th
 Penul&ma e
fitness function:
Elas&city of Microservices

icroservices architecture and want to ensure that as new requests appear that th
 Penul&ma e
fitness function:
Elas&city of Microservices

icroservices architecture and want to ensure that as new requests appear that th
 Penul&ma e
fitness function:
Elas&city of Microservices
• build a Hello, World service to allow instan&a&on of the
service template
• deploy the service and direct enough requests to it to
trigger elas&city in the naming service
• verify that the number of instances rose properly
 Penul&ma e
fitness function:
Determinis&c Distributability

• Require that developers use configura&on files for all database

configura&on (using tools like ArchUnit)
• During tes&ng, randomly switch database configura&ons between
tests across several working instances
 Penul&ma e
fitness function:
Missing Messages

Penul&mate Widgets has had a problem with lost / mis-routed &ckets,

and the architects have narrowed it down to lost messages in a message
queue because of a hard to reproduce error.
Penul&ma e
fitness function:
Missing Messages
Penul&ma e
fitness function:
Missing Messages

Any of the above!

 Fitness Func*ons
fitness func1ons . . .
monitors

chaos
unit engineering
tests metrics
 Penul&ma e
fitness function:
Maintaining Good Throughput

heir website has become and want to make sure they maintain good throughput
 Penul&ma e
fitness function:
Degrade Gracefully

• build a run-&me monitor that reports on number of instances, performance

• if performance or scalability violates the established thresholds, raise an ala
Concrete Architecture Defini/ons

maintainable?
Concrete Architecture Defini/ons

maintainable?

Degrade gracefully versus Maintain Throughput ?

Breaking Bad Habits
 Penul&ma e
fitness function:
No More View Models

Penul&mateWidgets has a group of developers who cut their teeth on

mainframe programming, where all work is done via a terminal. They
have (mostly) switched to modern web development, but architects
find that they oDen create models and workflows directly in web pages
rather than support a cleaner MVC separa&on, and they would like to
discourage this behavior.
 Penul&ma e
fitness function:
No More View Models

• use PMD (h7ps://pmd.github.io/) or Language server protocol

(h7ps://langserver.org/) to build a tool that scans which classes
developers instan&ate in view code
• build a list of restricted model and controller packages
• if any view code instan&ates a class from the restricted list, fail the
build
 Penul&ma e
fitness function:
Deploy All The Things

Penul&mateWidgets has go7en &red of each applica&on requiring an

elaborate environment setup to deploy and run, and has decided that all
new applica&ons must be deployed within Docker containers. The
Penul&mateWidgets architects want to ensure that all applica&ons follow
the same standard deployment guidelines.
 Penul&ma e
fitness function:
Deploy All The Things
Build a monitor that runs within the developers' staging area that
inspects each running service to ensure that it is running within a container
and supports common opera&onal concerns such as logging, monitoring,
and service discovery.
 Penul&ma e
fitness function:
Deploy All The Things
Build a monitor that runs within the developers' staging area that
inspects each running service to ensure that it is running within a container
and supports common opera&onal concerns such as logging, monitoring,
and service discovery.

“legacy in a Box”
Make New Habits
 Penul&ma e
fitness function:
Upgrade All The Things

Penul&mateWidgets has a bad habit of wai&ng too long to update

core libraries and frameworks they depend upon for
development. Wai&ng past several versions makes the eventual
upgrade quite painful, and they have resolved to perform
upgrades in a more &mely manner.
 Penul&ma e
fitness function:
Upgrade All The Things
• Use Dependabot to flag new major versions of frameworks
and libraries.
• When a library updates, make an entry in a build database
and start a 3-month clock.
• If the library or framework in ques&on hasn't been updated
in 3 months, fail the build.
 Penul&ma e
fitness function:
Depend on Dependencies

Penul&mateWidgets has recently been burned by a Maven update that

broke several applica&ons yet wasn't detected un&l produc&on.
Architects and business people want to make sure this doesn't happen
again.
 Penul&ma e
fitness function:
Depend on Dependencies

Use a deployment pipeline to trigger new builds and test suite

execu&on any &me any ar&fact of the project changes, including
external dependencies. If the automa&c build breaks, a developer must
locate the problem and restore automa&cally.
OR
Use Dependabot (hDps://dependabot.com/)
Penul&ma e
Penul&ma e
Penul&ma e

incremental
Penul&ma e

Evolving Rou+ng

<> <home> <catalog><item>

Penul&ma e

Evolving Rou+ng
Penul&ma e

Evolving Rou+ng
Penul&ma e

Evolving Rou+ng
Penul&ma e

Evolving Rou+ng
Penul&ma e Experimen)ng
with New Markets
 Penul&ma e
fitness function:
MaintainTheMock
Penul&mateWidgets has started experimen&ng with
flashing the chips on their 'smart' widgets during the
manufacturing process, allowing them to make changes
in hardware-based code more easily and quickly.
However, they must make sure that the mock version of
the hardware API the developers use is always up to date
with the real API.
 Penul&ma e
fitness function:
MaintainTheMock

Build a fitness func&on that matches the mock API method signatures,
parameters, and types to the real API on the hardware; fail the build if
the team has allowed it to get out of sync.
Penul&ma e
 Penul&ma e
fitness function:
Doc Sync With API

Penul&mateWidgets publishes an API for external vendors to check on

order status, availability, and other B2B perks. However, the external
partners rely on the documenta&on of the API, which some&mes
becomes outdated when developers add new func&onality. The
Penul&mateWidgets architects want to ensure that documenta&on is
always up to date, especially method signatures and data formats.
 Penul&ma e
fitness function:
Doc Sync With API

• Parse the code to look for public method signatures

• Parse the documenta&on looking for <code> tags, which surround
method signatures and parameters
• Ensure that each tagged element includes a descrip&on
• Add a Publica&on Date value to the public website; aEer new
deployments, check to make sure the deployment &me matches the
Publica&on date
 Penul&ma e
fitness function:
Sell The Pla-orm
Penul&mateWidgets is in the process of rebuilding their core pla-orm,
incorpora&ng some key features that will appeal to others in a similar
problem space. Penul&mateWidgets plans to sell their pla-orm to other
companies, but worry that the differen&a&ng factors (such as
performance, scale, and elas&city) will suffer as clients start making
changes to the pla-orm, ul&mately harming Penul&mateWidgets
pla-orm's reputa&on. How can they sell their pla-orm and s&ll ensure
that the cri&cally important characteris&cs remain unchanged?
 Penul&ma e
fitness function:
Sell The Pla-orm

• Penul&mateWidgets sells not only the pla-orm but the running

deployment pipeline for the pla-orm as well. Penul&mateWidgets
engineers make sure that all the fitness func&ons run in the new client's
environment.
• Penul&mateWidgets will con&nue to cer&fy the pla-orm as long as all
their fitness func&ons con&nue to execute successfully.
Penul&ma e
Penul&ma e
 Penul&ma e

alien artifacts!
 Penul&ma e

integration
alien artifacts!
architecture
(Forced) Integra/on
(Forced) Integra/on
(Forced) Integra/on
(Selec&ve Use of) ESB
Enterprise Service Bus
“Legacy in a Box” Pa0ern
Enterprise Service Bus

“legacy in a Box”
Penul&ma e
Penul&ma e
Penul&ma e
 selected
experiments:
ideation

pivot

fold
double
down
portfolio
of experiments
Penul&ma e
Penul'ma e OpenUl'ma e
Penul'ma e OpenUl'ma e

Evolutionary architecture provides

the substrate for 21st century growth.
 Fitness Func*on Guidelines
most fitness function are local

global only when universally applied.

No architecture states!
 Fitness Func*on Guidelines
developers must collaborate

No architecture states!
 Fitness Func*on Guidelines
prefer automation but rely
on manual when needed

automation =

when needed
 Architecture Defined

architecture = +

prioritization cost
h"p://evolu+onaryarchitecture.com
iMpLemEN)Ng eVoLuTiONaRy
ARcHitECtuREs

@neal4d
nealford.com @rebeccaparsons @patkua

2020-09-10