The IEEE Software Editorial Board has selected it as the best practice papt'r presented at ICSE-IS. Cleanroom software engineering avoids dependence on cosy defect-removal processes. The process model incorporates the statistical quality certification of code increments.
The IEEE Software Editorial Board has selected it as the best practice papt'r presented at ICSE-IS. Cleanroom software engineering avoids dependence on cosy defect-removal processes. The process model incorporates the statistical quality certification of code increments.
The IEEE Software Editorial Board has selected it as the best practice papt'r presented at ICSE-IS. Cleanroom software engineering avoids dependence on cosy defect-removal processes. The process model incorporates the statistical quality certification of code increments.
The philosophy behind
(Ceancom software
engjnering to cvid
dependence on costly
deecremval processes
by wing code increments
Tht he fis ie
and vying the
correctness before testing.
is proces model
incorporates the statistical
cucy econ of ode
increments as they
ccumat into o syste,
ROWDC INGER
[BA Ceannon Stee
Ted ee
Trees onesie proses and
society’ inerexsing dependence on
sofware have led toa new foes devel
lopment process. The Cleanroom pro-
cs, which has evolved over the lst de
‘ad, has demonstrated that itan improve
both he producti of developers who
use itand the quality ofthe software they
proce
Cleanroom software engineering is 2
teanvorened process dt makes devele
copmentmoremarageableand predicable
1 Sg EE CS Pe a Alte
{Gish el rad
ICS Tebow one ar Dei OSE
1s ropan Catas ehhparrnpg bs
Peete
because itis dane under satisical quality
contr
‘Cleanroom is 2 modern approach to
software development. In traditional,
ccf baed development, defees are re-
{garded a inevitable and elaborate dee
removal techniquesarea partofthe devel
‘opment process. In such 2 process,
sofcrare proceeds from development to
unit esting and debugging, then to fanc-
tion and system testing for more debug-
{ging Inthe absence of workable alterna
tives, managers encourage programmers
to get onde into execution quickly, = de-
Dbuggingean begin. Today developers roc-
ognize that defect removal isan error
prone, inefiiene att that consumes
resources beter slloete to getting the
code right the frst meClenrom ems at IBM and ther
ocpaneaons are shieing enable
tpaliy msn bah now sem dol
Cpe an lication a eens
Spay anes The ql ofsoirre
protec by Clamvoom development
Sane oe near des)
torte sftar ener oem ting
Gen or seater ecrton by |
The heontal oundaions of Cean-
room formal spctcatn and des,
Correomestvehcion, and tse
tag bie been refed pase
hd demarsted nary on ines
Tok Some Clenoan pres ae
pro inthe axon p58
‘QUALITY COMPARISON
Quay comparisons berween tad |
sional methods andthe Cleanroom pro-
ceo are meaningfl when mead om
fist excaton. Most wana develop
nen methods begin to mesure eras a
function texting (rate) oiting rors
foun in pia wi ting. A raion
project experiencing. ve errs pet
{thousand ines of ende (KLOG in ane
Sion testing may have encounteel 5 of
snore enors pet KLOG when measured
‘Eom rsexceion nun sing.
‘Arenuytounitesting, wana of
ware gpily exis 25 1035 or more
trrors per KLOC.! In contrast, the
veighead average of errs found in 17
Greanroom pro, involving nary a
non lines of ode, i 2 errors per
KLOC.Thisnuberrepreensallertn
found in ll cing, messed fon frst.
exer exceuontughetcompletion—
stich average numberof eid eons
present after the developmen tam has
Perfor corecanes verison
Tn aon to th remarabl die
coce in the number of error experience
has shown a qualtasve diference inthe
complet of ror fond in Clenvoot |
‘emus tonal sofware. Error Il be
hind by Cleanroom eorectes vero
tin tend noe to be complex design or n=
terface eros, bt sine mises sly
found and ied by sata esting
is arte, I dexbe the Clean
oom development process, fom specif-
Cation and design throwgh comrectness
‘eration and sastcal usage testing for
‘quality cerifention,
INCREMENTAL DEVELOPMENT
‘The Clesnroom proces is based on
eveloping and ceraiving 2 pipeline of
soltware increments tit accumulate into
the final stem, The increments are de-
‘eloped and certified by smal, indepen
den eas with eams of teams for arge
projects.
System incegrason is continual, and
famotonalyy grows with
the aldition of uccesive
Aste figure shows, Cleanroom devel-
opment involves two cooperating teams
and fe msjor activities
‘Specification. Cleanroom develop-
ment begins with specification. Together,
the development team and the ceriica~
sion team produce two specificitons
fanetional and use, Large prjecs may
havea separate specication team,
“The functional specification defines
the required enema sytem behavior in
allcreumstances of use; the wage spei-
‘ation defines usage scenarios and their
probabilities for al posible sytem usage,
toh correct and incorrect. The fare”
‘ional specification the
brs fo incremental sft
inerements. In this ap- J CLEANROOM, wire development. The |
the harmon ata the
erent DEVELOPMENT os
‘eperaion of fare incre-
bass for generating test
Monsanchenceteett WO ISINTENDED —swstoesencnelsc,
trinewca i pecined EN Seal estan ul
TOBE “QUICK v—
by increments already in
certiicason Usage spec
exci, thereby mini HM AND CLEAN,” eons are explned in
mizing interface and de-
the section on cetiica
Sov cros ani behing NOT“QUICK i.
developers maincin i> WAND DIRTY.”
tellecsl control
‘The Cleanroom de-
velopment process i in
‘ended tbe “quick an clean," nor“quick
| and diy." The ideas to quickly develop
‘heright producewid high quality or ee
wer, thon go an to the net version to
incorporate new requirements arising
| from tse experince
Tn :he Cleanroom proces, correctness
fs built in by the development tearm
‘through formal speciation, design, and |
verification. Tenn coreemes:veriietion
takes the place of une vesting and debug-
ging, and sofware enters yatem testing
ddrecly, with execution by thedevelop-
‘meat team. All erors are accounted for
fiom fist execution on, with no priate
dehogging permitted,
Figure 1 illatrates the Cleanroom
proces of incremental development and
‘quality cereficaton. ‘The Cleanroom
team fist analyzes and clarifies cstomer
equirement, with subsantal user nter-
action and feedbck- requirements arein
ddoube, the ten ean develop Cleanroom
prowoypes to elicit eedhack teratively.
1 Increment planning
‘On the basis of these
specifications, the devel:
‘opment and certification
teams together define an inital plan for
developing increments that wil accu
late into the final system. For example, a
100 KLOC sytem might be develop in
fiveincrements averaging 20 KLOC each,
The time it takes to design and verify in
crements varies with thet size and cor
plexity. Increments that require long lead
‘mes my cll for parallel development.
* Design anderson. The develop
rent team then carries out design and
‘correczessverieation eyde foreach in-
‘rement. The certification tem proceeds
inparalel using the wsige specication to
senerate test eases that reflect the exe
pected use of the accumulating incre
* Quality cerifcaton. Periodical, the
evelopment team integrates a completed
Jncrement with prior increments and de-
livers them to the test tam fr execution
of saistial text eases. The test eases are
run agaist the accumulated increments
and the ress checked for comrectnssFigure 1. Clezaroam process made. Te staked aves indicate sucesveinerements.
axgsins the fancional specification. Inter-
fal mes, that ithe elapred Snes he
‘pen ila, are passed 193 quality-cer-
‘ction mel that computes abjecsive
stadtial measures of quality, sch as
ean time oar. ‘The quality-certii=
‘ton model employs selablty growth
fextimator to derive the statistical mea
‘Certification is dane continuously,
cover the lite ofthe project: Higher leva
‘neremens enter the cerfeston pipeline
fist. This means major architectural and
des decisions are vated in execution
Tefre the development team elaborates
‘onthem. And becausecetficaon done
for all inerements as they accumulate,
higher level increments ae subjected (0
more testing than lower level increments,
‘which smplement localized functions,
+ Foadack, Erors ae returned the
Adevelopnent tea for correction. If the
aguality slow, managers and team mem
erst proces imprenensent. ASwith
any process a goon del of iteration and
feotbac is alays present ty accommo-
shite problems and solutions
In the next sections, I describe the
specification, design a verification, and
ualisycertificstion procedures. A de-
tailed desenpon of increment planning
and feedback mechanisms 6 ouside the
Scope ofthis artic
FUNCTIONAL SPECICATION
“The object-based technology of box
structures as proved) be an efecive
technique for functional specification.»
‘Through stepwise refinement, objects are
define and refined directx: |
tures sultingins usage hierarchy ofa)
jectsin which the services an abjectmsy
Fre sed an resem many plaes aad at
many kvels Bor strctures then, define |
reaqued sym behavior and derive and
unmet objets comprising a system ar
shitecure
Tn the past, without rigorous speci
‘tion technology, there was lle incen-
‘pve to devote mich effort o the spect
tion process, Specifications were
frequently writen in natural Ianguage,
with inevitableambiguies and omissions,
and often rege ashen stepping
sues to code
Box structures provide an exonomic
Jncenive for precision Inia Issue
ture specifications ofien reveal aps and
rnisunerstandings in custoaner require
nents that woul ondinaly be discovered
later in development at high cost an sk
tothe proeet
They ae adres the two engineering
| problemsasoctatedwithsstemspecii
tion: dining the night funceion for users
and dtningthe ight sruceure fe dhesp=
‘cation ine, Box structares ares the
Fs prblem by precisely defining the cr
rent undersnlng ui funesons at
‘ich stag of wlan so hat the n=
fins canbe review and oid i eco
‘32, The second problem iserteal, spe-
lly for large-spstem development Hose
‘ean ke organize che myriad deals of be
havior and processing into eoerent b>
stractons humans can understan
Bo structures ineorporte the cca
rmathematieal property of eeferental
transparency — dhe inforimion content
of cach bon specification is stint t0
define is refinement, without depending
fn the implementation of any ether box.
‘This propery les us organize large 3
tem spetications hierarehiclly, without
serifcing precision at high evel ordeal
alow levels
Bax stares. Thrve principles govern |
the we of box structures |
‘# Alldatadefined in design sen |
slated in boxes.
‘All procesing is defined by using
boxes sequentially oreoncurent‘Each box oceupies a distin place in
asjstem’s usage hierarchy.
Each box has dhe forms — black,
state, and clear — which have identical
‘external behavior but whose internal are
increasingly denied
Blak x. An objec’ black box is pre
ce specication of enema, were
behavior inal posible cieumstances ofits
‘use. The object may bean ensiresystem or
any par ofa sytem. Is wer may be a
person or another abject.
‘black baxaceepsa stimulus) from
ser and produces a response (R). Each
response ofa black bos determined by is
current stimulus history (SH), with a
‘black-box transition function
A given stimalus will produce different
‘responses thavare based on history of se,
‘ot aston the curentstimils. Imagine
‘aleulator with ewo simul hisories
and
Ifthe nexstimulusis6, the isthistory
produces a response of 7136 the second,
iG
‘The objective ofa black-box specifca-
tion i to deine the remonses proinced
for every posible scimulus and stimulus
history, ineloding erroneous and unex:
pected tml. By defining bchaviorscely
tn terms of stimuls histories, black-box
specifcaons neither depend on nor pre-
rmaturely define design internal.
‘Black-box specications are ofien re=
corded as tables In each row, the tials
and the condition on stimuhs stony are
sufficient w define the required response.
“To record large specifications, clases of
behavior are grouped in nested tables and
‘compact specification functions are used
‘oneal codons onstimulushis-
Side ba. An objec’ sate box is derived
from ts black box by identifying the ele-
‘ments of simul history that mist be re
tained as sate dita between transitions to
achieve the requied black-box behavior.
“The transion function ofa state box is
inininam o¢
flue of
[lser w vo
Inininin of {set y to absotute
set y to abaosute | dist |
value fee el |
sot 1 £0 value’ of wd) yi |
minim of aio
(Eitnd apsorute|| et w co minimin |=
fraive of xl |"] of andy) {set w to ainimun
ei ie
Figure 2, Stepwise refinement ofa lear-box design fragment tha can be verified. Each
fragment bas identical functional bebavor, eventhough the level of deal inereae.
‘where OS and NS represent old state and | procesing needs at each stage of refine-
new state, Although the external behavior | ment, not iavented 3 priori with uncer
‘fate boxis identical witscorrespond> | tin connections let to be defined later.
ing black boy, the stimulus histories are | ‘The design and verification of cear-bor
replaced with references to an old state | procedures isthe focus ofthe next scc-
and the generation of anew ste, a its | Gon.
‘wansions require. Because state boxes can be verified
[Asin the traditional view of objects, _ with espect to their black boxes and dear
sate boxes encapsulate sate data and ser_ | boxes with respect ther state bones, box
vices (methods) on tha dats, In this view, | srucearesbring corecmes verification to
stimuli and responses are inputs and out- | objectarchitecrues*
puts respectively, ofspeifiestte-boxser-
‘ice invocations thar operat: on state dt. | pESGH AND VERIFICATION
(Gearbox. An object's clear boxisderived The procedural control structures of |
fom itesareboxby defining procedure | srvctured programming sed clear-
toca ou the sate-box mnstonfime- box dexgn — sequence, alteration (F-
tin. The tnstion incon ofacearbox | ghen-ee) ad isto whe do) are
i Single-enty, singles structures dat
isn» mam mses | emo rie eee con
So a clear box is simply a progr tat | Bo. ssructres for concurrent
implement the corepondiag ate box. | Xrwon are de with in box structs,
‘Acleat box may invoke black bones at the | butare outside the scope ofthis article)
nextevel so the refinement procesisre-| When ie executes, «given contol
‘sve wth each dear box posblyinto- | structure simply tarsi ata em a
‘ducing opporunides for debningnew b- | input sate oan ouput ate, This trans
jeoarommmstociangons | frm, own 2 pan a,
Clear bows pay a crucial role inthe | cOmespondstoamathematical fnction:
sage hierarchy by ensuring the amon | defines a mapping fom # domain to a
‘mcooperton ofobjecsattenest level | Fnge by parla,
dfreinement. Object andtheirdearox | _Forimegers,x,y,and2,foresimpl,
‘connections are derived from immediate | the program function ofthe sequence,|
thle?
‘Nteration
Contra Cortese
strate: endtion
(2
Ip Wompisne
THEN oes gio fd
o whet i ee
LSE shot?
bh
de contra.stracure,
Subproofie:
a
| mm es) ta
Ese (16)
Ea 6
§ in concurrent asignment form,
For integer» 20, the program func |
tion ofthe eration |
'in English,
Desig ramets dsigning clea box
procedtres, you define an mend f=
tam thn refine tnt a contol ocr
and new intended fansions ss Figure 2
‘hse Teves neon ened in
tracey, are record in the design anda
‘ached to their construct eine
tment Inesenc cer oar compened
faint nuberofconel sre oc
ofwhich cn be checked for cores.
Design snplieaton san important
Figure 3. Corcnesconitons in quer | objx6¥e in the stepwise refinement of
{i orm fr verifing each typeof ear Cleat bnes. Te gals generate com
pact straightforward, vesifable designs
(D0 gigas (£2) No) 2
i Do [£3] END) 2
(wo 93:(84):98 END) 7
(OP p2 THM [f5) HLSE (£6) EMD) ?
(oo gf:95 Ex)
(00 96:7 ex)
Cmrecess via verify he cor
reemessof each contol structire, you de-
‘veitsprogram function —the function it
actualy computes — and compare tt
‘neended function, as recorded inthe de-
‘sign. A correcnes theorem” defines how
to do this comparison in terme of lan-
_guage- and application-independent cr
ee miy which on a each
igure 3 shows the correctness con
tions foe the saquence alteration, snd t=
‘erstion control sctures. Veriing ase
“qoenceinvolres function composition nd
requires checking exact one condition.
Verifying an alteration involves case
analysis and requires checking exacdy 00
conditions, Verifying an iteration involes
funcsion composition and case analysis in
arenriveeqitionand requireschecking |
‘eacly three condivions,
(Correctness verification has sevralad=
anges
‘Te raucessrfion tft pros
As Figure 4 illustrates, dhe nested, se
quenced way that control sutures are
“organized ina dear box naturally defines
hierarchy that reveals the eorectesscon-
sltions chat musthe verted. An axiom of
replacement” lets us substitute intended
funcions for thir congo scucture re-
finements in the hierarchy of subproos.
For example, the subproof forthe in-
tended function #1 in Figure 4 requires
roving thatthe competion of opers=
tions @1 and 92 with intended subfunc-
tion £2 has the sme effect on data as #1
Note that £2 substitutes forall the details
ofitsrefinerentin this proof This substi
tution localizes the proof argument tothe
contro structure at hand. In fet, it ets
{you cary ovtproofsin any ort.
Teisimpossble w overstate the positive
effect chat reducing verieation to a
nite process has on quality. Eventhough
allbuc the most aval programs exhibit
an essentially infinite number of execu
tion paths, hey canbe verified ina finite
number of tps. For example the cleat
hx in Figure § has exzely 15 correct
ness conditions that mast be verified.
It les Cleanrion teams very ery
Figure 4. A cear-bax procedure and its constituent sprnf. tn the figure, each p\ ix | Hine of design and coe. Tears ean carry
ft predcaeach 91 an operation and each £1 a ened fon
‘out the verification through groupanalysisanddiseussiononthebasisofthe
‘cormecess theorem, and they ean pro-
‘duce writen proofwhen ext confidence
ina life- or mission-critical stem i e-
‘quired.
16 rsule jn a meerzer deft ko.
During a team review, every corecmness
‘condition ofevery contol sractare isver-
ied in turn. Every team member must
agree that each contin score, 0 an
eroris posible only if every team mem-
ber incorreely verifies condition. The
requirement for unanimous agreement
based on individual verifications resus
in software tha has few or no delet be-
fore fist exccuion.
1 Irzales up Every software sytem, 10
‘mater how large, has top-level, cearbox
procedures composed of sequence, alte
ation, and iteration sroceires. Bach of
‘hese spially invokes a large subsystem.
with thowsandsoflinesofcode—andeach
of those subsystems has is own toplevel
intended fanctions and procedures, Sothe
correcmess conditions for these high-level
‘control stuctes ae verified in dhe sie
tray as are those of low-level sacar,
‘Verfication at high levels may take, and
well be worth, moze time, butt doesnot
take more theory.
(6 Teper cde han ui ting
Unit tering checks only the effect of
executing selected test paths out of many
possible paths. By basing verification
fn function theory the Cleanroom ap-
roach un verify every posible fet
fn all data, because while a program
‘may have many execution path, ie has
‘only one function. Verification is also
‘mote efficent than unit testing. Most
‘erfcatiom conditions cn be checkedin 8
few minutes, butunic tests ake subsatal
time to prepare, execute and check.
‘QUALITY CERTNCATION
Statistical quality contol used when
youhivetoo many temsto etal of hem
‘exhaustively. Instead, you statistically
Sample and analyze some items to obtain
scientific asesament of the quality of all
items. This technique is widely used in
‘manutturing, in whic ier on 3 pro
esi line ae sampled, thei quay is
Q += oda nunbers(Q) 1! oven manors tal |
‘rade God betorecfven (ALT 0)
ace + queve of integer {ini
evens | queue of integer {
es @ 1 odes,
eae 2 Expty |
| 3b lonaia
snaigh i= x
Lo 1 evers,
venas= roby]
32> fersveh
a
2 Scat)
7
Figure 5. A cear-box prowdure with 15 corrciness conditions 10 be verified. The
wal contra sractares and the muntber of corrciness conditions that mast be
Checked are sho in bol, Seg indicates a sequence, te indicates an alternation (if then-
i), and do indicates an iteration (cbile-do).
measured against presumably perfect de-
sgn, che sample quality enrapoated to
the entire production line, and flaws in
production are corrected if the quality is
tooo.
In hardware products, the statistics
sed to establish quality ae derived from.
slight variations inthe products physical
properties. Busoftare copies are ident
fl bit frit. What eta ean we sam
ple tocxrmapolate qual?
age esi. taros ot dat sofware
has sts propery of reatinse
developers and users iS exewton be-
havior How lng, on average, wil of
‘wae product eacete before ls?
From this noton is ened he pro-
cess ott geting in which os
‘+ simple the essotally init) pop:‘CLEANROOM QUALITY RESULTS
(Cleanroom projects report
tein err rt porta
Iie of de which represents
‘sida eros inthe safware
shercurecnes erica
“The pojcs briefly deserbod
here are among 17 Cleanroom
proj involving nena ml-
| ion ines ofcode thachave e-
| ported a oeghed cerage of 23
eraser KLOC fn i a t=
ng meando is-cer ex
ction f be nde a remark
able quay achievement!
| IBM Coe Searing Fe
| into fanedonaly equa
| Stscured oom rin
| | proved sodreaisiy
—
tcaigeor oe
Ton per LOG oe
tmbreompenee conic
oo
foe eects
saratoga open
Son, cerieston wr pub
tons and management, ver
aged ™40LOC per person
month, So fa a sal con
‘ofa person-year per year
| has been reuired forall
| maintenance and customer
| support Although the prod
| ucteahibis a complesty
| evel on the order ofa Cobol
| compiler, just seven minor
| eros were reported in he
fit dre yea oS we all
| | manent Re
amis, Cai, 198, pp 10-17
4 NASA litera pre
«et. The Coarse/Fine Ainade
Determination System
(CFADS) ofthe NASA Ars-
tude Ground Support Sytem
{(AGSS) was the ise Clean-
room projec cried ou bythe
Software Engincerng Labors-
‘wry ofthe NASA Goddard
Space Fight Cemer They
tem, comprising 40 KLOCof
Foran exited sexing
cerorrate of 4.5 errs per
‘KLOC. Prodacvty was 780
‘LOC per persn- month, 80
ercetimprovement ove pre-
‘ios SEL averages Some 60
percent ofthe programs com-
CGremand Vt Bal Evaae
‘ion of the Cleanroom Method
‘logy in the Sferare Eng
team developed the prowype
for thissystem, a 1$20-ine e-
latonal database sppication
‘written in Fork. Ithad a
testing errr rate of0.0er-
rors per KLOC— no com-
pilation errors were found
and nofllures were encoun-
tered in statistical esting
and quality certification
‘The sofeware was certified
attarget levels of reliability
‘and confidence. Team men-
‘ers atrbtederrorree
compilation and Shree
‘esting w the rigo of the
‘Cleanroom metbod. — CI.
“Tramunell 1H. Binder and
(CE Snyder, "The Aneomaced
‘Production Conral Syste A
(Case Sey in Cleanroom Sot
ware Engineering" ACM
“Tome Soars Bg. and Mab
dab, Jn. 192, pp. 81-94,
‘¢ IBM AOEXPERTIMVS.
AS0-peron ten developed
this comple decson-suppor
faci hat uses arial imali-
sence to predict and prevent
‘operating problems inn
MVS environment. The sys.
tem, write in PLT, C, Rexx,
and TIRS, totaled 107
KLOG, developed in thre in-
crements. thad testing
ror rate of 2.6 erors per
LOC. Cavs analysis of the
first 16KLOC increment re=
vealed that fv of eight
‘components experiencal no er-
rors xing.
“The project reported deve-
‘opment team producty of
44861,0C per person-monch
‘Nooperaonaleorshave
‘been reported to date from
‘beta test and cart wer sites —
PA Hauser, “A Recent Clean.
oor Success Story: The Red
wing Proje" Pra. 70h Sift-
sare Bg. Warkinp, NASA
(Godard Space Flight Center,
Greenbelt, Ma, 1992
NASA slit mere pro-
at Tro sale projec 3 20-
‘KLOC stinade- determination
subystem anda 150-KLOC
Aligh-ymamics syste, were
‘he cond ad third Clean
oom pojs undertaken a
[NASA Software Engineering
Laboratory. These gems had
‘combined testing ero ate of
$2erorsperKLOC.—SE.
Geeen and Rose Presi,
Cleanroom Proce Evolution
inthe SEL,” Pro 16 Soffzare
Eng. Wirklup, NASA God-
dard Space Fight Cente,
Greenbelt, Md, 1991
(IBV 30908 tape drive. §
fve-prsoa tam developed the