Realising the Beneﬁts of Formal Methods
Independent Consultant, UKemail:
What are the real beneﬁts of formal methods and Why shouldwe care about them? When and Where should we expect to usethem, and Who should be involved? I suggest some answersto those questions and describe one approach,
, that has achieved practical success on severalreal industrial developments. Based on this I propose somechallenges for formal methods research.II. S
What have formal methods ever done for us?
Formal methods consist of writing formal descriptions, ana-lyzing those descriptions and in some cases producing newdescriptions for example reﬁnements from them. In whatway is this a useful activity?First, experience shows that the very act of writing theformal description is of beneﬁt: it forces the writer to ask all sorts of questions that would otherwise be postponed untilcoding. Of course, that’s no help if the problem is so simplethat one can write the code straight away, but in the vastmajority of systems the code is far too big and detailed to bea useful description of the system for any human purpose. Aformal speciﬁcation, on the other hand, is a description that isabstract, precise and in some senses complete. The abstractionallows a human reader to understand the big picture; theprecision forces ambiguities to be questioned and removed;and the completeness means that all aspects of behaviour for example error cases are described and understood.Second, the formality of the description allows us to carryout rigorous analysis. By looking at a single descriptionone can determine useful properties such as consistency ordeadlock-freedom. By writing different descriptions from dif-ferent points of view one can determine important propertiessuch as satisfaction of high level requirements or correctnessof a proposed design.There are, however, stronger claims sometimes made forformal methods that are not, in my opinion, justiﬁed. Thewhole notion of proof as qualitatively superior to other analysismethods seems to me wrong: proof is no more a guaranteeof correctness than testing, and in many cases far less of one.Furthermore, formal methods are descriptive and analytic: theyare not creative. There is no such thing as a formal designprocess, only formal ways of describing and analyzing designs.So we must combine formal methods with other approachesif we actually want to build a real system.
There sometimes seems to be a belief that formal methodsare somehow morally better than other approaches to softwaredevelopment, and that they can lead to the holy grail of zero defect software. This is nonsense, and the fact thatit’s so obviously untrue is part of the reason for the strongbacklash against formal methods. What is true, however, isthat formal methods contribute to demonstrably cost-effectivedevelopment of software with very low defect rates. It iseconomically perverse to try to develop such software withoutusing them. Figure 1 shows the defect rates achieved byorganisations at different capability levels together with themuch better defect rates achieved by using
, a formal-methods based process describedlater in this paper.
Average Defect Density of Delivered Software
012345678CMMLevel 1(7.50)CMMLevel 2(6.24)CMMLevel 3(4.73)CMMLevel 4(2.29)CMMLevel 5(1.05)Praxis(MGKC,0.04)
CMM data from Jones, Capers. Software Assessments,Benchmarks, and Best Practices. Reading, MA: Addison-Wesley, 2000
Fig. 1. Evidence for the achievement of low defect rates.
The reason that, contrary to popular belief, formal methodsactually save money is illustrated in Figure 2. This shows thecost of ﬁxing a requirements error the most common kinddepending on when it is discovered. Since formal methods helpus discover errors early in the lifecycle, they actually reducethe overall cost of the project.