Design Control Principles

Alford R. Taylor, Jr.

Presented to the ASQ Washington, DC Section

February 17, 2000

Thanks for having me as a speaker tonight. Frankly, the best thing about ASQ is
the opportunity to talk about quality with people who are interested in the topic,
because at work, a lot of people duck and cover when they see me coming..
I clocked this talk at an hour and fifteen the other day, so hopefully, with the benefit
of a little adrenaline, we’ll get out a little quicker than that, even. I won’t be
offended if someone beats a hasty retreat for any reason.
A lot of people assume that since I’m from the FDA, I’m going to provide a
“regulatory perspective”---whatever that is. But I’m not here tonight to tell you
what the FDA thinks, and I doubt that most of you care what the FDA thinks,
anyway.
The fact is that I’ve only been at FDA for about seven years, and prior to that, I
worked for a living, most recently, as a design engineer for a telecommunications
company. So my perspective is basically that of a design engineer and engineering
manager.
As an engineer in the trenches, I found that I spent a significant portion of my time
trying to educate my bosses about the designs I was producing and the steps
necessary to get to the end result. I used to get frustrated with all the time I wasted
trying to explain to them what was obvious to me and my peers. But over time, I
began to see things from their perspective, and started to figure out that the time I
spent teaching my bosses was well rewarded. That was probably the first step in
my own enlightenment regarding design controls..

1
Author’s Notes
You are welcome to use (or adapt) any or all of this material in your
own work, since its development was paid for the U. S. taxpayers.
Obviously, attribution would be appreciated, but I won’t be checking
up on you.

The opinions expressed are mine, and do not necessarily have the
backing of my employer.

A lot of the information in this presentation is contained on an FDA

document, Design Control Guidance for Medical Device
Manufacturers. In spite of the limited scope implied by the title, most
of the guidance contained therein is applicable to any product or
service. You can download the document from
www.fda.gov/cdrh/dsma/cgmphome.html

Incidentally, I would be pleased to receive any comments you have

on this presentation and/or the guidance document.

2
How to Reach the Author
Alford R. Taylor, Jr.
Engineering Team Leader
Medical Electronics Branch
Center for Devices and Radiological Health
U. S. Food and Drug Administration
12720 Twinbrook Pkwy HFZ-141
Rockville, MD 20852-1720
(301) 443-2536 x147
alt@cdrh.fda.gov

3
 The Corporate Manager’s
Perspective

u Why all the fuss?

u What are design controls?
u What do they do?
u How do they work?

The focus of this presentation is not the language of the

standard, but rather the underlying concepts. The
approach is pragmatic and issues-oriented.

I still spend a lot of my time talking to senior managers, but most of them are
not my boss. When I talk to these folks about quality management systems,
they generally have a handle on document control, corrective and preventive
actions, internal audits, and so forth. But when the subject turns to design and
development, you can often actually see their eyes start to glaze over. These
are managers who have been neglected by their design engineers.
It’s not that they are bored. They have the same kind of hangdog look that you
tend to see in nursing homes. It’s the look of someone who has lost control,
someone with chronic pain.
So I have searched for a way to reach out to these people, and tonight, I’m
going to share with you the approach I’ve found. If you’re one of those people
who feels pain whenever you walk into the design engineering department, I
hope that my presentation helps you. CEOs, CFOs, and marketing managers
typically aren’t interested in clause 4.4.3 of ISO 9001. A senior manager at
FDA was quoted just the other day as saying, “I’m pulling on the levers, but
the ship isn’t turning.” Just because we preach quality management to
industry doesn’t me we do it ourselves---yet. But this manager wants to know
what a QMS is, and how it’s going to help him steer the ship.
Now I realize that most of you are not typical corporate managers, so if I’m
preaching to the choir, feel free to let loose with a hallelujah or two. Even if
you are familiar with design controls, you may find aspects of my approach
useful in your own quality consulting work.

4
 Why All The Fuss?

A Real-World Example

Back in 1987, I was engaged by a corporate quality manager to review a

design for a software-controlled instrument used in orthopedic surgery. A
subsidiary of this corporation had just introduced this new product, only to
recall it when a significant software design error was detected by a user.
The problem had been corrected, but corporate management wanted a second
opinion before they approved the design change. They wanted to be sure that
there were no additional defects lurking in the design.
When I arrived at the subsidiary, I found a classical small manufacturing
company. They had five people in engineering, three or four in QA, a small
production shop, marketing, and front office. They had three basic product
lines, all hi-tech electronic instruments used in surgery. The failed product
was their first foray into microprocessors.
They had hired a software engineer to implement the microprocessor design,
then a contract engineer to pull in the schedule a little. These guys were the
stereotypical Lone Ranger programmer personalities (ISTJ). No one in the
company had any idea how to develop microprocessor software, or oversee
their effort. Management adopted a two-step oversight process:. Step number
one, they explained to these two guys what they wanted the product to do, and
step number two, they prayed.---for about a year
Their prayers were not answered.

5
 Why All The Fuss?

(the corporate manager’s perspective of what happened)

After the product crashed and burned, I was engaged to perform a technical
review of the design, which I did. But in my presentations to management, I
emphasized that the fundamental issue for this company was not flawed
design, but flawed design process. After the crisis, I stuck around and helped
the VP of engineering to document a rudimentary design control process.

OK, so how do I explain design controls to a VP of Engineering?

6
 What Are Design
Controls?
Design controls are:
u an integrated set of management practices
(policies, processes, and procedures), which
are
u applied to design activities
u to control the quality of products and
services.

Here is my working definition, parsed to make it a little easier to read.

Let’s look at that second bullet for a minute. The focus isn’t on how you do
design, it’s how you manage design. ISO 9001 is a quality management
standard, right? A lot of so-called quality management experts try to break
design and development down into a linear sequence of steps to be followed,
and that’s just not how things work in real organizations. That’s not what ISO
9001 is all about.
Now go back to the first bullet. Design controls aren’t any one specific thing
you do; these management practices are only meaningful in the aggregate. It’s
like trousers. Someone asked me the other day whether the word “trousers” is
singular or plural. I looked it up in the dictionary, and it says that trousers are
singular at the top and plural at the bottom. Others may differ, but as far as
I’m concerned, “design controls,” like “trousers,” are always referred to in the
plural.
ISO 9001 is widely referred to as a “process standard,”and if you’ve seen the
2000 edition of ISO 9001, they have really gone wild with this notion of
process. So let’s just talk about that for a minute.

7
 Typical
Process
Model

When people talk about processes, they tend to draw models that look
something like this. You start with an objective, perform some tasks, measure
the results, and apply constructive feedback, if you will, to optimize the
desired output. If you think in terms of the classical closed loop feedback
system, the parameter you are optimizing by this process is the quality of the
product or service you are designing.
Now, from a management point of view, that measurement step, represented
by the diamond, is critical because that’s where design decisions get confirmed
and cast in concrete. So it’s important to understand that ISO 9001 doesn’t
talk so much about what happens in the boxes. The focus of design controls is
on the the management and oversight of design activities, and that has more to
do with the diamonds, or decision points.
It’s also critically important to realize that design isn’t a straightforward linear
process, like I’ve shown here. It’s more of a three-dimensional array of
processes.

8
 Environmental Management
Risk Management

Quality Management

Top
Management
Quality
Assurance

Production

Design

Concept Requirements Preliminary Detailed

Formation Definition Design Design

There is no single design process that gets a firm from concept to production.
Rather, the organization must define and document its organizational structure
for performing designs, identify the information flows between organizational
elements, define when and how reviews are conducted, and so forth. The
design processes tend to be defined in terms of development stages. At each
development stage, various organizational elements have specific roles to play,
and they have a range of overlapping concerns.
So design controls don’t exist in isolation. It doesn’t make sense characterize
any specific management activity as “design controls” versus “risk control” or
“production control.” The CEO doesn’t wear a Design Control hat one minute
and a Risk Management hat the next. Rather, at any stage of development,
there are some basic questions to be asked, and some checks and balances that
can be employed, that touch on a range of management concerns and
responsibilities.

9
 ISO 9001 ISO 14001 ISO 14971
Quality Environmental Risk
Management Management Management

YOUR
MANAGEMENT
PROCESSES

Any management process that touches upon design activities should be

examined to see how it relates to the design control provisions of ISO 9001.
Ultimately, it may be desirable to create a traceability matrix linking your
management processes to the requirements of ISO 9001. If you subscribe to
ISO 14001 for environmental management, or ISO 14971 for risk
management, or any other management standard, you can do the same for each
of them.
You don’t have to organize your documentation in this way. You can have a
separate quality manual and procedures for each management standard you
subscribe to. But when you get down to the details, you’re only going to have
one document signoff procedure, not three or four. And you’re only going to
have one change request form. So it makes sense to have that one form serve
multiple purposes.
And let’s not forget your stockholders, or underwriters, who impose
requirements and influence decisions. Strictly speaking, I could draw a fourth
or fifth little box labeled “business plan” or “strategic plan.”
But in the real world, it looks more like this.

10
 ISO 9001 ISO 14001 ISO 14971
Quality Environmental Risk

$$
Management Management Management

YOUR
MANAGEMENT
PROCESSES

It seems like the issue of money is always looming large in the background,
and for a lot of organizations, money issues tend to take over. Fortunately,
management systems help to tame the money monster and keep it in the
background, under control.

11
 What Are Design
Controls?
The specific design management practices
you need in your organization depend on:
u the size of your organization and
u the technologies you employ

Now I want to re-emphasize that ISO 9001 talks about general principles that
apply across the board. But the specific management practices you need
depend on the size of your organization and the technologies you employ.
If your organization is three people working in a garage, you don’t need a lot
of formal procedures for information flow and design approvals. Everybody
knows what everybody else is doing all the time. You do need formal
procedures regarding recordkeeping, however, because in that environment,
it’s all too easy to fly completely by the seat of your pants and not write
anything down.
When I talk to medical device manufacturers, they are always talking about the
industry practice for this or that. I tell anyone who will listen that there is no
such thing as as a medical device industry when it comes to design
management. I’m sure it’s the same at the Government Printing Office, or
NASA, or wherever you happen to work.
For example, if your product or service incorporates software---and these days,
that’s everybody---you are in the software engineering industry.
If your product needs to have structural integrity, whether it’s a bridge, a bone
screw, an automobile bumper, or a Barbie doll, there are mechanical
engineering tools and practices to be followed.
You find out about these management practices by studying textbooks,
technology-sector standards, and trade magazines. Remember that the
management responsibility section of ISO 9001 requires that your people have
the appropriate expertise. That applies to design managers as well as
production workers.

12
 How Do Design Controls
Work?
u Via mechanisms to provide visibility (i.e.,
means to measure the controlled variable)
throughout the development process
u Via documented procedures to exercise
continuous (or at least frequent) control of
resources (i.e., feedback mechanisms)
u Via a semantic structure (language,
taxonomy) to facilitate communications

Visibility: If you’re buying a tract house, you can walk through the builder’s
model home to see what you are getting. A friend of mine is currently having
a custom house built, and he’s having a devil of a time communicating with
the builder. Most people aren’t trained in reading blueprints, and they can’t
generate a three-dimensional picture from what they see on the paper. It’s a
problem from the builder’s point of view, too.
Some examples of visualization techniques include: the requirements
document, rapid prototyping, computer simulation, design walkthroughs.
Here’s another practical example of visibility. A couple of my engineers are
currently designing some custom instrumentation for an X-ray calibration
facility. The customer, a physicist, was complaining that my engineers were
spending his money but not doing anything. So I had my engineers lay out a
detailed list of development tasks and we started having weekly status
meetings with the physicist. Now he has visibility into the process and the
progress. He has become more involved with the decision-making, because he
understands the context better when one of the engineers asks him for input.
Side benefits: developers can see what each other are doing, and developers
gain visibility into the needs of customers and bosses. (generals and trenches)
Resource controls: defined development phases, design reviews, supervisory
reviews, signoff procedures for design documents (the old principle of divide
and conquer)
Taxonomy---the principles of classification. R&D, V&V, design management
vs. risk management. There will be other examples of taxonomies later in the
presentation.

13
 What Are The Limitations?
u Design controls do not assure the quality of
products and services (but they provide a
framework for assessing and documenting
quality).
u Design controls do not prevent design errors
(but they facilitate finding and correcting
errors earlier in the development process).
u Management still needs the right people and
the right tools to do the design work and
review the results for adequacy.

The most frequent criticism of ISO 9001 is that it’s only an end unto itself, that
it doesn’t affect quality. The “evidence” for this is firms who are registered to
9001 and still manage to consistently produce shoddy products or services.
I carry an excellent road map in my car. I still manage to get lost once in a
while, but that doesn’t mean the road map is worthless. In fact, it helps far
more often than not.
This road map might be worthless to some people, either because they don’t
know how to read it, or they don’t know how to transform its information into
meaningful action.
It’s definitely worthless if it sits under the driver’s seat and only comes out
once a year when you replace it with the latest edition you just bought.

14
 Needs &
Intended Design Input
Uses

R eview
Process

Require-
Initial Design Possible
ments
Stage Interim
Reviews

Stage 1
Design
Output
... Nth Design
Stage
(N $ 1)

v erification Final
Design
Output
Production

V alidation Test
Articles

This diagram originated with Pierre Landry of Health-Canada, and a version of

the diagram was used with his permission in the DCGD that FDA put out. The
original version of the diagram had plain boxes down the diagonal, labeled
user needs, design input, design process, design output, and product or service.
The problem is that user needs don’t come in a box. I redrew the user needs
box as a cloud, because that’s what it looks like from a designer’s perspective.
Of all the user’s needs, you have to identify a few that you are going to try to
fulfill. And who is the user?. If the product is a bus, the person who buys the
bus is different from the person who drives it or rides it. All the rest of us have
to pay for the bus and breathe the fumes. There are a lot of needs that have to
be considered.
Similarly, design input doesn’t come in a box. Contrary to popular belief,
design input is not a tangible entity at all, but a process of gathering all
available information about how a device might fulfill one or more user needs,
and extracting requirements which completely characterize the device to be
developed. A requirements document is the tangible embodiment of the user
needs.
Design output is all the accumulated information and instructions for building
and maintaining the product, which if followed, will result in a device that
consistently meets specified user need(s). The design output also includes
explanatory material which illuminates the design and makes it easier for
reviewers to assess the adequacy of the design. You don’t need the theory of
operation and the software source code to build the product, but you do need
those things to review and maintain the product.
15
 Requirements Definition

If you don’t have a good requirements document, you needn’t bother with the
rest of design controls. Throughout the development process, the requirements
document acts as the surrogate for the actual user needs. It’s the ground truth
that you come back to over and over to help decide how to implement the
design. At each stage of development, the requirements establish the basis for
verification of the design
Corporate managers I talk to have a lot of misconceptions about this business
of requirements, so let’s look a little closer.

16
 Requirements Triad
u Functions
– What the device does
u Performance
– Accuracy, speed, reliability, environmental
influences
u Interfaces
– Input/output, power, data protocols, user
interface

This taxonomy derives from an obsolete military standard on specification

practices, MIL-STD-1490.
What functions are performed is a no-brainer.
How accurately and how fast those functions are performed is also a no-
brainer, but the detailed performance requirements often get sidestepped at the
requirements definition stage, leading to problems later in the development
process.
There are a host of other performance considerations that are commonly
referred to as the “ilities”---maintainability, testability, manufacturability,
profitability... .
By environmental influences, I mean how well the product performs when
exposed to environmental variations such as temperature, pressure, corrosive
chemicals, shock and vibration, electromagnetic interference, etc.
Every product or service is part of a larger system. The interface requirements
characterize those elements of the larger system that are outside the direct
control of the designer but which impose constraints on the design.

17
 Requirements---Guiding
Principles
u Must specify what is needed, not the
solution (example of machined aluminum
housing)
u Complete to an engineering level of detail
u Requirements are developed by engineers,
not by marketing department or users
u Capturing the requirements may consume
as much as 30 percent of the entire project
time budget

First bullet: housing example machined aluminum vs. casting or plastic.

Reduced time to market; superior EMI shielding, stronger than cast, but much
more expensive in volume production. It’s OK to specify the solution if there
is a good reason and that reason is explicitly stated.
Second bullet: electrical example AC power. 120 (North America), 240
(Europe), or 100 (Japan). 108-132, or 95to 132. What is appropriate for the
intended use?
Third bullet: 1987 case study example. When I asked for requirements
document, they gave me a five-page document prepared by their marketing
dept. It was the level of detail you might see on a glossy marketing brochure,
grossly inadequate for purposes of implementing the design. You need a
lawyer to write a will, and you need an engineer to write a requirements
document.
Fourth bullet: This is a tough sell for corporate managers from the old school.
Right now, FDA is undergoing “reengineering” of our regulatory processes.
Management want to see pilot programs, they want results they can brag about
at Congressional hearings and industry forums.
As I mentioned earlier, part of the solution is for managers to understand the
design and development process better and be able to track the paper
deliverables. Bean counters need something to count. They are almost as
happy counting documents as hardware items.

18
 When Do Design Controls
Begin?
u There is necessarily some overlap between
research and development
u Rapid prototyping may occur outside of
design control process, but don’t make the
mistake of assuming that an apparently
functional prototype is anything close to a
finished design
u Design controls should be in place prior to
approval of the system-level requirements
document

This is something that companies always ask.

There was a Dilbert cartoon a while back where Dilbert’s pointy-haired boss is looking at
a prototype of a product and saying, “This looks great. We’ll announce availability
tomorrow and ship next week.” In the next panel, Dilbert is saying, “But this is only a
prototype. We haven’t worked out the details yet.” In the last panel, there is a close-up of
the product. It is simply their competitor’s product with their logo pasted over the
competitor’s logo.
That’s good for a laugh in the comics, but I’ve actually seen it happen in real life! Don’t
ever confuse a prototype with an actual design. The purpose of research is to figure out
what the requirements are (and in some cases, prototypes help to shake some money off
the money tree). Many real-world prototypes are only intended to explore one or a few
novel design issues. The “routine” or non-critical features are ignored or simulated, often
by running “wires under the table” or putting your company’s logo on someone else’s
product.
The prototype is often a “best-case scenario.” In production, tolerances build up and the
pieces won’t fit together or the holes don’t line up as well. In addition, the prototype is
probably built by master technicians, whose training far exceeds that of the production
employee who will mass-produce the product. The same is true when you are designing
services. The pilot program is going to staffed by your more experienced employees, who
you have tapped to help you with the design. You may not get the same results when you
ramp up to full-scale operation.
So its OK to let your designers run free while you are trying to figure out what the
requirements are. If the research phase is resource-intensive, you may want to apply some
controls, from a purely cost-control standpoint. But here’s the bottom line: Design
implementation doesn’t begin until the requirements are solid, and the requirements aren’t
solid until design controls are in place.

19
 Requirements---Reviewing
For Adequacy
u Unambiguous (objectively verifiable)
u Quantitative limits expressed with a
realistic measurement tolerance
u Self-consistent
u Environment completely characterized
u Completeness and relevance of external
references

First bullet: Ex. “finish shall be durable”. When challenged, they said that it
should withstand repeated cleaning as described in the service manual. OK,
but it’s ok to scrape off big chips of paint every time another instrument bumps
into it in the O.R. The best approach is to specify some objective durability
test, often a consensus standard test method, and let a qualified reviewer
determine whether the test is representative of the intended use.
Second bullet: Quantitative limits can go to either extreme. If the requirement
is for a 10 centimeter hole, the designer must infer what that means, and you
are likely to end up with grossly inaccurate solution, or an overly complex and
expensive solution. If you say you need a hole big enough to accommodate a
mechanic’s gloved hand, that’s better, but the designer has to do some legwork
to figure out what the actual dimension needs to be before designing the
component and its production process. If you say the diameter must be a
minimum of 14 cm to accommodate a mechanic’s gloved hand, the designer
has enough information to implement the design, and a qualified reviewer has
a basis for determining whether the requirement is is appropriate for the
intended use.
Last bullet. Completeness and relevance of external references (MIL-ST-810
example)

20
 V&V
u Verification = assessing conformance to
requirements (did I do the design right?)
u Validation = objective evidence that
devices fulfills intended use (did I do the
right design?)
u I.e., verification is details-oriented and
validation is a cumulative summation of all
efforts to assess suitability of design.
u Validation almost always includes user
evaluation.

There is great confusion over these terms.

The military pioneered the concepts back in the 60’s and 70’s, but used words
like “first-article test” and “qualification”.

When software systems became complex, with many iterative stages of design
activity, the terms “V&V” came to mean something concrete. Let’s look at
that for a minute.

21
 REQUIREMENTS PRELIMINARY DETAILED
CODING
DEFINITION DESIGN DESIGN

v v v

LEGEND

v = VERIFY

V = VALIDATE Software V&V

In the software world, verification is a check at the completion of each stage to

see that the output conforms to the input for that stage. Because each stage
adds visibility to the overall product, it is also useful at each stage to check
back with the original requirements to see whether the design has strayed from
the original intentions.
You’ve heard of the term “design creep,” where designers keep adding and
changing features until the product becomes the Taj Mahal and bears little
resemblance to the original intent.
So Software V&V is a sophisticated set of audits intended to deal with this
kind of problem.
But what if you captured the requirements incorrectly in the first place? It’s
not good enough to conform to the requirements if the requirements don’t
satisfy the user needs. But software V&V doesn’t really address that issue.
Why not? Because software V&V arose back in the days of military contracts.
In that world, the customer wrote the requirements into the contract, and if the
requirements were wrong, it was the customer’s problem. In those days, the
software developer wanted the customer to make errors in the requirements,
because every time your customer changed the requirements, you could
renegotiate the contract and sock them with a hefty bill.

22
 Needs &
Intended Design Input
Uses

R eview
Process

Require-
Initial Design Possible
ments
Stage Interim
Reviews

Stage 1
Design
Output
... Nth Design
Stage
(N $ 1)

v erification Final
Design
Output
Production

V alidation Test
Articles

When ISO 9001 was being formulated back in the 70’s, there were two
problems with this approach to V&V. First, some of the ISO committee
members really didn’t understand the complexities of software development.
Manufacturers of toothpicks had no need for all that complexity. And more
importantly, they realized that in most design situations, you can’t blame the
customer if you got the requirements wrong.
So they redefined verification to mean conformance to the specified
requirements, and validation to mean that the requirements for a specific
intended use are consistently fulfilled.
By these definitions, software V&V is all verification, because everything is
traced back to the written requirements. Validation is a whole different ball
game.

23
 V & V---Semantic Differences
Software ISO 9000/Quality
Process Engineering
Engineering System Regulation
Domain
Domain Domain

Process Validation-- Verification--Conformance

Establishing objective to documented output
evidence that a process Design Verification--
from predecessor stage
consistently produces Conformance to
results or products Validation--Conformance documented requirements
meeting documented to documented system
specifications requirements

User testing-- Design Validation--

Conformance with user Conformance with user
needs and intended uses needs and intended uses

This slide summarizes what I’ve said in a different form. I won’t dwell on it
here, but you have it in your handout.

24
 V & V---Guiding
Principles
u V & V encompasses many activities: Tests,
Inspections, and Analyses
u V & V overlaps with design review to some
extent. Companies may draw the dividing
line anywhere reasonable.
u The design records should contain one or
more verification and validation reports
which summarize V & V activities, explain
discrepancies, and document approvals.

The first bullet here is really important.

A lot of people equate V&V with testing. Testing is a big part of V&V, but it
is by no means the only way to go.
A lot of requirements can be verified by inspection. An inspection is simply a
judgement of conformity, based on measurement or observation of a
characteristic.
Testing is a technical operation, usually procedural in nature, to assess one or
more characteristics of a product, process, or service.
Some folks (but not ISO 9000) talk about demonstration testing.
Demonstration testing shows the capability of a product or service to satisfy a
requirement or intended use, but does not prove that the entity will consistently
work as intended. Those Star Wars missile defense tests we have been reading
about in the papers recently are demonstration tests. It’s definitely bad to fail
one of those tests, but even if the system succeeds in shooting down a missile
once, that doesn’t mean the next one will work.
Analysis is used when it is impractical to test, or to supplement testing.
Analysis is a process of gathering evidence, often circumstantial or subjective
in nature, which tends to confirm the assertion at issue. This may include
techniques such as mathematical modeling or simulation of a process, or
extrapolation from prior experience. Example of roof that will withstand a
hurricane.

25
 Design Reviews
u The cycle is:
– Design
– Audit (V&V)
– Review

For example, a design review for a molded housing would not take place until
the documentation was complete, and various verification activities had taken
place. These might include a tolerance buildup analysis, inspection and stress
testing of a prototype, etc.
There can be a lot of overlap between verification and review. Many activities
can be classified in either category.
Verification connotes structured labor-intensive activities, and often produces
what might be termed interim judgements of adequacy. Verification activities
are often unilateral---one analyst or one department performs the activity .

Review connotes multilateral examination of results, and produces final

judgements of adequacy. Review teams typically examine both the design
artifacts and the relevant verification results. Reviewers may override adverse
verification results, or direct further investigation prior to resolving the
finding.

26
 How Many Reviews?
u At least one; no pat answer
u Tied to business risk---review prior to
escalation of resource commitment
u If the requirements are at all complex,
review the requirements prior to
committing design resources

Companies always ask this question.

Sector-specific standards provide guidance for some technologies.

Suppose you have had five systems engineers working on a system-level

approach, and you are getting ready to assign 25 engineers to do circuit design,
software development, mechanical layout, and manufacturing work
instructions. It just makes good business sense to get a multi-disciplinary
review of the preliminary design before committing all those people to the
project.

In all but the simplest of designs, a review of the requirements is in order

before implementation begins.

27
 Who Are The Reviewers?
u Independent review means only a fresh
perspective---can be from same
organization, but must not have been
involved in phase of development being
reviewed
u Must have appropriate expertise
u Users should be represented

Independence is often interpreted as meaning freedom from undue influence,

or the appearance of undue influence. But in this case, independence only
means a fresh set of eyes, with appropriate technical competence to assess the
matter under review.

Second bullet---this is a big problem for small organizations. I have heard of

companies trading off with other companies having the same kind of experts---
hopefully, not your direct competitor. You may have to hire a consultant to
participate in the review.

Third bullet---example of company who asked surgeon to participate in design

review. The surgeon may buy the product, but the nurse is the actual user, and
has insights the surgeon may lack.

28
Design Review Procedures
u May be one-on-one, a meeting, or off-line
review followed by a meeting to discuss
findings
u Should not be adversarial---the designers
know better than anyone where the weak
points are

29
Resolution of Design
Review Findings
u Not all “problems” detected by reviewers
are real, or need to be corrected.
u There should be a procedure for tracking
concerns and ensuring followup.
u There should be a procedure for resolving
differences of opinion.
u Design review procedures should identify
who is in charge.

30
 Everything Else
u Change control and configuration
management
u Design output---form and content
u Control of design documentation
u Design release

I could go on ad nauseam, but I think I have touched on the “interesting parts.”

Most manufacturers seem to have a pretty good handle on change control,
document control, and document approval procedures.

I hope that the next time you read the standard, it will make a lot more sense.

31