Auditing For Quality Manufacturing

Auditing for Quality
Manufacturing
Five Areas of Risk for Drug and
Device Manufacturers
© 2020 FDAnews. Digital version ISBN: 978-1-60430-125-0. Price: $397. All rights reserved. Photocopying or reproduc-
ing this report in any form, including electronic or facsimile transmission, scanning or electronic storage, is a violation of
federal copyright law and is strictly prohibited without the publisher’s express written permission.
This report may not be resold. FDAnews only sells its publications directly or through authorized resellers. Information
concerning authorized resellers may be obtained from FDAnews, 300 N. Washington St., Suite 200, Falls Church, VA
22046-3431. Main telephone: 703.538.7600. Toll free: 888.838.5578.
While every effort has been made by FDAnews to ensure the accuracy of information in this report, this organization
accepts no responsibility for errors or omissions. The report is sold as is, without warranty of any kind, either express
or implied, respecting its contents, including but not limited to implied warranties for the report’s quality, performance,
merchantability, or fitness for any particular purpose. Neither FDAnews nor its dealers or distributors shall be liable to the
purchaser or any other person or entity with respect to any liability, loss, or damage caused or alleged to be caused directly
or indirectly by this report.
Auditing for Quality Manufacturing: Five Areas of Risk
for Drug and Device Manufacturers
Table of Contents
Introduction...................................................................................................................1
The Importance of Data Integrity.....................................................................................2
A Brief History of Data Integrity Issues.........................................................................4
Industry Guidance......................................................................................................5
Making Data Integrity an Audit Focus...........................................................................5
Creating a Culture of Quality...........................................................................................7
Measuring Quality......................................................................................................9
Mitigating the Risk of Aging Facilities...........................................................................12
Auditing an Aging Facility...........................................................................................13
Making the Case for Modernization............................................................................14
Keeping Abreast of Changing Standards.....................................................................14
Slowing the Aging Process...........................................................................................14
Investigations..............................................................................................................16
Troubleshooting the CAPA Process.............................................................................17
Risk Management........................................................................................................20
Principles of Risk Assessment..................................................................................20
Implementing a Quality Risk Management Plan...........................................................21
Appendices..................................................................................................................23
A. WHO Annex 5: Guidance on Good Data and Record Management Practices
B. MHRA GxP Data Integrity Guidance and Definitions
C. FDA Data Integrity and Compliance with CGMP Guidance
D. ICH Q10: Pharmaceutical Quality System
E. GHTF Quality Management System — Medical Devices — Guidance on Corrective
Action and Preventive Action and Related QMS Processes
Auditing for Quality Manufacturing: Five Areas of Risk for Drug and Device Manufacturers
Introduction
Quality problems at manufacturing facilities can manifest in countless ways, from small
and insidious to sudden and overwhelming. Large or small, quality issues will nearly always
hurt drugmakers and device manufacturers in terms of both output and product quality. At best,
a persistent quality problem may lead to manufacturing stoppages and lost batches. At worst,
quality issues can result in large-scale recalls, cause drug shortages, hurt consumer trust and
cause regulatory blowback.
By regularly monitoring and auditing for quality, companies can stay on top of these risks,
identifying deviations before they have a chance to eat too deeply into the bottom line. Monitor-
ing tools, internal investigations of both manufacturing equipment and processes, and a focus
on risk management can all contribute to an overall culture of quality, helping companies catch
problems before regulators do.
In this report, readers will learn the key principles of auditing for quality. These include a
focus on data integrity, building an overall culture of quality that permeates the entire organi-
zation, mitigating the risk of aging equipment, investigating manufacturing deviations as they
arise and using the most current risk management tools. The report includes specific warning
signs that a company’s culture of quality is eroding, as well as research-backed attributes of a
mature quality culture. The report also walks readers through relevant quality regulations from
the FDA and international regulatory bodies and previews what’s coming as these agencies con-
tinue to reassess their own auditing strategies and priorities.
Portions of this report come from an FDAnews-sponsored webinar featuring Susan
Schniepp, currently a distinguished fellow with Regulatory Compliance, Inc., and a board mem-
ber and editorial advisor for Pharmaceutical Technology and BioPharm International maga-
zines.
1
The Importance of Data Integrity

Data integrity is at the heart of many of the quality issues that drug and device manufactur-
ers face. FDA warning letters frequently cite data integrity issues when they take issue with a
manufacturer’s corrective and protective action (CAPA) plans and investigations.
“If we look at the historical perspective, 483s for CAPA and investigations are always high,
at least in the top five, for both the device and pharmaceutical industry,” says Susan Schniepp,
a distinguished fellow with Regulatory Compliance, Inc., and chair-elect of the Parenteral Drug
Association (PDA). That scenario has not changed today. That’s why we focus on data integrity,
because it’s in the area of CAPA and investigations where you find elements of data integrity,”
she says.
Schniepp cites a definition of data integrity from the Regulatory Affairs Professional Society:
“Data integrity is the assurance that data records are accurate, complete, intact and main-
tained within their original context, including their relationship to other data records. This
definition applies to data recorded in electronic and paper formats or a hybrid of both. …
Ensuring data integrity means protecting original data from accidental or intentional modi-
fication, falsification or even deletion, which is the key to reliable and trustworthy records
that will withstand scrutiny during regulatory inspections.”
At the heart of data integrity is the familiar acronym ALCOA: attributable, legible, contem-
poraneous, original, and accurate. Those in the quality world will have heard these terms time
and again, but it can be useful for drug- and devicemakers to maintain a chart that defines the
terms and maps them to the relevant regulatory citations.
Figure 1. Data Integrity Principles and Regulatory Reference
Data Integrity Element Description 21 CFR Reference

Attributable All data generated or collected must be §§ 211.101(d), 211.122,
attributable to the person generating the data 211.186, 211.188(b)(11),
and 212.50(c)(10)
Legible All data recorded must be legible (readable) §§ 211.180(e) and
and permanent 212.110(b)
Contemporaneous results, measurements and data is recorded §§ 211.100(b) and
at the time the work is performed 211.160(a)
Original Original data, sometimes referred to as source §§ 211.180 and
data or primary data, is the medium in which 211.194(a)
the data point is recorded for the first time.
Accurate complete, consistent, truthful, and §§ 211.22(a), 211.68,
representative of facts 211.188, and 212.60(g)
Source: Regulatory Compliance Associates Inc., 2019
2
“As an auditor, there’s always someone who will ask you, ‘Tell me in the regulations where
it says that results and measurements and data need to be recorded at the time the work is per-
formed,’” Schniepp says. “So it’s helpful to have a chart with this information, something you
can point to quickly.”
Increasingly, quality departments and auditors are hearing the term “ALCOA Plus,”
Schniepp adds. This is rooted in the World Health Organization’s (WHO) 2010 report on good
distribution practices, and more specifically from the section of that report focused on data
management (see Appendix A). ALCOA Plus adds several data integrity elements that quality
departments must consider:
º Complete: Information that is critical to recreating and understanding an event, includ-

ing any repeat or reanalysis performed on a laboratory test sample.
º Consistent: The data are presented, recorded, dated or time-stamped in the expected and
defined sequence.
º Enduring: The data or information must be maintained, intact and accessible throughout
their defined retention period.
º Available: The data or information must be able to be accessed at any time during the
defined retention period.
“Data integrity is definitely foremost in the minds of regulatory authorities worldwide,”
Schniepp says.
º Data integrity violations could include:

º Fabricating data;
º Backdating;
º Presenting existing data as new data;
º Reporting only passing values;
º Testing into compliance by re-running samples until results pass;
º Discarding or altering data (either electronic or hard copies);
º Reporting data that’s not supported by raw data;
º Forging signatures, or otherwise unauthorized signatures; and
º Failing to record activities in real time.
Historically, these kinds of violations have applied to paper records, but electronic records
provide new wrinkles, Schniepp says. “You need to look at security, password security and
making sure that everybody in the lab has their dedicated password and dedicated login. If they
don’t, how are you controlling for potential manipulation of the data?”
3
A Brief History of Data Integrity Issues

Big, headline-grabbing events from the past three decades have helped set the FDA’s course
on data integrity regulation. In 1989, for example, a scandal erupted when several manufactur-
ers falsified data submitted to the FDA while seeking marketing authorization for generic drugs.
Out of that came what’s commonly known as the Barr decision (United States v. Barr Labora-
tories), which forced a recall of the drugs in question and led to more than 25 consent decrees
between the FDA and the companies. To date, only a couple of those companies have actually
been removed from the consent decrees.
In 2005, Able Laboratories made headlines when it came to light that the company had not
disclosed to the FDA that several drug products had failed quality tests. In some cases, the com-
pany substituted passing results for failing ones, with supervisors and lab analysts cutting and
pasting computer records. All the company’s drugs were recalled; Able Labs eventually went
out of business.
In 2013, an Indian pharmaceutical company, Ranbaxy Laboratories, was swept up in a simi-
lar scandal after a whistleblower told authorities that the company had “created a complicated
trail of falsified records and dangerous manufacturing practices,” according to court documents.
The company pled guilty to felony charges, including knowingly making false statements to
the FDA. In 2014, the agency prohibited Ranbaxy from making and distributing regulated drug
products.
A number of industry changes have also brought data integrity to the forefront of both drug
and device approvals. These trends can exacerbate existing data integrity issues and create new
ones, Schniepp explains. Examples include the emergence of:
º The generic industry;

º Biosimilars;
º Virtual companies;
º Contract manufacturing organizations (CMOs); and
º Compounding pharmacies and outsourcing facilities.
Others include a loss of institutional knowledge through increased mergers and acquisitions,
and increased reliance on information technology. “I’m not saying these are good or bad trends,
but these are some of the dynamics that have led to a resurgence in data integrity violations,”
Schniepp says. There are now more prescriptions written for generic drugs than for proprietary
products, and the FDA—at the direction of Congress—is pushing for quicker approvals of ge-
nerics. For example, many virtual companies have no manufacturing facilities of their own and
may rely on contracted companies to do their quality management.
4
Industry Guidance
Because of the slew of data integrity issues across the pharmaceutical and medical device
industries, the FDA and other regulatory bodies have released a number of industry guidance
documents. These include:
º The UK’s Medicines and Healthcare products Regulatory Agency (MHRA) GxP Data
Integrity Definitions and Guidance for Industry, July 2016 draft version for consultation
(see Appendix B);
º The FDA’s Data Integrity and Compliance with CGMP, April 2016 draft guidance (see
Appendix C);
º Draft Pharmaceutical Inspection Co-operation Scheme (PIC/S) Guidance: Good Prac-

tices for Data Management and Integrity in Regulated GMP/GDP Environments, Au-
gust 2016; and
º The PDA’s Elements of a Code of Conduct for Data Integrity, 2015.

Making Data Integrity an Audit Focus
Data integrity should be first and foremost in an auditor’s mind, Schniepp says, because it
certainly will be first and foremost in the mind of any regulatory inspectors. “It’s not just data
integrity in and of itself, but data integrity leads to a whole host of issues and problems in the
manufacturing arena,” she says.
As an example, she offers a story from a manufacturing facility she audited after taking
over as vice president of quality for the company. She was reviewing deviations and investiga-
tions, and came across an operator—we’ll call him Craig—who had at least 20 deviations over
a three-month period for the same thing: a failure to sign the batch record at the time the work
was done.
Every deviation she looked at indicated that Craig had been retrained. “I questioned the staff
and said, ‘Is Craig a good operator? Because he doesn’t seem to be getting this.’ But they all
said, ‘Oh, no, he’s one of the best. He’s very conscientious.’ So I said, ‘Well, why is he not get-
ting this? Why does he keep having the same deviation?’”
The staff told her that they’d retrained Craig, but that for some reason he seemed to have a
problem when it came to this one specific product. He worked the second shift, and so Schniepp
observed him for a shift while he produced the product. She questioned Craig about the re-
peated deviations, and he explained that to sign the batch record, he’d have to leave the prod-
uct unattended, go out to the sterile core of the facility—this was an aseptic process—sign the
document, then re-gown, go back in and again watch the product. From a quality standpoint, it
didn’t make sense.
“So, the real root cause of this deviation was a poorly designed batch record,” Schniepp
says. “Well, that’s pretty easy, right? Redesign the batch record, put the signature at the end and
move on. Except the data integrity issue was that Craig wasn’t the only operator making this
5
product. But none of the others had this deviation. That’s a bit of a data integrity nightmare,
because of everybody there, only Craig was really doing the right thing … everyone else was
kind of faking it.”
No one in the organization examined this particular deviation, which caused a bigger issue.
The story also points to the fact that data metrics, in order to be meaningful and effective, must
be combined with a strong culture of quality.
6
Creating a Culture of Quality

In 2005, the FDA signaled a new priority, urging drugmakers and device manufacturers to
cultivate an overall culture of quality. The agency went so far as to suggest that companies who
could demonstrate a commitment to a culture of quality might see fewer or less burdensome
inspections. Janet Woodcock, director of the FDA’s Center for Drug Evaluation and Research
(CDER), said she wanted to see “a maximally efficient, agile, flexible pharmaceutical manu-
facturing sector that reliably produces high-quality drug products without extensive regulatory
oversight.”
Beyond the FDA, a number of international regulatory bodies and advisory groups have
issued guidance on the importance of building a culture of quality. The PIC/S, for instance, in
its Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments,
says:
“Management should aim to create a work environment (i.e. quality culture) that is trans-
parent and open, one in which personnel are encouraged to freely communicate failures and
mistakes. Organizational reporting structures should permit the information to flow between
personnel at all levels (6.3, Quality Culture).”
The MHRA, meanwhile, includes in its GxP Data Integrity Definitions and Guidance for
Industry, several principles of data integrity, including two that specifically mention a culture of
quality:
“3.1 The organization needs to take responsibility for the systems used and the data they
generate. The organizational culture should ensure data is complete, consistent and accu-
rate in all its forms, i.e. paper and electronic.”
“3.3 The impact of organizational culture, the behavior driven by performance indicators,
objectives, and senior management behavior on the success of data governance measures
should not be underestimated. The data governance policy (or equivalent) should be en-
dorsed at the highest levels of the organization.”
WHO also speaks of a culture of quality in its Guidance on Good Data and Record Manage-
ment Practices:
“1. Introduction
1.4 Examples of controls that may require development and strengthening to ensure
good data management strategies include, but are not limited to: adoption of a quality
culture within the company that encourages personnel to be transparent about failures
so that management has an accurate understanding of risks and can then provide the
necessary resources to achieve expectations and meet data quality standards.”
Later, in the same document, WHO expands on its vision for a company’s culture of quality,
including among its key principles for data management best practices:
7
“4. Principles
4.7 Quality culture. Management, with the support of the quality unit, should establish
and maintain a working environment that minimizes the risk of non-compliant records
and erroneous records and data. An essential element of the quality culture is the trans-
parent and open reporting of deviations, errors, omissions and aberrant results at all
levels of the organization, irrespective of hierarchy. Steps should be taken to prevent,
and to detect and correct weaknesses in systems and procedures that may lead to data
errors so as to continually improve the robustness of scientific decision-making within
the organization. Senior management should actively discourage any management prac-
tices that might reasonably be expected to inhibit the active and complete reporting of
such issues, for example, hierarchical constraints and blame cultures.”
A true quality-focused culture, she argues, is one in which employees not only follow
guidelines, but consistently see others focusing on quality and making quality-based decisions.
A corporate culture indicates—to employees and others—what the company values and what’s
important to its management.
Some warning signs that a company does not have a culture of quality include:
º The CEO and other senior executives rarely discuss quality, let alone performance
against quality objectives;
º The company’s quality vision is either nonexistent or has minimal linkages to business
strategy;
º Managers throughout the organization either fail to consistently emphasize quality or

are resistant to quality initiatives;
º The organization has few, if any, feedback loops to continuous improvement of pro-
cesses;
º The company lacks formal mechanisms for collecting and analyzing customer feed-
back;
º Metrics used for performance evaluation feature little to no mention of quality goals;
º Employees are unfamiliar with the company’s quality vision and values—or, perhaps
worse, they view them as mere slogans;
º Training and development do not emphasize quality;

º New hires are not formally introduced to the organization’s quality vision and values;
or
º The organization experiences frequent, though often minor, setbacks owing to inconsis-
tent quality.
8
Measuring Quality
But how can you measure, in an audit, something as seemingly amorphous as a company’s
culture? PDA has been working on a standard that could help quantify a culture of quality,
Schniepp says. “PDA set out to determine: Is there a set of mature quality attributes that are a
surrogate for quality culture behaviors? That’s what we asked ourselves. The theory was that if
quality attributes equal quality behaviors, and quality behaviors equal a quality culture, then by
measuring the quality attributes you’d be able to measure the culture,” she says.
PDA conducted a survey and a statistical analysis of the survey data, which was ultimately
published in its Journal of Pharmaceutical Science and Technology, and which the PDA hopes
will lead to an ANSI-approved standard for measuring quality.
For the study, PDA began by identifying quality attributes that could be measured. These
included:
º Deviations reporting;
º A change control system;
º A CAPA system;
º A complaints management system; and
º An environmental monitoring program.
Quality behaviors, on the other hand, must be carefully observed or experienced and are
more subjective. These include communication and transparency, rewards and recognition,
engagement and a company’s cross-functional vision. The idea of the study was that if the attri-
butes could be measured and linked to quality outcomes these behaviors could be assumed. The
study authors came up with survey questions in seven areas:
1. Prevention programs;
2. Quality management and issue escalation;
3. Training and personnel development;
4. Quality system management;
5. People and communications;
6. Continuous improvement; and
7. Site metric reporting.
The questions yielded information that the researchers mapped to 42 specific quality-related
behaviors. The results were peer-reviewed and confirmed. An analysis of the data revealed 15
quality attributes and actions that Schniepp says are most closely correlated with an overall
culture of quality:
1. Attending and participating in professional conferences to stay current in the field;
2. Collecting error prevention metrics;
9
3. Communicating that quality is everyone’s responsibility at the management level;

4. Utilizing proven technologies;
5. Creating clear performance criteria for feedback and coaching;
6. Maintaining an environmental program with trained staff (risk assessment, emission
controls, spill prevention and response);
7. Implementing formal quality improvement objectives and targets;
8. Including quality topics in at least half of “all-hands” meetings;
9. Collecting management review metrics;
10. Collecting employee turnover rate metrics;
11. Creating a program to show how employee’s specific goals contribute to overall
quality goals;
12. Creating a program to measure, share and discuss product quality performance and
improvement from shop floor to executive management;
13. Instituting a continuous improvement program;
14. Putting in place a program that establishes a quality system maturity model, action
plan and tracking to measure progress; and
15. Collecting internal survey data to measure a company’s quality culture.
Schniepp says the study also highlighted a dividing line between traditional quality systems
and what she calls “enhanced” quality systems. The former might include, for example, systems
for deviations, complaints, change control, disposition, CAPA, specifications and environmental
monitoring. The latter might include a risk management program, a knowledge management
program and a specific quality manual.
“Those are the things you want to look for in a company when you’re talking about their
quality system,” Schniepp says. “You’re going to start by looking at the basics, and that’s o.k.
Then you’re going to look to see if there are any enhancements. This will give you a flavor of
how serious they are about their culture. It’s not the culture of the quality department. It’s the
culture of quality within the organization. It has to be both top-down and bottom-up.”
Both the MHRA and the FDA have trained inspectors on the quality assessment tool de-
veloped by PDA through this study. PDA is lobbying for the tool to become an ANSI standard.
“We determined that there really was nothing there to help the industry get a handle on how to
effectively audit a culture,” Schniepp says.
It could take up to two years before the tool becomes a standard, she says. If ANSI does
ultimately recognize it, the standard wouldn’t become a regular requirement, but rather a vol-
untary—perhaps widespread—tool that both industry and regulators could use to benchmark
and study quality culture. “It’s not enforced by regulatory authorities unless they choose to,”
Schniepp says. “It’s just a tool we’re trying to develop to give the industry a chance to start
assessing a quality culture a bit better, whether that’s auditing a CMO, a supplier or their own
internal manufacturing facilities.”
10
Regardless of what happens with the standard, Schniepp says she expects FDA inspections
to continue to include more quality metrics. The agency has been developing its New Inspec-
tion Program (NIP), and while it’s far from set in stone, all indications are that it will include an
explicit focus on quality culture.
“In a traditional inspection, the quality culture of a company was really only covered in an
informal, associative way,” she says. “But in the NIP protocol, they are going to give explicit
coverage for quality culture. I think we’re seeing the door open to more questions that try to
pinpoint the culture of an organization.”
11
Mitigating the Risk of Aging Facilities

The Food and Drug Administration Safety and Innovation Act, signed into law in July
2012, requires the FDA to make annual reports to Congress about the state of drug shortages
in the pharmaceutical industry. In the years since the law was enacted, the agency has placed a
good deal of the blame for shortages on aging manufacturing facilities. The FDA has reported
that roughly 54 percent of drug shortages are due to product quality issues, while 21 percent
are caused by manufacturing delays. Both can be caused by an aging facility, according to
Schniepp. “It’s like when your car ages,” she says. “You can only do so much maintenance be-
fore it breaks down badly enough that you’re better off investing in a new car.”
It’s not just regulators who have sounded the alarm. Many in industry are also concerned
about aging facilities. Aging becomes a particular problem when companies focus primarily—
or even exclusively—on a cost-of-goods business model. This often leads to short-term thinking
that resists upgrading or replacing aging equipment because of the cost of initial investment.
That way of thinking can lead to shortages when a single, aging facility is the sole supplier of a
particular drug product.
A quality culture is even more crucial around an aging facility, Schniepp says. More pro-
cesses will tend to be nonautomated, introducing more chances for human error. Precision will
be critical, which often relies on long-term employees with a strong working knowledge of
relevant processes—and sometimes of a facility’s quirks.
“Not every line running critical products is a current, state-of-the-art line,” Schniepp says.
“That doesn’t mean it’s bad. It’s just a situation that exists, and how you manage that situation
is going to be critical, since you’ll likely be running up against risk management issues.”
Older facilities tend to have more equipment breakdowns. They also tend to have more con-
tamination issues: old wall panel material can degenerate and old floor tiles are vulnerable to
seepage. “These rooms weren’t built with the state-of-the-art floor tiles,” Schniepp says. “There
are still gaps like you’d see in a bathroom tile.”
Quality issues tend to arise more often in aging facilities, leading to more frequent shut-
downs. Remediation costs tend to be higher, since replacement parts for older equipment can
be more difficult to come by. “Sometimes the equipment breaks down and you can’t get change
parts, so you’re jerry-rigging a part or you’re trying to fix a broken part,” Schniepp says.
It’s not just the manufacturing equipment that can become a problem. Old equipment often
means old procedures and outdated validation processes. So not only is the equipment less reli-
able, the means for monitoring and controlling it may be less reliable, as well. “Unit operations
break down. Supply chains can be disrupted. If you start to see scrap rate increases and yield
decreases, you’re probably dealing with a ticking time bomb,” Schniepp says.
12
Auditing an Aging Facility

When auditing an aging facility, the first step is to understand everything that comes under
the umbrella of “facility” for the purposes of an audit, Schniepp says. That includes the struc-
ture itself, as well as buildingwide systems that support manufacturing, including:
º Wall, ceiling and floor composition and layout;

º Water systems;
º Compressed air systems;
º Clean steam systems;
º Automated facility control systems; and
º HVAC systems.
An audit of a facility should also look at facility personnel, material and water flows, and
the overall facility layout, including cleanroom classifications and pressure cascades.
Next, an auditor should understand what qualifies as a “process” for purposes of an audit.
That should include the manufacturing process (e.g., formulation, sterilization and filling) and
equipment related to that process (e.g., bio reactors, process vessels, filling lines and liophilyz-
ers). An auditor should also look at process flows, product transfers and the flow of raw materi-
als or components into the process.
“As you get more mature with product, you’re going to change the process,” Schniepp says.
“You’re going to improve the process. You have to make sure that your validation supports that.
If you have a process validation that’s, say, five years old, and you haven’t revisited it, it’s time
to look. Don’t be scared to look. You can’t be scared of these things. Improvement and updating
are good.”
An audit should also dig into analytics. This should include data from in-process tests of the
manufacturing process (e.g., host cell proteins, conductivity, pH, potency, prefiltration total mi-
crobial count and sterility) as well as in-line testing, analytical technology tools and sensor tech-
nology. Simulation tools can be used to mimic specific quality and attribute shifts. Data from
these tools should be analyzed and corrected as necessary. An audit should also include auditing
the technology used for these analytics, Schniepp says, because that technology also ages.
“You need to consider looking at not just your manufacturing equipment when it comes to
aging, but looking at the processes and also your analytical lab,” she says. “There’s a lot of old
technology out there. You have to look at how old your equipment is. If you have a POC ana-
lyzer and it’s on its last legs, if it breaks down and you can’t get it fixed or you can’t get a new
one fast enough, you’re going to halt production. You won’t be able to release your product,
because you won’t be able to release your water to manufacture that product.”
There are a number of signs that indicate a facility is showing the effects of aging. These
could include a decrease in lot acceptance rates or an increase in recalls. Aging facilities will of-
ten show physical deterioration that will be noticeable in an audit. There may also be a measur-
13
able uptick in the amount of required maintenance. Decreased yields and increased deviations
are also signs of aging. These are all important trends to examine, Schniepp says.
Making the Case for Modernization

There may be impediments to modernizing manufacturing equipment, including the initial
cost. Management may be unaware of the problems or unaware that problems cropping up are
related to aging. Regulatory cost and time can also be factors, particularly if new equipment
would trigger the need to file postapproval supplements with the FDA.
However, a well-designed business case can help convince management that an investment
in new equipment will pay dividends over time, or in some cases even in the near-term, through
increasing both production capability and production quality.
The regulatory work may also be less burdensome than management imagines. Any time
an upgrade is made, it’s important to communicate that to regulators. But in a lot of cases the
filings may be relatively straightforward, Schniepp says, if the replacement is “like-for-like.”
“If you keep it simple, you can replace a whole line under a comparability protocol,” she says.
“It’s important that you talk to your regulatory authorities, to say ‘this is what we want to do.’
As long as your washer is a washer, or your filling machine is a filling machine, you can update
sections of the line” without a major filing. Some facilities will simply make a note of replace-
ment equipment in an annual filing, she says.
Keeping Abreast of Changing Standards

Whenever standards are updated or revised, someone in your organization must perform
a gap analysis. It’s also important to monitor changes to the pharmacopeia for any drug prod-
ucts your company manufactures. “If you have generic products, you’re going to be controlled
somewhat by what’s published in the pharmacopeia, because that’s what you’re typically go-
ing to be following,” Schniepp points out. “The pharmacopeia can be very slow to update, so
you want to make sure there’s somebody who’s taking a look at that and updating as standards
change.”
Slowing the Aging Process

Predictive maintenance programs can be a useful tool to help offset the aging process.
Enhanced process controls are also a good idea to monitor any equipment deterioration. A risk
management program also comes into play, Schniepp says. “You may not be able to actually
slow the aging process, but you can give yourself more signals to be able to predict that you’ve
got something going on and to take appropriate action to stave it off.”
Schniepp illustrates with an aging facility where she did quality assessment work. The facil-
ity made a lyophilized product, and during a visual inspection one day, a few green vials ap-
peared. “They’re not supposed to be green,” she says. The team had the product analyzed, and
the green turned out to be copper. But there was no copper in the facility. So the company called
in a consultant to investigate.
14
Meanwhile, the organization continued the manufacturing process. The copper, everyone
decided, was simply a glitch. “We blamed it on the glass supplier,” Schniepp says. “They must
have copper in their facility. So we called for an immediate for-cause audit of the glass sup-
plier.” That audit didn’t find any copper in their facility, either. Then, about a week later, more
green vials appeared. “Now we’re scratching our heads,” she says, “because we don’t know
where this is coming from.”
In one of the investigation tools, there was a low probability of copper piping in the depyro-
genation tunnel for the ovens. Old lines, she says, have copper piping above the depyrogenation
tunnel, whereas newer lines have those pipes below the tunnel. It turned out that the 30-year-old
tunnel was failing. A dew pressure valve was not turning off correctly, and water was building
up on top of the HEPA filters. When the filters couldn’t handle it anymore, they breached cop-
per-colored water into the vials as they were going through the tunnel.
The upshot, Schniepp says, was that the problem didn’t get solved until 27 potentially
compromised lots of the product had been made. “It was $28 million worth of lost product. We
lost everything produced on the line, because we couldn’t predict when it would have breached
in that time period,” she says. Better monitoring could have predicted the risk. “It cost a lot to
overlook that one area,” she says. “That’s where an aging facility can really cost you money. If
you’re not on top of things, you can have an event like that where it becomes catastrophic.”
15
Investigations
Regular investigations are a crucial part of a quality culture—and also required by just about
every regulatory or licensing body worldwide. In the U.S., the regulatory authority for investi-
gations in drug manufacturing comes from several places, including 21 CFR 211.22(a):
“There shall be a quality control unit that shall have the responsibility and authority to
approve or reject all components, drug product containers, closures, in-process materials,
packaging materials, labeling and drug products, and the authority to review production
records to assure no errors have occurred, or if errors have occurred, that they have been
fully investigated. The quality control unit shall be responsible for approving or rejecting
drug products manufactured, processed, packed or held under contract by another com-
pany.”
For medical devices, 21 CFR 820.100 calls for “investigating the cause of nonconformi-
ties relating to product, processes and the quality system,” while 21 CFR 820.198 states: “Any
complaint involving the possible failure of a device, labeling or packaging to meet any of its
specifications shall be reviewed, evaluated and investigated.” All investigations must be docu-
mented under 21 CFR 820.90, while 21 CFR 820.100(a) codifies the need for CAPA: “Each
manufacturer shall establish and maintain procedures for implementing corrective and preven-
tive action.”
The European Medicines Agency, meanwhile, spells out similar requirements in its EudraLex
Volume 4:
“A Pharmaceutical Quality System appropriate for the manufacture of medicinal products
should ensure that:
º The results of product and process monitoring are taken into account in batch release,
in the investigation of deviations, and, with a view to taking preventive action to avoid
potential deviations in the future.
º An appropriate level of root cause analysis should be applied during the investigation
of deviations, suspected product defects and other problems.
º Appropriate corrective actions and/or preventive actions (CAPAs) should be identified

and taken in response to investigations.
º A review of all batches that failed to meet established qualifications and their investiga-
tion.
º A review of all quality-related returns, complaints and recalls and the investigations
performed at the time.”
Other places where manufacturers can find guidelines on investigations and CAPA include:
º ISO 13485: Quality Management for Medical Devices;

º ICH Q10: Pharmaceutical Quality System (see Appendix D);
16
º The Global Harmonization Task Force (GHTF) Quality Management System — Medi-
cal Devices — Guidance on Corrective Action and Preventive Action and Related QMS
Processes (see Appendix E).
Troubleshooting the CAPA Process

The purpose of an investigation is to get at the root cause of any deviation and take appro-
priate corrective action.
The key steps of the CAPA process are:
º Identifying issues: production and process nonconformances, complaints, audit obser-

vations and trends;
º Performing risk assessment: investigating to the root cause;

º Resolving the issues: corrections, interim controls, corrective action and preventive action; and
º Verifying the resolution: effectiveness checks.
Figure 2. The CAPA Process
Review
Detect Report Nonconformity
Nonconformity Nonconformity Against Criteria
Evaluate Need for Perform Root Issue CAR

Corrective Action Cause Analysis
Record the
Implement Results of Verify Effectiveness
Corrective Action Actions Taken of Actions Taken
Source: Regulatory Compliance Associates Inc., 2019
17
Schniepp says companies can fail at any step in that process and that few excel at them all.
The biggest issue she sees with manufacturing companies is a failure to truly get at the root
cause of an issue. Often, an investigation will find a likely cause, the company will make a
quick fix and then they’ll be surprised a few weeks later when the same problem comes back, or
when a similar problem affects another manufacturing line. This is usually because the investi-
gation stopped too soon and didn’t go deep enough to find the root cause.
The first step, Schniepp says, is to understand the difference between a cause and a root
cause. A cause is the most direct reason that something failed. A root cause is the fundamental
issue or chain of events that led to a failure. Sometimes, of course, a failure is a one-time issue
and a brief investigation can find and fix the cause. “Not everything is a CAPA,” Schniepp says.
“You don’t need to complicate it if it’s not complicated. Sometimes there’s a simple issue that
occurred, with a simple solution. Keep monitoring it and see if the issue’s been fixed.”
But often, a failure is systemic, repeats or is part of a larger pattern of issues that requires a
more in-depth investigation to get at the root cause. In these situations, an initial investigation
may turn up a cause, but until you get at the underlying root cause, the issue will keep occurring
and may even worsen.
One problem Schniepp sees frequently is companies that rely so heavily on investigative
tools that they lose the element of critical thinking. “There are lots of tools for figuring out what
went wrong in a process, and a lot of those tools are great,” Schniepp says. “But no tool is a re-
placement for critical thinking. Yes, use the tools, but you also have to think things through, and
keep an open mind when investigating to the root cause. It may not be what you expected.”
It’s crucial that investigators interview operators involved in the process in question.
They’re often a wealth of knowledge about what works and what doesn’t in a given process,
and sometimes they’ve never told anyone what they know simply because no one has ever
asked. It’s also important that these interviews take place in a timely manner, ideally within a
week or so of identifying the problem. Over time, memories get fuzzy, and an operator may
have a harder time recalling what went wrong with a process on a given day or whether he or
she did anything unusual.
A risk assessment should be part of the investigation. Not all deviations are created equally;
some will have a greater impact on the ultimate health or safety of the patient or end user of a
drug or device. There are one-time deviations that could have a huge impact, Schniepp says,
and there are repeated deviations that may only have a minor impact. Understanding the risk
will help you plan an investigation and plan what you’ll do about the manufacturing process
while an investigation is carried out. That might include disposing of product batches or halt-
ing a process until identifying the root cause and putting corrective and preventive actions into
place.
Schniepp offers the following example from her own work history: She’d been brought on
as quality manager for a company performing contract manufacturing for a drugmaker. One of
the company’s suppliers had stopped making the active pharmaceutical ingredient (API) for
one of their products, and the company had not been able to find a replacement supplier. There
was enough supply of the API for three final batches of the drug. During the manufacturing run
for the second batch, a floor manager halted the process due to a deviation and said they would
need to throw out the product from that batch. “But I said, ‘no, you won’t be throwing that
18
batch away, and in fact we can cease and desist testing for that particular deviation,’” Schniepp
recalls.
The deviation in question, Schniepp says, had to do with splashback that caused a small
amount of liquid to land between the stopper and the vial. “Just this tiny white spot, the size
of a pinhead,” she says. It was a cosmetic issue, rather than one that would affect the safety or
efficacy of the medication. And the drug in question was one that helped patients manage the
symptoms of leukemia. “I know it’s a deviation, and I know how to fix it—for the next run, we
had to adjust a filter head,” Schniepp says. “But there’s absolutely no reason to trash this prod-
uct that patients badly need and which will work just as well.”
She says this is just one example of what can happen if people blindly follow protocols
rather than engaging in critical, situational thinking. In another situation, perhaps the cosmetic
deviation would have warranted throwing out a batch of product. But in this case, the scale was
clearly tipped in favor of keeping the product. Schniepp notes that the FDA agreed with the
assessment; she reported the issue to regulators, who said that the company should indeed ship
the product.
So, a quality investigation should make use of risk management tools and should aim to dig
deep enough to find a root cause. The industry standard for an investigation is a 90-day process,
but Schniepp emphasizes that this comes from experience, not any regulatory requirement.
Some investigations will be simpler and shorter, and some may need to stretch beyond 90 days.
“What you really want to look for is whether [an investigation] is getting to the root cause,”
she says. “Are they using their tools well? Is there a transparent flow of information? Did they
interview operators right away, or was there a lag? If it takes two weeks to jump on a problem
or a deviation, that could be an issue. You may not get all the information you need to solve the
problem.”
19
Risk Management
Any manufacturing facility investing in a culture of quality must lean on risk management
principles in its ongoing auditing and monitoring plans. From a regulatory perspective, the
current risk management guidelines for drugmakers come from the International Council on
Harmonization’s (ICH), Q9 – Quality Risk Management. Those guidelines were published in
2005 and are scheduled for an update in spring or summer 2020. Schniepp says the upgrade is
overdue, as there have long been questions in the industry about how best to apply the current
guidance.
Currently, the training material published on the ICH website is meant to support imple-
mentation of ICH Q9, but that material could be improved, Schniepp says. She expects this will
be the focus of the 2020 update—not a rewrite of the guidance itself, but a clarification on how
it should be implemented by manufacturers. That clarification would likely come through an
addendum or partnering document. “You’re not going to see a total rewrite of ICH Q9. You’re
going to see an enhanced version,” she predicts.
Principles of Risk Assessment
A risk assessment should focus on the impact of the issue being studied on the health and
safety of the patient or end user of the drug or device. That’s both the starting place and the
endpoint: how will a given issue affect the patient or user?
The scope of a risk assessment should include:
º Product in the field;
º Multiple lots;
º Multiple locations; and
º The frequency of occurrence of an issue, as identified via hazard analysis tools.
ICH Q9 sets out to offer what it calls a “systematic approach to quality risk management”
that can then lead to better, more informed decisionmaking. Good risk-based decisionmaking
is both transparent and comprehensive. “And really, the purpose of ICH Q9 was to make better
and more informed decisions about things going on in the manufacturing process,” Schniepp
says.
But there are significant challenges to implementing the quality risk assessment measures
called for in ICH Q9. Schniepp points to a 2016 article in Pharmaceutical Engineering, by
James Vesper and Kevin O’Donnell. The authors identified a number of potential issues, includ-
ing:
Using formal quality risk management (QRM) tools in situations where less formal tools
would suffice;
º Using QRM to justify an action, rather than to assess risk;
º Failing to create quality culture, limiting how QRM is applied;
º Using a specific risk-assessment tool as the QRM process;
20
º Using attribute scales that are neither specific nor appropriate;

º Failing to acknowledge when uncertainty (lack of knowledge) is present;
º Neglecting to keep risk assessments current; and
º Using QRM to confirm a hypothesis or rationalize a noncompliance situation.
Schniepp says the danger of these QRM tools is that, without an overall culture of quality,
they can be misapplied or emphasized over or in place of critical thinking. “You can have the
best tool in the world, but without thinking, it won’t get you anywhere you want to go,” she
says. “You’ve got to have that human element and a focus on quality to make sure these tools
are being applied in a sensible, constructive way.”
Schniepp says there is no one, single quality risk management tool that is right for everyone.
“I don’t think anyone has come up with the tool yet, the end-all, be-all tool,” she says. “What
I would say is pick one and incorporate it. But don’t rely on [that tool] to be your quality risk
management program. Use it as an enhancement and use it only as a tool. Quality risk manage-
ment requires you to think, and there are lots of good tools out there, if you use them properly.”
Implementing a Quality Risk Management Plan
A risk management plan must be robust and take in just about every aspect of a manufac-
turer’s operations. Take supplier audits, for example. Using risk management principles, a
company would begin its supplier audits by auditing the suppliers it leans on most heavily to
ensure these suppliers are reliable. Then the company would move on to audit its other suppliers
and put plans into place to monitor them. In constructing those monitoring plans, the company
would consider the risks of various issues that could crop up at the supplier level and specifical-
ly how those issues could affect the health and safety of the patient or end user. “That’s what it
all comes back to,” Schniepp emphasizes. “What are the risks to the patient? What risks would
be minor, and what risks would be catastrophic? And prioritizing that way.”
Schniepp points to the FDA’s own risk management principles. When the agency started to
collect data from companies on specific metrics, it wanted to head off potential drug shortages
by monitoring drugmakers the same way a drugmaker might monitor its suppliers. The FDA
identified key metrics that would show a company might be in trouble and also attempted to
keep tabs on what the effects would be on drug supply if one or another company had to stop
production temporarily or couldn’t ship product. “The FDA’s attitude was basically, ‘o.k., if we
can collect data on these important metrics, then we could see, hey, this company looks like it
might be headed for trouble,’” Schniepp says. “‘And if that company is critical for keeping a
drug available, then we, the FDA, can step in and try to help them.’”
Manufacturers can take a similar attitude to their suppliers, she says. Figure out which sup-
pliers are most critical and expend the most resources monitoring them. Ensure you have back-
up plans in place should you lose the supplier for one reason or another.
Take the ongoing coronavirus pandemic. On the one hand, no one could have foreseen the ef-
fect COVID-19 would have both in the U.S. and globally. On the other hand, Schniepp says she
is already seeing the effects of companies’ risk management plans, both the good and the bad.
Some drugmakers, for instance, rely on a lot of supply from China, and when China essentially
21
shut down manufacturing, not everyone had a plan in place for how to mitigate that. As part of
a risk management plan, she suggests looking at where your suppliers are located and thinking
through your options should there be a major disruption in that country or region.
A quality risk management plan must also involve people at all levels of a company. It’s im-
portant to get management buy-in, but it’s equally important that line operators and employees
“in the trenches” have their voices heard. For example, during the 2008 financial crisis, some fi-
nancial institutions were quicker than others to stop offering so-called subprime mortgages—in
large part, Schniepp says, because of reports from field offices and loan officers, as well as em-
ployees who were looking at data and seeing that some of the numbers weren’t making sense.
Every department in an organization should come up with its own risk management plan,
and then these plans should be combined by the quality department. The plan should include
metrics and data measurement points and specify who’s tasked with monitoring them, as well
as what happens in the event of deviations or bad data trends. Each department collects its
own data, and then sends it to management either monthly or quarterly. Ideally, Schniepp says,
what’s sent to management includes raw data, “undiluted,” rather than just whatever takeaways
individual departments have gleaned from that data, which could be influenced by people’s
biases and expectations.
“We have a lot of data we all collect and report, but getting a metric that’s unbiased is really,
really tough,” Schniepp says.
It’s also important that these metrics don’t have unintended consequences. As an example,
she offers a story from her own experience working for a company that did contract manufac-
turing for several drugmakers. The company maintained a web portal for its customers, and one
of its desired metrics was to post batch information for every batch of product within 30 days
of its manufacture. As an incentive to meet this metric, the company’s management offered
rewards in the form of pizza parties. “Management said, ‘Hey, if you can hit this metric, we’ll
throw you a party,’ which sounds benign, at least on the surface,” she says.
What happened, however, was that the incentive caused the company’s employees to do
whatever they could to hit that metric. Investigations into deviations were closed early, for
instance, so they could post the batch information within 30 days. “That was actually how I fig-
ured out what was going on,” Schniepp says. “Because the numbers were just too good. No one
can post batch information within 30 days for every single batch.”
The lesson, she says, is that data metrics can’t come with “carrots” that could incentivize the
wrong behavior.
“At the end of the day, again, it’s really just the importance of that human element, of criti-
cal thinking,” she says. “The tools and the metrics are useful, but only if they’re used correctly.”
22
Appendices
A. WHO Annex 5: Guidance on Good Data and Record Management Practices
B. MHRA GxP Data Integrity Guidance and Definitions
C. FDA Data Integrity and Compliance with CGMP Guidance
D. ICH Q10: Pharmaceutical Quality System
E. GHTF Quality Management System — Medical Devices — Guidance on
Corrective Action and Preventive Action and Related QMS Processes
23
Appendices A: WHO Annex 5: Guidance on Good
Data and Record Management Practices
Annex 5
Guidance on good data and record management practices
Background
During an informal consultation on inspection, good manufacturing practices
and risk management guidance in medicines’ manufacturing held by the
World Health Organization (WHO) in Geneva in April 2014, a proposal for
new guidance on good data management was discussed and its development
recommended. The participants included national inspectors and specialists
in the various agenda topics, as well as staff of the Prequalification Team
(PQT)–Inspections.
The WHO Expert Committee on Specifications for Pharmaceutical
Preparations received feedback from this informal consultation during its
forty-ninth meeting in October 2014. A concept paper was received from PQT–
Inspections describing the proposed structure of a new guidance document,
which was discussed in detail. The concept paper consolidated existing normative
principles and gave some illustrative examples of their implementation. In
the Appendix to the concept paper, extracts from existing good practices and
guidance documents were combined to illustrate the current relevant guidance
on assuring the reliability of data and related GXP (good (anything) practice)
matters. In view of the increasing number of observations made during
inspections that relate to data management practices, the Committee endorsed
the proposal.
Following this endorsement, a draft document was prepared by
members of PQT–Inspection and a drafting group, including national inspectors.
This draft was discussed at a consultation on data management, bioequivalence,
good manufacturing practices and medicines’ inspection held from 29 June to
1 July 2015.
A revised draft document was subsequently prepared by the authors in
collaboration with the drafting group, based on the feedback received during
this consultation, and the subsequent WHO workshop on data management.
Collaboration is being sought with other organizations towards future
convergence in this area.
165
WHO Expert Committee on Specifications for Pharmaceutical Preparations Fiftieth report
1. Introduction 167
2. Aims and objectives of this guidance 169
3. Glossary 169
4. Principles 173
5. Quality risk management to ensure good data management 177
6. Management governance and quality audits 178
7. Contracted organizations, suppliers and service providers 180
8. Training in good data and record management 182
9. Good documentation practices 182
10. Designing and validating systems to assure data quality
and reliability 183
11. Managing data and records throughout the data life cycle 186
12. Addressing data reliability issues 189
References and further reading 190
Appendix 1 Expectations and examples of special risk management considerations
for the implementation of ALCOA (-plus) principles in paper-based and
electronic systems 192
WHO Technical Report Series No. 996, 2016
166
Annex 5
1. Introduction
1.1 Medicines regulatory systems worldwide have always depended upon the
knowledge of organizations that develop, manufacture and package, test,
distribute and monitor pharmaceutical products. Implicit in the assessment
and review process is trust between the regulator and the regulated that
the information submitted in dossiers and used in day-to-day decision-
making is comprehensive, complete and reliable. The data on which
these decisions are based should therefore be complete as well as being
attributable, legible, contemporaneous, original and accurate, commonly
referred to as “ALCOA”.
1.2 These basic ALCOA principles and the related good practice expectations
that assure data reliability are not new and much high- and mid-level
normative guidance already exists. However, in recent years, the number of
observations made regarding good data and record management practices
(GDRP) during inspections of good manufacturing practice (GMP) (1),
good clinical practice (GCP) and good laboratory practice (GLP) has been
increasing. The reasons for the increasing concern of health authorities
regarding data reliability are undoubtedly multifactorial and include
increased regulatory awareness and concern regarding gaps between
industry choices and appropriate and modern control strategies.
1.3 Contributing factors include failures by organizations to apply robust
systems that inhibit data risks, to improve the detection of situations where
data reliability may be compromised, and/or to investigate and address
root causes when failures do arise. For example, organizations subject to
medical product good practice requirements have been using validated
computerized systems for many decades but many fail to adequately review
and manage original electronic records and instead often only review and
manage incomplete and/or inappropriate printouts. These observations
highlight the need for industry to modernize control strategies and apply
modern quality risk management (QRM) and sound scientific principles to
current business models (such as outsourcing and globalization) as well as
technologies currently in use (such as computerized systems).
1.4 Examples of controls that may require development and strengthening to
ensure good data management strategies include, but are not limited to:
■■ a QRM approach that effectively assures patient safety and product
quality and validity of data by ensuring that management aligns
expectations with actual process capabilities. Management should
take responsibility for good data management by first setting realistic
and achievable expectations for the true and current capabilities of
167
a process, a method, an environment, personnel, or technologies,

among others;
■■ monitoring of processes and allocation of the necessary resources by
management to ensure and enhance infrastructure, as required (for
example, to continuously improve processes and methods, to ensure
adequate design and maintenance of buildings, facilities, equipment
and systems; to ensure adequate reliable power and water supplies;
to provide necessary training for personnel; and to allocate the
necessary resources to the oversight of contract sites and suppliers
to ensure adequate quality standards are met). Active engagement of
management in this manner remediates and reduces pressures and
possible sources of error that may increase data integrity risks;
■■ adoption of a quality culture within the company that encourages
personnel to be transparent about failures so that management
has an accurate understanding of risks and can then provide the
necessary resources to achieve expectations and meet data quality
standards: a reporting mechanism independent of management
hierarchy should be provided for;
■■ mapping of data processes and application of modern QRM and
sound scientific principles throughout the data life cycle;
■■ ensuring that all site personnel are kept up to date about the
application of good documentation practices (GDocP) to ensure
that the GXP principles of ALCOA are understood and applied
to electronic data in the same manner that has historically been
applied to paper records;
■■ implementation and confirmation during validation of computerized
systems and subsequent change control, that all necessary controls
for GDocP for electronic data are in place and that the probability of
the occurrence of errors in the data is minimized;
■■ training of personnel who use computerized systems and review

electronic data in basic understanding of how computerized systems
work and how to efficiently review the electronic data, which
includes metadata and audit trails;
■■ definition and management of appropriate roles and responsibilities
for quality agreements and contracts entered into by contract
givers and contract acceptors, including the need for risk-based
monitoring of data generated and managed by the contract acceptor
on behalf of the contract giver;
■■ modernization of quality assurance inspection techniques and
gathering of quality metrics to efficiently and effectively identify
risks and opportunities to improve data processes.
168
Annex 5
2. Aims and objectives of this guidance

2.1 This guidance consolidates existing normative principles and gives
detailed illustrative implementation guidance to bridge the gaps in
current guidance. Additionally, it gives explanations as to what these high-
level requirements mean in practice and what should be demonstrably
implemented to achieve compliance.
2.2 These guidelines highlight, and in some instances clarify, the application
of data management procedures. The focus is on those principles that are
implicit in existing WHO guidelines and that if not robustly implemented
can impact on data reliability and completeness and undermine the
robustness of decision-making based upon those data. Illustrative
examples are provided as to how these principles may be applied to
current technologies and business models. These guidelines do not define
all expected controls for assuring data reliability and this guidance should
be considered in conjunction with existing WHO guidelines and other
related international references.
2.3 This guidance is of an evolutionary, illustrative nature and will therefore be
subject to periodic review based upon experience with its implementation
and usefulness, as well as the feedback provided by the stakeholders,
including national regulatory authorities (NRAs).
3. Glossary
The definitions given below apply to the terms used in these guidelines. They
may have different meanings in other contexts.
ALCOA. A commonly used acronym for “attributable, legible,
contemporaneous, original and accurate”.
ALCOA-plus. A commonly used acronym for “attributable, legible,
contemporaneous, original and accurate”, which puts additional emphasis on
the attributes of being complete, consistent, enduring and available – implicit
basic ALCOA principles.
archival. Archiving is the process of protecting records from the
possibility of being further altered or deleted, and storing these records
under the control of independent data management personnel throughout
the required retention period. Archived records should include, for example,
associated metadata and electronic signatures.
archivist. An independent individual designated in good laboratory
practice (GLP) who has been authorized by management to be responsible
for the management of the archive, i.e. for the operations and procedures for
archiving. GLP requires a designated archivist (i.e. an individual); however, in
169
other GXPs the roles and responsibilities of the archivist are normally fulfilled
by several designated personnel or groups of personnel (e.g. both quality
assurance document control personnel and information technology (IT) system
administrators) without there being one single person assigned responsibility for
control as is required in GLP.
It is recognized that in certain circumstances it may be necessary for the
archivist to delegate specific archiving tasks, for example, the management of
electronic data, to specific IT personnel. Tasks, duties and responsibilities should
be specified and detailed in standard operating procedures. The responsibilities
of the archivist and the staff to whom archival tasks are delegated include –
for both paper and electronic data – ensuring that access to the archive is
controlled, ensuring that the orderly storage and retrieval of records and
materials is facilitated by a system of indexing, and ensuring that movement
of records and materials into and out of the archives is properly controlled and
documented. These procedures and records should be periodically reviewed by
an independent auditor.
audit trail. The audit trail is a form of metadata that contains information
associated with actions that relate to the creation, modification or deletion of
GXP records. An audit trail provides for secure recording of life-cycle details
such as creation, additions, deletions or alterations of information in a record,
either paper or electronic, without obscuring or overwriting the original record.
An audit trail facilitates the reconstruction of the history of such events relating
to the record regardless of its medium, including the “who, what, when and why”
of the action.
For example, in a paper record, an audit trail of a change would be
documented via a single-line cross-out that allows the original entry to remain
legible and documents the initials of the person making the change, the date
of the change and the reason for the change, as required to substantiate and
justify the change. In electronic records, secure, computer-generated, time-
stamped audit trails should allow for reconstruction of the course of events
relating to the creation, modification and deletion of electronic data. Computer-
generated audit trails should retain the original entry and document the user
identification, the time/date stamp of the action, as well as the reason for the
change, as required to substantiate and justify the action. Computer-generated
audit trails may include discrete event logs, history files, database queries or
reports or other mechanisms that display events related to the computerized
system, specific electronic records or specific data contained within the record.
backup. A backup means a copy of one or more electronic files created
as an alternative in case the original data or system are lost or become unusable
(for example, in the event of a system crash or corruption of a disk). It is
important to note that backup differs from archival in that back-up copies of
electronic records are typically only temporarily stored for the purposes of
170
Annex 5
disaster recovery and may be periodically overwritten. Such temporary back-up

copies should not be relied upon as an archival mechanism.
computerized system. A computerized system collectively controls the
performance of one or more automated processes and/or functions. It includes
computer hardware, software, peripheral devices, networks and documentation,
e.g. manuals and standard operating procedures, as well as the personnel
interfacing with the hardware and software, e.g. users and information technology
support personnel.
control strategy. A planned set of controls, derived from current
protocol, test article or product and process understanding, which assures
protocol compliance, process performance, product quality and data reliability,
as applicable. The controls should include appropriate parameters and quality
attributes related to study subjects, test systems, product materials and
components, technologies and equipment, facilities, operating conditions,
specifications and the associated methods and frequency of monitoring
and control.
corrective and preventive action (CAPA, also sometimes called
corrective action/preventive action) refers to the actions taken to improve an
organization's processes and to eliminate causes of non-conformities or other
undesirable situations. CAPA is a concept common across the GXPs (good
laboratory practices, good clinical practices and good manufacturing practices),
and numerous International Organization for Standardization business standards.
The process focuses on the systematic investigation of the root causes of
identified problems or identified risks in an attempt to prevent their recurrence
(for corrective action) or to prevent occurrence (for preventive action).
data. Data means all original records and true copies of original records,
including source data and metadata and all subsequent transformations and
reports of these data, which are generated or recorded at the time of the GXP
activity and allow full and complete reconstruction and evaluation of the GXP
activity. Data should be accurately recorded by permanent means at the time
of the activity. Data may be contained in paper records (such as worksheets
and logbooks), electronic records and audit trails, photographs, microfilm
or microfiche, audio- or video-files or any other media whereby information
related to GXP activities is recorded.
data governance. The totality of arrangements to ensure that data,
irrespective of the format in which they are generated, are recorded, processed,
retained and used to ensure a complete, consistent and accurate record
throughout the data life cycle.
data integrity. Data integrity is the degree to which data are complete,
consistent, accurate, trustworthy and reliable and that these characteristics of the
data are maintained throughout the data life cycle. The data should be collected
and maintained in a secure manner, such that they are attributable, legible,
171
contemporaneously recorded, original or a true copy and accurate. Assuring data

integrity requires appropriate quality and risk management systems, including
adherence to sound scientific principles and good documentation practices.
data life cycle. All phases of the process by which data are created,
recorded, processed, reviewed, analysed and reported, transferred, stored and
retrieved and monitored until retirement and disposal. There should be a
planned approach to assessing, monitoring and managing the data and the risks
to those data in a manner commensurate with potential impact on patient
safety, product quality and/or the reliability of the decisions made throughout
all phases of the data life cycle.
dynamic record format. Records in dynamic format, such as electronic
records, that allow for an interactive relationship between the user and the
record content. For example, electronic records in database formats allow the
user to track, trend and query data; chromatography records maintained as
electronic records allow the user (with proper access permissions) to reprocess
the data and expand the baseline to view the integration more clearly.
fully-electronic approach. This term refers to use of a computerized
system in which the original electronic records are electronically signed.
good data and record management practices. The totality of organized
measures that should be in place to collectively and individually ensure
that data and records are secure, attributable, legible, traceable, permanent,
contemporaneously recorded, original and accurate and that if not robustly
implemented can impact on data reliability and completeness and undermine
the robustness of decision-making based upon those data records.
good documentation practices. In the context of these guidelines, good
documentation practices are those measures that collectively and individually
ensure documentation, whether paper or electronic, is secure, attributable, legible,
traceable, permanent, contemporaneously recorded, original and accurate.
GXP. Acronym for the group of good practice guides governing the
preclinical, clinical, manufacturing, testing, storage, distribution and post-market

activities for regulated pharmaceuticals, biologicals and medical devices, such as
good laboratory practices, good clinical practices, good manufacturing practices,
good pharmacovigilance practices and good distribution practices.
hybrid approach. This refers to the use of a computerized system in
which there is a combination of original electronic records and paper records
that comprise the total record set that should be reviewed and retained. An
example of a hybrid approach is where laboratory analysts use computerized
instrument systems that create original electronic records and then print a
summary of the results. The hybrid approach requires a secure link between all
record types, including paper and electronic, throughout the records retention
period. Where hybrid approaches are used, appropriate controls for electronic
172
Annex 5
documents, such as templates, forms and master documents, that may be

printed, should be available.
metadata. Metadata are data about data that provide the contextual
information required to understand those data. These include structural
and descriptive metadata. Such data describe the structure, data elements,
interrelationships and other characteristics of data. They also permit data to
be attributable to an individual. Metadata necessary to evaluate the meaning
of data should be securely linked to the data and subject to adequate review.
For example, in weighing, the number 8 is meaningless without metadata, i.e. the
unit, mg. Other examples of metadata include the time/date stamp of an activity,
the operator identification (ID) of the person who performed an activity, the
instrument ID used, processing parameters, sequence files, audit trails and other
data required to understand data and reconstruct activities.
quality metrics. Quality metrics are objective measures used by
management and other interested parties to monitor the overall state of quality
of a GXP organization, activity or process or study conduct, as applicable. They
include measures to assess the effective functioning of quality system controls
and of the performance, quality and safety of medicinal products and reliability
of data.
quality risk management. A systematic process for the assessment,
control, communication and review of risks to the quality of the pharmaceutical
product throughout the product life cycle.
senior management. Person(s) who direct and control a company or site
at the highest levels with the authority and responsibility to mobilize resources
within the company or site.
static record format. A static record format, such as a paper or pdf
record, is one that is fixed and allows little or no interaction between the user
and the record content. For example, once printed or converted to static pdfs,
chromatography records lose the capability of being reprocessed or enabling
more detailed viewing of baselines.
true copy. A true copy is a copy of an original recording of data that
has been verified and certified to confirm it is an exact and complete copy that
preserves the entire content and meaning of the original record, including, in
the case of electronic data, all essential metadata and the original record format
as appropriate.
4. Principles
4.1 GDRP are critical elements of the pharmaceutical quality system and a
systematic approach should be implemented to provide a high level of
assurance that throughout the product life cycle, all GXP records and data
are complete and reliable.
173
4.2 The data governance programme should include policies and governance
procedures that address the general principles listed below for a good data
management programme. These principles are clarified with additional
detail in the sections below.
4.3 Applicability to both paper and electronic data. The requirements for
GDRP that assure robust control of data validity apply equally to paper
and electronic data. Organizations subject to GXP should be fully aware
that reverting from automated or computerized to manual or paper-based
systems does not in itself remove the need for robust management controls.
4.4 Applicability to contract givers and contract acceptors. The principles of
these guidelines apply to contract givers and contract acceptors. Contract
givers are ultimately responsible for the robustness of all decisions made on
the basis of GXP data, including those made on the basis of data provided
to them by contract acceptors. Contract givers should therefore perform
risk-based, due diligence to assure themselves that contract acceptors have
in place appropriate programmes to ensure the veracity, completeness and
reliability of the data provided.
4.5 Good documentation practices. To achieve robust decisions, the
supporting data set needs to be reliable and complete. GDocP should be
followed in order to ensure all records, both paper and electronic, allow
the full reconstruction and traceability of GXP activities.
4.6 Management governance. To establish a robust and sustainable good data
management system it is important that senior management ensure that
appropriate data management governance programmes are in place (for
details see Section 6).
Elements of effective management governance should include:
■■ application of modern QRM principles and good data management

principles that assure the validity, completeness and reliability of data;
■■ application of appropriate quality metrics;
■■ assurance that personnel are not subject to commercial, political,
financial and other organizational pressures or incentives that may
adversely affect the quality and integrity of their work;
■■ allocation of adequate human and technical resources such that the
workload, work hours and pressures on those responsible for data
generation and record keeping do not increase errors;
■■ ensure staff are aware of the importance of their role in ensuring
data integrity and the relationship of these activities to assuring
product quality and protecting patient safety.
174
Annex 5
4.7 Quality culture. Management, with the support of the quality unit, should
establish and maintain a working environment that minimizes the risk
of non-compliant records and erroneous records and data. An essential
element of the quality culture is the transparent and open reporting
of deviations, errors, omissions and aberrant results at all levels of the
organization, irrespective of hierarchy. Steps should be taken to prevent,
and to detect and correct weaknesses in systems and procedures that may
lead to data errors so as to continually improve the robustness of scientific
decision-making within the organization. Senior management should
actively discourage any management practices that might reasonably be
expected to inhibit the active and complete reporting of such issues, for
example, hierarchical constraints and blame cultures.
4.8 Quality risk management and sound scientific principles. Robust decision-
making requires appropriate quality and risk management systems, and
adherence to sound scientific and statistical principles, which must be
based upon reliable data. For example, the scientific principle of being an
objective, unbiased observer regarding the outcome of a sample analysis
requires that suspect results be investigated and rejected from the reported
results only if they are clearly attributable to an identified cause. Adhering
to good data and record-keeping principles requires that any rejected
results be recorded, together with a documented justification for their
rejection, and that this documentation is subject to review and retention.
4.9 Data life cycle management. Continual improvement of products to
ensure and enhance their safety, efficacy and quality requires a data
governance approach to ensure management of data integrity risks
throughout all phases of the process by which data are created, recorded,
processed, transmitted, reviewed, reported, archived and retrieved and
this management process is subject to regular review. To ensure that the
organization, assimilation and analysis of data into information facilitates
evidence-based and reliable decision-making, data governance should
address data ownership and accountability for data process(es) and risk
management of the data life cycle.
4.10 To ensure that the organization, assimilation and analysis of data into a
format or structure that facilitates evidence-based and reliable decision-
making, data governance should address data ownership and accountability
for data process(es) and risk management of the data life cycle.
4.11 Design of record-keeping methodologies and systems. Record-keeping
methodologies and systems, whether paper or electronic, should be
designed in a way that encourages compliance with the principles of
data integrity.
175
4.12 Examples include, but are not restricted to:

■■ restricting the ability to change any clock used for recording timed
events, for example, system clocks in electronic systems and
process instrumentation;
■■ ensuring controlled forms used for recording GXP data (e.g. paper
batch records, paper case report forms and laboratory worksheets)
are accessible at the locations where an activity is taking place, at the
time that the activity is taking place, so that ad hoc data recording
and later transcription is not necessary;
■■ controlling the issuance of blank paper templates for data recording
of GXP activities so that all printed forms can be reconciled and
accounted for;
■■ restricting user access rights to automated systems to prevent (or
audit trail) data amendments;
■■ ensuring automated data capture or printers are attached and
connected to equipment, such as balances, to ensure independent
and timely recording of the data;
■■ ensuring proximity of printers to sites of relevant activities;
■■ ensuring ease of access to locations of sampling points (e.g. sampling
points for water systems) to allow easy and efficient performance of
sampling by the operators and therefore minimizing the temptation
to take shortcuts or falsify samples;
■■ ensuring access to original electronic data for staff performing data
checking activities.
4.13 Data and record media should be durable. For paper records, the ink
should be indelible. Temperature-sensitive or photosensitive inks and
other erasable inks should not be used. Paper should also not be
temperature-sensitive, photosensitive or easily oxidizable. If this is not
feasible or limited (as may be the case in printouts from legacy printers
of balance and other instruments in quality control laboratories), then
true or certified copies should be available until this equipment is retired
or replaced.
4.14 Maintenance of record-keeping systems. The systems implemented and
maintained for both paper and electronic record-keeping should take
account of scientific and technical progress. Systems, procedures and
methodology used to record and store data should be periodically reviewed
for effectiveness and updated as necessary.
176
Annex 5
5. Quality risk management to ensure

good data management
5.1 All organizations performing work subject to GXP are required by
applicable existing WHO guidance to establish, implement and maintain
an appropriate quality management system, the elements of which should
be documented in their prescribed format, such as a quality manual or
other appropriate documentation. The quality manual, or equivalent
documentation, should include a quality policy statement of management’s
commitment to an effective quality management system and to good
professional practice. These policies should include a code of ethics and
code of proper conduct to assure the reliability and completeness of data,
including mechanisms for staff to report any quality and compliance
questions or concerns to management.
5.2 Within the quality management system, the organization should establish
the appropriate infrastructure, organizational structure, written policies
and procedures, processes and systems to both prevent and detect
situations that may impact on data integrity and, in turn, the risk-based
and scientific robustness of decisions based upon those data.
5.3 QRM is an essential component of an effective data and record validity
programme. The effort and resources assigned to data and record
management should be commensurate with the risk to product quality. The
risk-based approach to record and data management should ensure that
adequate resources are allocated and that control strategies for the assurance
of the integrity of GXP data are commensurate with their potential impact
on product quality and patient safety and related decision-making.
5.4 Strategies that promote good practices and prevent record and data
integrity issues from occurring are preferred and are likely to be the most
effective and cost-effective. For example, access controls that allow only
people with the appropriate authorization to alter a master processing
formula will reduce the probability of invalid and aberrant data being
generated. Such preventive measures, when effectively implemented, also
reduce the amount of monitoring required to detect uncontrolled change.
5.5 Record and data integrity risks should be assessed, mitigated,
communicated and reviewed throughout the data life cycle in accordance
with the principles of QRM. Examples of approaches that may enhance
data reliability are given in these guidelines but should be viewed as
recommendations. Other approaches may be justified and shown to be
equally effective in achieving satisfactory control of risk. Organizations
177
should therefore design appropriate tools and strategies for the management
of data integrity risks based upon their own GXP activities, technologies
and processes.
5.6 A data management programme developed and implemented upon the
basis of sound QRM principles is expected to leverage existing technologies
to their full potential. This in turn will streamline data processes in a
manner that not only improves data management but also the business
process efficiency and effectiveness, thereby reducing costs and facilitating
continual improvement.
6. Management governance and quality audits

6.1 Assuring robust data integrity begins with management, which has
the overall responsibility for the technical operations and provision
of resources to ensure the required quality of GXP operations. Senior
management has the ultimate responsibility for ensuring that an effective
quality system is in place to achieve the quality objectives, and that
staff roles, responsibilities and authorities, including those required for
effective data governance programmes, are defined, communicated and
implemented throughout the organization. Leadership is essential to
establish and maintain a company-wide commitment to data reliability as
an essential element of the quality system.
6.2 The building blocks of behaviours, procedural/policy considerations
and basic technical controls together form the foundation of good data
governance, upon which future revisions can be built. For example, a
good data governance programme requires the necessary management
arrangements to ensure personnel are not subject to commercial,
political, financial and other pressures or conflicts of interest that may

adversely affect the quality of their work and integrity of their data.
Management should also make staff aware of the relevance of data
integrity and the importance of their role in protecting the safety of
patients and the reputation of their organization for quality products
and services.
6.3 Management should create a work environment in which staff are
encouraged to communicate failures and mistakes, including data reliability
issues, so that corrective and preventive actions can be taken and the
quality of an organization’s products and services enhanced. This includes
ensuring adequate information flow between staff at all levels. Senior
management should actively discourage any management practices that
178
Annex 5
might reasonably be expected to inhibit the active and complete reporting

of such issues, for example, hierarchical constraints and blame cultures.
6.4 Management reviews and regular reporting of quality metrics facilitate
meeting these objectives. This requires designation of a quality manager
who has direct access to the highest level of management and can directly
communicate risks, so that senior management is made aware of any issues
and can allocate resources to address them. To fulfil this role the quality
unit should conduct and report to management formal, documented risk
reviews of the key performance indicators of the quality management
system. These should include metrics related to data integrity that will help
identify opportunities for improvement. For example:
■■ tracking and trending of invalid and aberrant data may reveal
unforeseen variability in processes and procedures previously
believed to be robust, opportunities to enhance analytical procedures
and their validation, validation of processes, training of personnel
or sourcing of raw materials and components;
■■ adequate review of audit trails, including those reviewed as part of
key decision-making steps (e.g. GMP batch release, issuance of a GLP
study report or approval of case report forms), may reveal incorrect
processing of data, help prevent incorrect results from being reported
and identify the need for additional training of personnel;
■■ routine audits and/or self-inspections of computerized systems may
reveal gaps in security controls that inadvertently allow personnel
to access and potentially alter time/date stamps. Such findings
help raise awareness among management of the need to allocate
resources to improve validation controls for computerized systems;
■■ monitoring of contract acceptors and tracking and trending of
associated quality metrics for these sites help to identify risks that
may indicate the need for more active engagement and allocation
of additional resources by the contract giver to ensure quality
standards are met.
6.5 Quality audits of suppliers, self-inspections and risk reviews should
identify and inform management of opportunities to improve foundational
systems and processes that have an impact on data reliability. Allocation
of resources by management to these improvements of systems and
processes may efficiently reduce data integrity risks. For example,
identifying and addressing technical difficulties with the equipment used
to perform multiple GXP operations may greatly improve the reliability
179
of data for all of these operations. Another example relates to identifying

conflicts of interests affecting security. Allocating independent technical
support personnel to perform system administration for computerized
systems, including managing security, backup and archival, reduces
potential conflicts of interest and may greatly streamline and improve data
management efficiency.
6.6 All GXP records held by the GXP organization are subject to inspection
by the responsible health authorities. This includes original electronic data
and metadata, such as audit trails maintained in computerized systems.
Management of both contract givers and contract acceptors should
ensure that adequate resources are available and that procedures for
computerized systems are available for inspection. System administrator
personnel should be available to readily retrieve requested records and
facilitate inspections.
7. Contracted organizations, suppliers

and service providers
7.1 The increasing outsourcing of GXP work to contracted organizations, e.g.
contract research organizations, suppliers and other service providers,
emphasizes the need to establish and robustly maintain defined roles
and responsibilities to assure complete and accurate data and records
throughout these relationships. The responsibilities of the contract giver
and acceptor, should comprehensively address the processes of both
parties that should be followed to ensure data integrity. These details
should be included in the contract described in the WHO GXPs relevant
to the outsourced work performed or the services provided.
7.2 The organization that outsources work has the responsibility for
the integrity of all results reported, including those furnished by any
subcontracting organization or service provider. These responsibilities
extend to any providers of relevant computing services. When outsourcing
databases and software provision, the contract giver should ensure that
any subcontractors have been agreed upon and are included in the quality
agreement with the contract accepter, and are appropriately qualified and
trained in GRDP. Their activities should be monitored on a regular basis
at intervals determined through risk assessment. This also applies to
cloud‑based service providers.
7.3 To fulfil this responsibility, in addition to having their own governance
systems, outsourcing organizations should verify the adequacy of the
180
Annex 5
governance systems of the contract acceptor, through an audit or other

suitable means. This should include the adequacy of the contract acceptor’s
controls over suppliers and a list of significant authorized third parties
working for the contract acceptor.
7.4 The personnel who evaluate and periodically assess the competence of a
contracted organization or service provider should have the appropriate
background, qualifications, experience and training to assess data integrity
governance systems and to detect validity issues. The nature and frequency
of the evaluation of the contract acceptor and the approach to ongoing
monitoring of their work should be based upon documented assessment
of risk. This assessment should include an assessment of relevant data
processes and their risks.
7.5 The expected data integrity control strategies should be included in
quality agreements and in written contract and technical arrangements,
as appropriate and applicable, between the contract giver and the contract
acceptor. These should include provisions for the contract giver to have
access to all data held by the contracted organization that are relevant
to the contract giver’s product or service as well as all relevant quality
systems records. This should include ensuring access by the contract
giver to electronic records, including audit trails, held in the contracted
organization’s computerized systems as well as any printed reports and
other relevant paper or electronic records.
7.6 Where data and document retention is contracted to a third party,
particular attention should be paid to understanding the ownership and
retrieval of data held under this arrangement. The physical location where
the data are held, and the impact of any laws applicable to that geographical
location, should also be considered. Agreements and contracts should
establish mutually agreed consequences if the contract acceptor denies,
refuses or limits the contract giver’s access to their records held by the
contract acceptor. The agreements and contracts should also contain
provisions for actions to be taken in the event of business closure or
bankruptcy of the third party to ensure that access is maintained and the
data can be transferred before the cessation of all business activities.
7.7 When outsourcing databases, the contract giver should ensure that if
subcontractors are used, in particular cloud-based service providers, they
are included in the quality agreement and are appropriately qualified and
trained in GRDP. Their activities should be monitored on a regular basis
at intervals determined through risk assessment.
181
8. Training in good data and record management

8.1 Personnel should be trained in data integrity policies and agree to
abide by them. Management should ensure that personnel are trained
to understand and distinguish between proper and improper conduct,
including deliberate falsification, and should be made aware of the
potential consequences.
8.2 In addition, key personnel, including managers, supervisors and quality

unit personnel, should be trained in measures to prevent and detect data
issues. This may require specific training in evaluating the configuration
settings and reviewing electronic data and metadata, such as audit trails,
for individual computerized systems used in the generation, processing
and reporting of data. For example, the quality unit should learn how to
evaluate configuration settings that may intentionally or unintentionally
allow data to be overwritten or obscured through the use of hidden fields
or data annotation tools. Supervisors responsible for reviewing electronic
data should learn which audit trails in the system track significant data
changes and how these might be most efficiently accessed as part of
their review.
8.3 Management should also ensure that, at the time of hire and periodically
afterwards, as needed, all personnel are trained in procedures to
ensure GDocP for both paper and electronic records. The quality unit
should include checks for adherence to GDocP for both paper records
and electronic records in their day-to-day work, system and facility
audits and self-inspections and report any opportunities for improvement
to management.
9. Good documentation practices

9.1 The basic building blocks of good GXP data are to follow GDocP and
then to manage risks to the accuracy, completeness, consistency and
reliability of the data throughout their entire period of usefulness – that
is, throughout the data life cycle.
Personnel should follow GDocP for both paper records and
electronic records in order to assure data integrity. These principles
require that documentation has the characteristics of being attributable,
legible, contemporaneously recorded, original and accurate (sometimes
referred to as ALCOA). These essential characteristics apply equally for
both paper and electronic records.
182
Annex 5
9.2 Attributable. Attributable means information is captured in the record so

that it is uniquely identified as executed by the originator of the data (e.g.
a person or a computer system).
9.3 Legible, traceable and permanent. The terms legible and traceable and
permanent refer to the requirements that data are readable, understandable,
and allow a clear picture of the sequencing of steps or events in the record
so that all GXP activities conducted can be fully reconstructed by the
people reviewing these records at any point during the records retention
period set by the applicable GXP.
9.4 Contemporaneous. Contemporaneous data are data recorded at the time
they are generated or observed.
9.5 Original. Original data include the first or source capture of data or
information and all subsequent data required to fully reconstruct the
conduct of the GXP activity. The GXP requirements for original data
include the following:
■■ original data should be reviewed;
■■ original data and/or true and verified copies that preserve the
content and meaning of the original data should be retained;
■■ as such, original records should be complete, enduring and readily
retrievable and readable throughout the records retention period.
9.6 Accurate. The term “accurate” means data are correct, truthful, complete,
valid and reliable.
9.7 Implicit in the above-listed requirements for ALCOA are that the records
should be complete, consistent, enduring and available (to emphasize
these requirements, this is sometimes referred to as ALCOA-plus).
9.8 Further guidance to aid understanding as to how these requirements
apply in each case and the special risk considerations that may need to be
taken into account during implementation are provided in Appendix 1.
10. Designing and validating systems to

assure data quality and reliability
10.1 Record-keeping methodologies and systems, whether paper or electronic,
should be designed in a way that encourages compliance and assures data
quality and reliability. All requirements and controls necessary to ensure
GDRP should be adhered to for both paper and electronic records.
183
Validation to assure good documentation

practices for electronic data
10.2 To assure the integrity of electronic data, computerized systems should be
validated at a level appropriate for their use and application. Validation
should address the necessary controls to ensure the integrity of data,
including original electronic data and any printouts or PDF reports from
the system. In particular, the approach should ensure that GDocP will
be implemented and that data integrity risks will be properly managed
throughout the data life cycle.
10.3 The “Supplementary guidelines on good manufacturing practices:
validation” (WHO Technical Report Series, No. 937, 2006, Annex 4 (2–4) 1
provide a more comprehensive presentation of validation considerations.
The key aspects of validation that help assure GDocP for electronic data
include, but are not limited to, the following.
10.4 User involvement. Users should be adequately involved in validation
activities to define critical data and data life cycle controls that assure
data integrity.
■■ Examples of activities to engage users may include: prototyping,
user specification of critical data so that risk-based controls can be
applied, user involvement in testing to facilitate user acceptance and
knowledge of system features, and others.
10.5 Configuration and design controls. The validation activities should ensure
configuration settings and design controls for GDocP are enabled and
managed across the computing environment (including both the software
application and operating systems environments).
Activities include, but are not limited to:
■■ documenting configuration specifications for commercial off-the-

shelf systems as well as user-developed systems, as applicable;
■■ restricting security configuration settings for system administrators
to independent personnel, where technically feasible;
■■ disabling configuration settings that allow overwriting and
reprocessing of data without traceability;
■■ restricting access to time/date stamps.
Currently under review.

1
184
Annex 5
For systems to be used in clinical trials, configuration and

design controls should be in place to protect the blinding of the trial,
for example, by restricting access to randomization data that may be
stored electronically.
10.6 Data life cycle. Validation should include assessing risk and developing
quality risk mitigation strategies for the data life cycle, including controls
to prevent and detect risks throughout the steps of:
■■ data generation and capture;
■■ data transmission;
■■ data processing;
■■ data review;
■■ data reporting, including handling of invalid and atypical data;
■■ data retention and retrieval;
■■ data disposal.
Activities might include, but are not limited to:
■■ determining the risk-based approach to reviewing electronic data
and audit trails based upon process understanding and knowledge
of potential impact on products and patients;
■■ writing SOPs defining review of original electronic records and
including meaningful metadata such as audit trails and review of
any associated printouts or PDF records;
■■ documenting the system architecture and data flow, including the
flow of electronic data and all associated metadata, from the point of
creation through archival and retrieval;
■■ ensuring that the relationships between data and metadata are
maintained intact throughout the data life cycle.
10.7 SOPs and training. The validation activities should ensure that adequate
training and procedures are developed prior to release of the system for
GXP use. These should address:
■■ computerized systems administration;
■■ computerized systems use;
■■ review of electronic data and meaningful metadata, such as audit
trails, including training that may be required in system features that
enable users to efficiently and effectively process data and review
electronic data and metadata.
185
10.8 Other validation controls to ensure good data management for both
electronic data and associated paper data should be implemented as
deemed appropriate for the system type and its intended use.
11. Managing data and records

throughout the data life cycle
11.1 Data processes should be designed to adequately mitigate and control and
continuously review the data integrity risks associated with the steps of
acquiring, processing, reviewing and reporting data, as well as the physical
flow of the data and associated metadata during this process through
storage and retrieval.
11.2 QRM of the data life cycle requires understanding the science and
technology of the data process and their inherent limitations. Good data
process design, based upon process understanding and the application of
sound scientific principles, including QRM, would be expected to increase
the assurance of data integrity and to result in an effective and efficient
business process.
11.3 Data integrity risks are likely to occur and to be highest when data processes
or specific data process steps are inconsistent, subjective, open to bias,
unsecured, unnecessarily complex or redundant, duplicated, undefined,
not well understood, hybrid, based upon unproven assumptions and/or
do not adhere to GDRP.
11.4 Good data process design should consider, for each step of the data process,
ensuring and enhancing controls, whenever possible, so that each step is:
■■ consistent;
■■ objective, independent and secure;
■■ simple and streamlined;

■■ well-defined and understood;
■■ automated;
■■ scientifically and statistically sound;
■■ properly documented according to GDRP.
Examples of considerations for each phase of the data life cycle
are provided below.
11.5 Data collection and recording. All data collection and recording should
be performed following GDRP and should apply risk-based controls to
protect and verify critical data.
186
Annex 5
11.6 Example consideration.

Data entries, such as the sample identification for laboratory tests or the
recording of source data for inclusion of a patient in a clinical trial, should
be verified by a second person or entered through technical means such
as barcoding, as appropriate for the intended use of these data. Additional
controls may include locking critical data entries after the data are verified
and review of audit trails for critical data to detect if they have been altered.
11.7 Data processing. To ensure data integrity, data processing should be done
in an objective manner, free from bias, using validated/qualified or verified
protocols, processes, methods, systems, equipment and according to
approved procedures and training programmes.
11.8 Example considerations.
GXP organizations should take precautions to discourage testing or
processing data towards a desired outcome. For example:
■■ to minimize potential bias and ensure consistent data processing,
test methods should have established sample acquisition and
processing parameters, established in default version-controlled
electronic acquisition and processing method files, as appropriate.
Changes to these default parameters may be necessary during
sample processing, but these changes should be documented (who,
what, when?) and justified (why?);
■■ system suitability runs should include only established standards or
reference materials of known concentration to provide an appropriate
comparator for the potential variability of the instrument. If a sample
(e.g. a well-characterized secondary standard) is used for system
suitability or a trial run, written procedures should be established
and followed and the results included in the data review process.
The article under test should not be used for trial run purposes or to
evaluate suitability of the system;
■■ clinical and safety studies should be designed to prevent and detect
statistical bias that may occur through improper selection of data to
be included in statistical calculations.
11.9 Data review and reporting. Data should be reviewed and, where
appropriate, evaluated statistically after completion of the process to
determine whether outcomes are consistent and compliant with established
standards. The evaluation should take into consideration all data,
including atypical, suspect or rejected data, together with the reported
data. This includes a review of the original paper and electronic records.
187
11.10 For example, during self-inspection, some key questions to ask are: Am I
collecting all my data? Am I considering all my data? If I have excluded
some data from my decision-making process, what is the justification
for doing so, and are all the data retained, including both rejected and
reported data?
11.11 The approach to reviewing specific record content, such as critical data
fields and metadata such as cross-outs on paper records and audit trails
in electronic records, should meet all applicable regulatory requirements
and be risk-based.
11.12 Whenever out-of-trend or atypical results are obtained they should be

investigated. This includes investigating and determining corrective
and preventive actions for invalid runs, failures, repeats and other
atypical data. All data should be included in the dataset unless there is a
documented scientific explanation for their exclusion.
11.13 During the data life cycle, data should be subject to continuous
monitoring, as appropriate, to enhance process understanding and
facilitate knowledge management and informed decision-making.
11.14 Example considerations

To ensure that the entire set of data is considered in the reported data, the
review of original electronic data should include checks of all locations
where data may have been stored, including locations where voided,
deleted, invalid or rejected data may have been stored.
11.15 Data retention and retrieval. Retention of paper and electronic records
is discussed in the section above, including measures for backup and
archival of electronic data and metadata.
11.16 Example consideration
1) Data folders on some stand-alone systems may not include all audit
trails or other metadata needed to reconstruct all activities. Other
metadata may be found in other electronic folders or in operating
system logs. When archiving electronic data, it is important to
ensure that associated metadata are archived with the relevant
data set or securely traceable to the data set through appropriate
documentation. The ability to successfully retrieve from the archives
the entire data set, including metadata, should be verified.
188
Annex 5
2) Only validated systems are used for storage of data; however, the
media used for the storage of data do not have an indefinite lifespan.
Consideration must be given to the longevity of media and the
environment in which they are stored. Examples include the fading
of microfilm records, the decreasing readability of the coatings of
optical media such as compact disks (CDs) and digital versatile/
video disks (DVDs), and the fact that these media may become
brittle. Similarly, historical data stored on magnetic media will also
become unreadable over time as a result of deterioration.
12. Addressing data reliability issues

12.1 When issues with data validity and reliability are discovered, it is important
that their potential impact on patient safety and product quality and on
the reliability of information used for decision-making and applications
is examined as a top priority. Health authorities should be notified if the
investigation identifies material impact on patients, products, reported
information or on application dossiers.
12.2 The investigation should ensure that copies of all data are secured in a
timely manner to permit a thorough review of the event and all potentially
related processes.
12.3 The people involved should be interviewed to better understand the
nature of the failure and how it occurred and what might have been done
to prevent and detect the issue sooner. This should include discussions
with the people involved in data integrity issues, as well as supervisory
personnel, quality assurance and management staff.
12.4 The investigation should not be limited to the specific issue identified but
should also consider potential impact on previous decisions based upon
the data and systems now found to be unreliable. In addition, it is vital that
the deeper, underlying root cause(s) of the issue be considered, including
potential management pressures and incentives, for example, a lack of
adequate resources.
12.5 Corrective and preventive actions taken should not only address
the identified issue, but also previous decisions and datasets that are
impacted, as well as deeper, underlying root causes, including the need
for realignment of management expectations and allocation of additional
resources to prevent risks from recurring in the future.
189
References and further reading

References
1. WHO good manufacturing practices for pharmaceutical products: main principles. In: WHO
Expert Committee on Specifications for Pharmaceutical Preparations: forty-eighth report.
Geneva: World Health Organization; 2014: Annex 2 (WHO Technical Report Series, No. 986), also
available on CD-ROM and online.
2. Supplementary guidelines on good manufacturing practice: validation. In: WHO Expert
Committee on Specifications for Pharmaceutical Preparations: fortieth report. Geneva: World
Health Organization; 2006: Annex 4 (WHO Technical Report Series, No. 937).
3. Supplementary guidelines on good manufacturing practice: validation. Qualification of systems
and equipment. In: WHO Expert Committee on Specifications for Pharmaceutical Preparations:
fortieth report. Geneva: World Health Organization; 2006: Annex 4, Appendix 6 (WHO Technical
Report Series, No. 937).
4. Supplementary guidelines on good manufacturing practices: validation. Validation of
computerized systems. In: WHO Expert Committee on Specifications for Pharmaceutical
Preparations: fortieth report. Geneva: World Health Organization; 2006: Annex 4, Appendix 5
(WHO Technical Report Series, No. 937).
Further reading
Computerised systems. In: The rules governing medicinal products in the European Union. Volume 4:
Good manufacturing practice (GMP) guidelines: Annex 11. Brussels: European Commission (http://
ec.europa.eu/enterprise/pharmaceuticals/eudralex/vol-4/pdfs-en/anx11en.pdf).
Good automated manufacturing practice (GAMP) good practice guide: electronic data archiving.
Tampa (FL): International Society for Pharmaceutical Engineering (ISPE); 2007.
Good automated manufacturing practice GAMP good practice guide: A risk-based approach to GxP
compliant laboratory computerized systems, 2nd edition. Tampa (FL): International Society for
Pharmaceutical Engineering (ISPE); 2012.
MHRA GMP data integrity definitions and guidance for industry. London: Medicines and Healthcare
Products Regulatory Agency; March 2015 (https://www.gov.uk/government/uploads/system/uploads/
attachment_data/file/412735/Data_integrity_definitions_and_guidance_v2.pdf).
OECD series on principles of good laboratory practice (GLP) and compliance monitoring. Paris:
Organisation for Economic Co-operation and Development (http://www.oecd.org/chemicalsafety/
testing/oecdseriesonprinciplesofgoodlaboratorypracticeglpandcompliancemonitoring.htm).
Official Medicines Control Laboratories Network of the Council of Europe: Quality assurance documents:
PA/PH/OMCL (08) 69 3R – Validation of computerised systems – core document (https://www.edqm.
eu/sites/default/files/medias/fichiers/Validation_of_Computerised_Systems_Core_Document.pdf )
and its annexes:
■■ PA/PH/OMCL (08) 87 2R – Annex 1: Validation of computerised calculation systems: example
of validation of in-house software
(https://www.edqm.eu/sites/default/files/medias/fichiers/NEW_Annex_1_Validation_of_
computerised_calculation.pdf).
■■ PA/PH/OMCL (08) 88 R – Annex 2: Validation of databases (DB), laboratory information
management systems (LIMS) and electronic laboratory notebooks (ELN)
Databases_DB_Laboratory_.pdf).
190
Annex 5
■■ PA/PH/OMCL (08) 89 R – Annex 3: Validation of computers as part of test equipment

computers_as_part_of_tes.pdf).
Title 21 Code of Federal Regulations (21 CFR Part 11): Electronic records; electronic signatures. US Food
and Drug Administration. The current status of 21 CFR Part 11 Guidance is located under Regulations
and Guidance at: http://www.fda.gov/cder/gmp/index.htm — see background: http://www.fda.gov/
OHRMS/DOCKETS/98fr/03-4312.pdf.
PIC/S guide to good manufacturing practice for medicinal products annexes: Annex 11 – Computerised
systems. Geneva: Pharmaceutical Inspection Co-operation Scheme.
PIC/S PI 011-3 Good practices for computerised systems in regulated GxP environments. Geneva:
Pharmaceutical Inspection Co-operation Scheme.
WHO good manufacturing practices for active pharmaceutical ingredients. In: WHO Expert Committee
on Specifications for Pharmaceutical Preparations: forty-fourth report. Geneva: World Health
Organization; 2010: Annex 2 (WHO Technical Report Series, No. 957).
WHO good practices for pharmaceutical quality control laboratories. In: WHO Expert Committee
on Specifications for Pharmaceutical Preparations: forty-fourth report. Geneva: World Health
Organization; 2010: Annex 1 (WHO Technical Report Series, No. 957).
191
Appendix 1
Expectations and examples of special risk management
considerations for the implementation of ALCOA (-plus)
principles in paper-based and electronic systems
Organizations should follow good documentation practices (GDocP) in order
to assure the accuracy, completeness, consistency and reliability of the records
and data throughout their entire period of usefulness – that is, throughout
the data life cycle. The principles require that documentation should have the
characteristics of being attributable, legible, contemporaneously recorded,
original and accurate (sometimes referred to as ALCOA).
The tables in this appendix provide further guidance on the
implementation of the general ALCOA requirements for both paper and
electronic records and systems. In addition, examples of special risk management
considerations as well as several illustrative examples are provided of how these
measures are typically implemented.
These illustrative examples are provided to aid understanding of the
concepts and of how successful risk-based implementation might be achieved.
These examples should not be taken as setting new normative requirements.
Attributable. Attributable means information is captured in the record so that
it is uniquely identified as having been executed by the originator of the data
(e.g. a person or computer system).
Attributable
Expectations for paper records Expectations for electronic records

Attribution of actions in paper Attribution of actions in electronic records should
records should occur, as occur, as appropriate, through the use of:
appropriate, through the use of: • unique user logons that link the user to actions
• initials; that create, modify or delete data;
• full handwritten signature; • unique electronic signatures (can be either
• personal seal; biometric or non-biometric);
• date and, when necessary, time. • an audit trail that should capture user
identification (ID) and date and time stamps;
• signatures, which must be securely and
permanently linked to the record being signed.
192
Annex 5
Special risk management considerations for controls to ensure

that actions and records are attributed to a unique individual
■■ For legally-binding signatures, there should be a verifiable, secure
link between the unique, identifiable (actual) person signing and
the signature event. Signatures should be permanently linked to the
record being signed. Systems which use one application for signing
a document and another to store the document being signed should
ensure that the two remain linked to ensure that the attribution is
not broken.
■■ Signatures and personal seals should be executed at the time of
review or performance of the event or action being recorded.
■■ Use of a personal seal to sign documents requires additional risk
management controls, such as handwritten dates and procedures that
require storage of the seal in a secure location with access limited
only to the assigned individual, or equipped with other means of
preventing potential misuse.
■■ Use of stored digital images of a person’s handwritten signature
to sign a document is not acceptable. This practice compromises
confidence in the authenticity of these signatures when these stored
images are not maintained in a secure location, access to which
is limited only to the assigned individual, or equipped with other
means of preventing potential misuse, and instead are placed in
documents and emails where they can be easily copied and reused
by others. Legally binding, handwritten signatures should be dated at
the time of signing and electronic signatures should include the time/
date stamp of signing to record the contemporaneous nature of the
signing event.
■■ The use of hybrid systems is discouraged, but where legacy systems
are awaiting replacement, mitigating controls should be in place.
The use of shared and generic logon credentials should be avoided
to ensure that actions documented in electronic records can be
attributed to a unique individual. This would apply to the software
application level and all applicable network environments where
personnel may perform actions (e.g. workstation and server
operating systems). Where such technical controls are not available
or feasible, for example, in legacy electronic systems or where
logon would terminate an application or stop the process running,
combinations of paper and electronic records should be used to meet
the requirements to attribute actions to the individuals concerned.
In such cases, original records generated during the course of GXP
193
activities must be complete and must be maintained throughout

the records retention period in a manner that allows the full
reconstruction of the GXP activities.
■■ A hybrid approach might exceptionally be used to sign electronic
records when the system lacks features for electronic signatures,
provided adequate security can be maintained. The hybrid approach
is likely to be more burdensome than a fully-electronic approach;
therefore, utilizing electronic signatures, whenever available, is
recommended. For example, the execution and attribution of an
electronic record by attachment of a handwritten signature may
be performed through a simple means that would create a single-
page controlled form associated with the written procedures for
system use and data review. The document should list the electronic
dataset reviewed and any metadata subject to review, and would
provide fields for the author, reviewer and/or approver of the
dataset to insert a handwritten signature. This paper record with
the handwritten signatures should then be securely and traceably
linked to the electronic dataset, either through procedural means,
such as use of detailed archives indexes, or technical means, such as
embedding a true-copy scanned image of the signature page into the
electronic dataset.
■■ Replacement of hybrid systems should be a priority.
■■ The use of a scribe to record an activity on behalf of another
operator should be considered only on an exceptional basis and
should only take place where:
–– the act of recording places the product or activity at risk, e.g.
documenting line interventions by aseptic area operators;
–– to accommodate cultural differences or mitigate staff literacy/
language limitations, for instance, where an activity is performed

by an operator, but witnessed and recorded by a supervisor or
officer.
In both situations, the supervisory recording should be contemporaneous
with the task being performed and should identify both the person performing
the observed task and the person completing the record. The person performing
the observed task should countersign the record wherever possible, although
it is accepted that this countersigning step will be retrospective. The process
for supervisory (scribe) documentation completion should be described in
an approved procedure which should also specify the activities to which the
process applies.
194
Annex 5
Legible, traceable and permanent

The terms legible, traceable and permanent refer to the requirements that data
are readable, understandable and allow a clear picture of the sequencing of
steps or events in the record so that all GXP activities conducted can be fully
reconstructed by people reviewing these records at any point during the records
retention period set by the applicable GXP.
Legible, traceable, permanent

Legible, traceable and permanent Legible, traceable and permanent controls
controls for paper records include, but for electronic records include, but are not
are not limited to: limited to:
• use of permanent, indelible ink; • designing and configuring computer
• no use of pencil or erasures; systems and writing standard operating
procedures (SOPs), as required, that
• use of single-line cross-outs to record
enforce the saving of electronic data
changes with name, date and reason
at the time of the activity and before
recorded (i.e. the paper equivalent to
proceeding to the next step of the
the audit trail);
sequence of events (e.g. controls that
• no use of opaque correction fluid or prohibit generation and processing and
otherwise obscuring the record; deletion of data in temporary memory
• controlled issuance of bound, and that instead enforce the committing
paginated notebooks with sequentially of the data at the time of the activity to
numbered pages (i.e. that allow durable memory before moving to the
detection of missing or skipped pages); next step in the sequence);
• controlled issuance of sequentially • use of secure, time-stamped audit trails
numbered copies of blank forms that independently record operator
(i.e. that allow all issued forms to be actions and attribute actions to the
accounted for); logged-on individual;
• archival of paper records by • configuration settings that restrict
independent, designated personnel in access to enhanced security permissions
secure and controlled paper archives (such as the system administrator role
(archivist is the term used for these that can be used to potentially turn off
personnel in quality control, good the audit trails or enable overwriting
laboratory practices (GLP) and good and deletion of data), only to persons
clinical practices (GCP) settings. independent of those responsible for
In good manufacturing practices the content of the electronic records;
(GMP) settings this role is normally • configuration settings and SOPs, as
designated to specific individual(s) in required, to disable and prohibit the
the quality assurance unit); ability to overwrite data, including
prohibiting overwriting of preliminary
and intermediate processing of data;
195
Table continued
Legible, traceable, permanent
• preservation of paper/ink that • strictly controlled configuration and use
fades over time where their use is of data annotation tools in a manner
unavoidable. that prevents data in displays and
printouts from being obscured;
• validated backup of electronic records
to ensure disaster recovery;
• validated archival of electronic records
by independent, designated archivist(s)
in secure and controlled electronic
archives.
Special risk management considerations for legible,

traceable and permanent recording of GXP data
■■ When computerized systems are used to generate electronic data, it
should be possible to associate all changes to data with the people
who make those changes, and those changes should be time-
stamped and a reason for the change recorded where applicable. This
traceability of user actions should be documented via computer-
generated audit trails or in other metadata fields or system features
that meet these requirements.
■■ Users should not be able to amend or switch off the audit trails or
alternative means of providing traceability of user actions.
■■ The need for the implementation of appropriate audit trail
functionality should be considered for all new computerized systems.

Where an existing computerized system lacks computer-generated
audit trails, personnel may use alternative means such as procedurally-
controlled use of logbooks, change control, record version control
or other combinations of paper and electronic records to meet GXP
regulatory expectations for traceability to document the what, who,
when and why of an action. Procedural controls should include
written procedures, training programmes, review of records and
audits and self-inspections of the governing process(es).
196
Annex 5
■■ When archival of electronic records is used, the archiving process

should be done in a controlled manner to preserve the integrity
of the records. Electronic archives should be validated, secured
and maintained in a state of control throughout the data life cycle.
Electronic records archived manually or automatically should be
stored in secure and controlled electronic archives, accessible only by
independent, designated archivists or by their approved delegates.
Appropriate separation of duties should be established so that

business process owners, or other users who may have a conflict of
interest, are not granted enhanced security access permissions at
any system level (e.g. operating system, application and database).
Further, highly privileged system administrator accounts should
be reserved for designated technical personnel, e.g. information
technology (IT) personnel, who are fully independent of the
personnel responsible for the content of the records, as these types
of accounts may include the ability to change settings to overwrite,
rename, delete, move data, change time/date settings, disable audit
trails and perform other system maintenance functions that turn off
the good data and record management practices (GDRP) controls
for legible and traceable electronic data. Where it is not feasible to
assign these independent security roles, other control strategies
should be used to reduce data validity risks.
–– To avoid conflicts of interest, these enhanced system access

permissions should only be granted to personnel with system
maintenance roles (e.g. IT, metrology, records control,
engineering), that are fully independent of the personnel
responsible for the content of the records (e.g. laboratory
analysts, laboratory management, clinical investigators, study
directors, production operators and production management).
Where these independent security role assignments are not
feasible, other control strategies should be used to reduce data
validity risks.
It is particularly important that individuals with enhanced access permissions

understand the impact of any changes they make using these privileges. Personnel
with enhanced access should therefore also be trained in data integrity principles.
197
Contemporaneous
Contemporaneous data are data recorded at the time they are generated
or observed.
Contemporaneous
Contemporaneous recording of actions Contemporaneous recording of actions
in paper records should occur, as in electronic records should occur, as
appropriate, through use of: appropriate, through use of:
• written procedures, and training and • configuration settings, SOPs and
review and audit and self-inspection controls that ensure that data recorded
controls that ensure personnel record in temporary memory are committed
data entries and information at the to durable media upon completion
time of the activity directly in official of the step or event and before
controlled documents (e.g. laboratory proceeding to the next step or event
notebooks, batch records, case in order to ensure the permanent
report forms); recording of the step or event at the
• procedures requiring that activities time it is conducted;
be recorded in paper records with the • secure system time/date stamps that
date of the activity (and time as well, cannot be altered by personnel;
if it is a time-sensitive activity); • procedures and maintenance
• good document design, which programmes that ensure time/date
encourages good practice: documents stamps are synchronized across the
should be appropriately designed GXP operations;
and the availability of blank forms/ • controls that allow for the
documents in which the activities are determination of the timing of one
recorded should be ensured; activity relative to another (e.g. time
• recording of the date and time of zone controls);
activities using synchronized time • availability of the system to the user at

sources (facility and computerized the time of the activity.
system clocks) which cannot be
changed by unauthorized personnel.
Where possible, data and time
recording of manual activities
(e.g. weighing) should be done
automatically.
198
Annex 5
Special risk management considerations for

contemporaneous recording of GXP data
■■ Training programmes in GDocP should emphasize that it is

unacceptable to record data first in unofficial documentation (e.g. on
a scrap of paper) and later transfer the data to official documentation
(e.g. the laboratory notebook). Instead, original data should be
recorded directly in official records, such as approved analytical
worksheets, immediately at the time of the GXP activity.
■■ Training programmes should emphasize that it is unacceptable to
backdate or forward date a record. Instead the date recorded should
be the actual date of the data entry. Late entries should be indicated as
such with both the date of the activity and the date of the entry being
recorded. If a person makes mistakes on a paper document he or she
should make single-line corrections, sign and date them, provide
reasons for the changes and retain this record in the record set.
■■ If users of stand-alone computerized systems are provided with full
administrator rights to the workstation operating systems on which
the original electronic records are stored, this may inappropriately
grant permission to users to rename, copy or delete files stored
on the local system and to change the time/date stamp. For this
reason, validation of the stand-alone computerized system should
ensure proper security restrictions to protect time/date settings
and ensure data integrity in all computing environments, including
the workstation operating system, the software application and any
other applicable network environments.
Original
Original data include the first or source capture of data or information and all
subsequent data required to fully reconstruct the conduct of the GXP activity.
The GXP requirements for original data include the following:
■■ original data should be reviewed;
■■ original data and/or true and verified copies that preserve the
content and meaning of the original data should be retained;
■■ as such, original records should be complete, enduring and readily
retrievable and readable throughout the records retention period.
Examples of original data include original electronic data and metadata in
stand-alone computerized laboratory instrument systems (e.g. ultraviolet/visible
spectrophotometry (UV/Vis), Fourier transform infrared spectroscopy (FT-IR),
199
electrocardiogram (ECG), liquid chromatography-tandem mass spectrometry

(LC/MS/MS) and haematology and chemistry analysers), original electronic data
and metadata in automated production systems (e.g. automated filter integrity
testers, supervisory control and data acquisition (SCADA) and distributed
control system (DCS)), original electronic data and metadata in network database
systems (e.g. laboratory information management system (LIMS), enterprise
resource planning (ERP), manufacturing execution systems (MES), electronic
case report form/electronic data capture (eCRF/EDC), toxicology databases, and
deviation and corrective and preventive action (CAPA) databases), handwritten
sample preparation information in paper notebooks, printed recordings of
balance readings, electronic health records and paper batch records.
Review of original records

Controls for review of original paper Controls for review of original electronic
records include, but are not limited to: records include, but are not limited to:
• written procedures and training and • written procedures and training and
review and audit and self-inspection review and audit and inspection
controls to ensure that personnel controls that ensure personnel conduct
conduct an adequate review and an adequate review and approval of
approval of original paper records, original electronic records, including
including those used to record human readable source records of
the contemporaneous capture of electronic data;
information; • data review procedures describing
• data review procedures describing review of original electronic data
review of relevant metadata. For and relevant metadata. For example,
example, written procedures for review written procedures for review should
should require that personnel evaluate require that personnel evaluate
changes made to original information changes made to original information

on paper records (such as changes in electronic records (such as changes
documented in cross-out or data documented in audit trails or history
correction) to ensure these changes fields or found in other meaningful
are appropriately documented, and metadata) to ensure these changes
justified with substantiating evidence are appropriately documented and
and investigated when required; justified with substantiating evidence
and investigated when required;
200
Annex 5
Table continued
Review of original records
• documentation of data review. For • documentation of data review. For
paper records this is typically signified electronic records, this is typically
by signing the paper records that have signified by electronically signing
been reviewed. Where record approval the electronic data set that has been
is a separate process this should also be reviewed and approved. Written
similarly signed. Written procedures for procedures for data review should
data review should clarify the meaning clarify the meaning of the review and
of the review and approval signatures approval signatures to ensure that
to ensure that the people concerned the personnel concerned understand
understand their responsibility as their responsibility as reviewers and
reviewers and approvers to assure the approvers to assure the integrity,
integrity, accuracy, consistency and accuracy, consistency and compliance
compliance with established standards with established standards of the
of the paper records subject to review electronic data and metadata subject
and approval; to review and approval;
• a procedure describing the actions • a procedure describing the actions
to be taken if data review identifies to be taken if data review identifies
an error or omission. This procedure an error or omission. This procedure
should enable data corrections or should enable data corrections or
clarifications to be made in a GXP- clarifications to be made in a GXP-
compliant manner, providing visibility compliant manner, providing visibility
of the original record and audit-trailed of the original record and audit trailed
traceability of the correction, using traceability of the correction, using
ALCOA principles. ALCOA principles.
Special risk management considerations for review of original records

■■ Data integrity risks may occur when people choose to rely solely
upon paper printouts or PDF reports from computerized systems
without meeting applicable regulatory expectations for original
records. Original records should be reviewed – this includes
electronic records. If the reviewer only reviews the subset of data
provided as a printout or PDF, risks may go undetected and harm
may occur.
■■ Although original records should be reviewed, and all personnel
involved are fully accountable for the integrity and reliability of the
subsequent decisions made based upon original records, a risk-
based review of the content of original records is recommended.
201
■■ Systems typically include many metadata fields and audit trails. It is

expected that during validation of the system the organization will
establish – based upon a documented and justified risk assessment
– the frequency, roles and responsibilities, and the approach used
to review the various types of meaningful metadata, such as audit
trails. For example, under some circumstances, an organization may
justify periodic review of audit trails that track system maintenance
activities, whereas audit trails that track changes to critical GXP
data with a direct impact on patient safety or product quality would
be expected to be reviewed each and every time the associated data
set is being reviewed and approved – and prior to decision-making.
Certain aspects of defining the audit trail review process (e.g.
frequency) may be initiated during validation and then adjusted
over time during the system life cycle, based upon risk reviews and
to ensure continual improvement.
■■ A risk-based approach to reviewing data requires process
understanding and knowledge of the key quality risks in the given
process that may impact patients, products, compliance and the
overall accuracy, consistency and reliability of GXP decision-making.
When original records are electronic, a risk-based approach to
reviewing original electronic data also requires an understanding of
the computerized system, the data and metadata, and the data flows.
■■ When determining a risk-based approach to reviewing audit trails
in GXP computerized systems, it is important to note that some
software developers may design mechanisms for tracking user actions
related to the most critical GXP data using metadata features and
may not have named these “audit trails” but may instead have used
the naming convention “audit trail” to track other computer system
and file maintenance activities. For example, changes to scientific
data may sometimes be most readily viewed by running various

database queries or by viewing metadata fields labelled “history
files” or by review of designed and validated system reports, and the
files designated by the software developer as audit trails alone may
be of limited value for an effective review. The risk-based review
of electronic data and metadata, such as audit trails, requires an
understanding of the system and the scientific process governing the
data life cycle so that the meaningful metadata are subject to review,
regardless of the naming conventions used by the software developer.
■■ Systems may be designed to facilitate audit trail review by various
means; for example, the system design may permit audit trails to
be reviewed as a list of relevant data or by a validated exception
reporting process.
202
Annex 5
■■ Written procedures for data review should define the frequency, roles
and responsibilities and approach to review of meaningful metadata,
such as audit trails. These procedures should also describe how
aberrant data are to be handled if found during the review. Personnel
who conduct such reviews should have adequate and appropriate
training in the review process as well as in the software systems
containing the data subject to review. The organization should make
the necessary provisions for personnel reviewing the data to access
the system(s) containing the electronic data and metadata.
■■ Quality assurance should also review a sample of relevant audit trails,
raw data and metadata as part of self-inspection to ensure ongoing
compliance with the data governance policy and procedures.
■■ Any significant variation from expected outcomes should be fully
recorded and investigated.
■■ In the hybrid approach, which is not the preferred approach, paper
printouts of original electronic records from computerized systems
may be useful as summary reports if the requirements for original
electronic records are also met. To rely upon these printed summaries
of results for future decision-making, a second person would have to
review the original electronic data and any relevant metadata such
as audit trails, to verify that the printed summary is representative
of all results. This verification would then be documented and the
printout could be used for subsequent decision-making.
■■ The GXP organization may choose a fully electronic approach to
allow more efficient, streamlined record review and record retention.
This would require authenticated and secure electronic signatures
to be implemented for signing records where required. This, in turn,
would require preservation of the original electronic records, or
true copy, as well as the necessary software and hardware or other
suitable reader equipment to view the records during the records
retention period.
■■ System design and the manner of data capture can significantly
influence the ease with which data consistency can be assured. For
example, and where applicable, the use of programmed edit checks
or features such as drop-down lists, check boxes or branching of
questions or data fields based on entries are useful in improving
data consistency.
■■ Data and their metadata should be maintained in such a way that
they are available for review by authorized individuals, and in a
format that is suitable for review for as long as the data retention
requirements apply. It is desirable that the data should be maintained
203
and available in the original system in which they were generated

for the longest possible period of time. When the original system is
retired or decommissioned, migration of the data to other systems or
other means of preserving the data should be used in a manner that
preserves the context and meaning of the data, allowing the relevant
steps to be reconstructed. Checks of accessibility to archived data,
irrespective of format, and including relevant metadata, should be
undertaken to confirm that the data are enduring, and continue to
be available, readable and understandable by a human being.
Retention of original records or true copies

Controls for retention of original paper Controls for retention of original electronic
records or true copies of original paper records or true copies of original electronic
records include, but are not limited to: records include, but are not limited to:
• controlled and secure storage areas, • routine back-up copies of original
including archives, for paper records; electronic records stored in another
• a designated paper archivist(s) who location as a safeguard in case of disaster
is independent of GXP operations is that causes loss of the original electronic
required by GLP guidelines; in other records;
GXPs the roles and responsibilities • controlled and secure storage areas,
for archiving GXP records should be including archives, for electronic records;
defined and monitored (and should • a designated electronic archivist(s) such
normally be the responsibility of as is required in GLP guidelines who is
the quality assurance function or independent of GXP operations (the
an independent documentation designated personnel should be suitably
control unit); qualified and have relevant experience
• indexing of records to permit ready and appropriate training to perform
retrieval; their duties);
• periodic tests at appropriate intervals • indexing of records to permit ready

based upon risk assessment, to verify retrieval;
the ability to retrieve archived paper or • periodic tests to verify the ability to
static format records; retrieve archived electronic data from
• the provision of suitable reader storage locations. The ability to retrieve
equipment when required, such as archived electronic data from storage
microfiche or microfilm readers if locations should be tested during the
original paper records are copied as validation of the electronic archive.
true copies to microfilm or microfiche After validation the ability to retrieve
for archiving; archived electronic data from the
storage locations should be periodically
reconfirmed, including retrieval from
third-party storage;
204
Annex 5
Table continued
Retention of original records or true copies
• written procedures, training, review • the provision of suitable reader
and audit, and self-inspection of equipment, such as software, operating
processes defining conversion, as systems and virtualized environments,
needed, of an original paper record to view the archived electronic data
to true copy should include the when required;
following steps: • written procedures, training, review and
– a copy/copies is/are made of the audit and self-inspection of processes
original paper record(s), preserving defining conversion, as needed, of
the original record format, the static original electronic records to true copy
format, as required (e.g. photocopy, to include the following steps:
scan), – a copy/copies is/are made of
– the copy/copies need to be the original electronic data set,
compared with the original record(s) preserving the original record format,
to determine if the copy preserves the dynamic format, as required (e.g.
the entire content and meaning of archival copy of the entire set of
the original record, that metadata are electronic data and metadata made
included, that no data are missing using a validated back-up process),
in the copy. The way that the record – a second person verifier or technical
format is preserved is important for verification process (such as use of
record meaning if the copy is to meet technical hash) to confirm successful
the requirements of a true copy of backup) whereby a comparison is
the original paper record(s), made of the electronic archival copy
– the verifier documents the with the original electronic data set
verification in a manner securely to confirm the copy preserves the
linked to the copy/copies indicating entire content and meaning of the
it is a true copy, or provides original record (i.e. all of the data
equivalent certification. and metadata are included, no data
are missing in the copy, any dynamic
record format that is important for
record meaning and interpretation
is preserved and the file was not
corrupted during the execution of the
validated back-up process),
– if the copy meets the requirements
as a true copy of the original, then
the verifier or technical verification
process should document the
verification in a manner that is
securely linked to the copy/copies,
certifying that it is a true copy.
205
Special risk management considerations for retention

of original records and/or true copies
■■ Data and document retention arrangements should ensure the
protection of records from deliberate or inadvertent alteration or loss.
Secure controls should be in place to ensure the data integrity of the
record throughout the retention period. Archival processes should be
defined in written procedures and validated where appropriate.
■■ Data collected or recorded (manually and/or by recording
instruments or computerized systems) during a process or procedure
should show that all the defined and required steps have been taken
and that the quantity and quality of the output are as expected, and
should enable the complete history of the process or material to be
traced and be retained in a comprehensible and accessible form. That
is, original records and/or true copies should be complete, consistent
and enduring.
■■ A true copy of original records may be retained in lieu of the
original records only if the copy has been compared to the original
records and verified to contain the entire content and meaning of
the original records, including applicable metadata and audit trails.
■■ If true copies of original paper records are made by scanning the
original paper and conversion to an electronic image, such as PDF,
then additional measures to protect the electronic image from
further alteration are required (e.g. storage in a secure network
location with access limited to electronic archivist personnel only,
and measures taken to control potential use of annotation tools or
other means of preventing further alteration of the copy).
■■ Consideration should be given to preservation where necessary of
the full content and meaning of original hand-signed paper records,

especially when the handwritten signature is an important aspect of
the overall integrity and reliability of the record and in accordance
with the value of the record over time. For example, in a clinical
trial it may be important to preserve original hand-signed informed
consent records throughout the useful life of this record as an
essential aspect of the trial and related application integrity.
■■ True copies of electronic records should preserve the dynamic
format of the original electronic data as this is essential to
preserving the meaning of the original electronic data, e.g. if the old
software or equipment is retired. For example, the original dynamic
electronic spectral files created by instruments such as FT-IR, UV/
206
Annex 5
Vis, chromatography systems and others can be reprocessed, but a

pdf or printout is fixed or static and the ability to expand baselines,
view the full spectrum, reprocess and interact dynamically with the
data set would be lost in the PDF or printout. As another example,
preserving the dynamic format of clinical study data captured in
an eCRF system allows searching and querying of data, whereas
a pdf of the eCRF data, even if it includes a PDF of audit trails,
would lose this aspect of the content and meaning of the original
eCRF data. Clinical investigators should have access to original
records throughout the study and records retention period in a
manner that preserves the full content and meaning of the source
information. It may be decided to maintain complete copies of
electronic data as well as PDF/printed summaries of these electronic
data in the archives to mitigate risks of a complete loss of ability to
readily view the data should the software and hardware be retired.
However, under these circumstances, especially for data that
support critical decision-making, even if PDF/printed summaries
are maintained, the complete copies of electronic data should
continue to be maintained throughout the records retention period
to allow for investigations that may be necessary under unexpected
circumstances, such as application integrity investigations.
■■ Preserving the original electronic data in electronic form is also
important because data in dynamic format facilitate usability of the
data for subsequent processes. For example, having temperature
logger data maintained electronically facilitates subsequent tracking
and trending and monitoring of temperatures in statistical process
control charts.
■■ In addition to the option of creating true copies of original electronic
data as verified back-up copies that are then secured in electronic
archives, another option for creating a true copy of original
electronic data would be to migrate the original electronic data from
one system to another and to verify and document that the validated
data migration process preserved the entire content, including
all meaningful metadata, as well as the meaning of the original
electronic data.
■■ Electronic signature information should be retained as part of the
original electronic record. This should remain linked to the record
and be readable throughout the retention period, regardless of the
system used for archiving the records.
207
Accurate
The term “accurate” means data are correct, truthful, complete, valid and reliable.
For both paper and electronic records, achieving the goal of accurate
data requires adequate procedures, processes, systems and controls that comprise
the quality management system. The quality management system should be
appropriate to the scope of its activities and risk-based.
Controls that assure the accuracy of data in paper records and electronic
records include, but are not limited to:
■■ qualification, calibration and maintenance of equipment, such as

balances and pH meters, that generate printouts;
■■ validation of computerized systems that generate, process, maintain,
distribute or archive electronic records;
■■ systems must be validated to ensure their integrity while transmitting
between/among computerized systems;
■■ validation of analytical methods;
■■ validation of production processes;
■■ review of GXP records;
■■ investigation of deviations and doubtful and out-of-specifications
results; and
■■ many other risk management controls within the quality
management system.
Examples of these controls applied to the data life cycle are provided
below.
Special risk management considerations for assuring accurate GXP records

■■ The entry of critical data into a computer by an authorized person

(e.g. entry of a master processing formula) requires an additional
check on the accuracy of the data entered manually. This check
may be done by independent verification and release for use by a
second authorized person or by validated electronic means. For
example, to detect and manage risks associated with critical data,
procedures would require verification by a second person, such as
a member of the quality unit staff, of: calculation formulas entered
into spreadsheets; master data entered into LIMS such as fields for
specification ranges used to flag out-of-specification values on the
certificate of analysis; and other critical master data, as appropriate.
208
Annex 5
In addition, once verified, these critical data fields would be locked

to prevent further modification, when feasible and appropriate, and
only modified through a formal change control process.
■■ The validity of the data capture process is fundamental to ensuring
that high-quality data are produced.
■■ Where used, standard dictionaries and thesauruses, tables (e.g. units
and scales) should be controlled.
■■ The process of data transfer between systems should be validated.
■■ The migration of data into and export from systems requires specific
planned testing and control.
■■ Time may not be critical for all activities. When the activity is time-
critical, printed records should display the time/date stamp.
For example: To ensure the accuracy of sample weights recorded on a paper

printout from the balance, the balance would be appropriately calibrated before
use and properly maintained. In addition, synchronizing and locking the
metadata settings on the balance for the time/date settings would ensure accurate
recordings of time/date on the balance printout.
209
Appendices B: MHRA GxP Data Integrity Guidance
and Definitions
Medicines & Healthcare products
Regulatory Agency (MHRA)
‘GXP’ Data Integrity Guidance and Definitions
March 2018
MHRA GXP Data Integrity Guidance and Definitions; Revision 1: March 2018
Page 1 of 21
Table of contents
1. Background .................................................................................................................................. 3
2. Introduction .................................................................................................................................. 3
3. The principles of data integrity ...................................................................................................... 4
4. Establishing data criticality and inherent integrity risk ................................................................... 5
5. Designing systems and processes to assure data integrity; creating the ‘right environment’......... 7
6. Definition of terms and interpretation of requirements................................................................... 8
6.1. Data ...................................................................................................................................... 8
6.2. Raw data (synonymous with ‘source data’ which is defined in ICH GCP) .............................. 8
6.3. Metadata ............................................................................................................................... 9
6.4. Data Integrity ......................................................................................................................... 9
6.5. Data Governance .................................................................................................................. 9
6.6. Data Lifecycle...................................................................................................................... 10
6.7. Recording and collection of data ......................................................................................... 10
6.8. Data transfer / migration ...................................................................................................... 10
6.9. Data Processing .................................................................................................................. 11
6.10. Excluding Data (not applicable to GPvP): ........................................................................ 11
6.11. Original record and true copy ........................................................................................... 11
6. 11.1. Original record ............................................................................................................... 11
6.11.2. True copy ....................................................................................................................... 12
6.12. Computerised system transactions: ................................................................................. 13
6.13. Audit Trail ........................................................................................................................ 13
6.14. Electronic signatures........................................................................................................ 14
6.15. Data review and approval ................................................................................................ 15
6.16. Computerised system user access/system administrator roles ........................................ 16
6.17. Data retention .................................................................................................................. 17
6.17.1. Archive ........................................................................................................................... 18
6.17.2. Backup ........................................................................................................................... 18
6.18. File structure .................................................................................................................... 19
6.19. Validation – for intended purpose (GMP; See also Annex 11, 15) .................................... 19
6.20. IT Suppliers and Service Providers (including Cloud providers and virtual service/platforms
(also referred to as software as a service SaaS/platform as a service (PaaS) / infrastructure as a
service (IaaS)). .............................................................................................................................. 19
7. Glossary ..................................................................................................................................... 20
8. References ................................................................................................................................. 21
Page 2 of 21
1. Background
The way regulatory data is generated has continued to evolve in line with the ongoing development of
supporting technologies such as the increasing use of electronic data capture, automation of systems
and use of remote technologies; and the increased complexity of supply chains and ways of working,
for example, via third party service providers. Systems to support these ways of working can range
from manual processes with paper records to the use of fully computerised systems. The main
purpose of the regulatory requirements remains the same, i.e. having confidence in the quality and
the integrity of the data generated (to ensure patient safety and quality of products) and being able to
reconstruct activities.
2. Introduction
2.1 This document provides guidance for UK industry and public bodies regulated by the
UK MHRA including the Good Laboratory Practice Monitoring Authority (GLPMA).
Where possible the guidance has been harmonised with other published guidance.
The guidance is a UK companion document to PIC/S, WHO, OECD (guidance and
advisory documents on GLP) and EMA guidelines and regulations.
2.2 This guidance has been developed by the MHRA inspectorate and partners and has
undergone public consultation. It is designed to help the user facilitate compliance
through education, whilst clarifying the UK regulatory interpretation of existing
requirements.
2.3 Users should ensure their efforts are balanced when safeguarding data from risk with
their other compliance priorities.
2.4 The scope of this guidance is designated as ‘GXP’ in that everything contained within
the guide is GXP unless stated otherwise. The lack of examples specific to a GXP
does not mean it is not relevant to that GXP just that the examples given are not
exhaustive. Please do however note that the guidance document does not extend to
medical devices.
2.5 This guidance should be considered as a means of understanding the MHRA’s position
on data integrity and the minimum expectation to achieve compliance. The guidance
does not describe every scenario so engagement with the MHRA is encouraged where
your approach is different to that described in this guidance.
2.6 This guidance aims to promote a risk-based approach to data management that
includes data risk, criticality and lifecycle. Users of this guidance need to understand
their data processes (as a lifecycle) to identify data with the greatest GXP impact.
From that, the identification of the most effective and efficient risk-based control and
review of the data can be determined and implemented.
2.7 This guidance primarily addresses data integrity and not data quality since the controls
required for integrity do not necessarily guarantee the quality of the data generated.
Page 3 of 21
2.8 This guidance should be read in conjunction with the applicable regulations and the
general guidance specific to each GXP. Where GXP-specific references are made
within this document (e.g. ICH Q9), consideration of the principles of these documents
may provide guidance and further information.
2.9 Where terms have been defined; it is understood that other definitions may exist and
these have been harmonised where possible and appropriate.
3. The principles of data integrity

3.1 The organisation needs to take responsibility for the systems used and the data they
generate. The organisational culture should ensure data is complete, consistent and
accurate in all its forms, i.e. paper and electronic.
3.2 Arrangements within an organisation with respect to people, systems and facilities
should be designed, operated and, where appropriate, adapted to support a suitable
working environment, i.e. creating the right environment to enable data integrity
controls to be effective.
3.3 The impact of organisational culture, the behaviour driven by performance indicators,
objectives and senior management behaviour on the success of data governance
measures should not be underestimated. The data governance policy (or equivalent)
should be endorsed at the highest levels of the organisation.
3.4 Organisations are expected to implement, design and operate a documented system
that provides an acceptable state of control based on the data integrity risk with
supporting rationale. An example of a suitable approach is to perform a data integrity
risk assessment (DIRA) where the processes that produce data or where data is
obtained are mapped out and each of the formats and their controls are identified and
the data criticality and inherent risks documented.
3.5 Organisations are not expected to implement a forensic approach to data checking on
a routine basis. Systems should maintain appropriate levels of control whilst wider data
governance measures should ensure that periodic audits can detect opportunities for
data integrity failures within the organisation’s systems.
3.6 The effort and resource applied to assure the integrity of the data should be
commensurate with the risk and impact of a data integrity failure to the patient or
environment. Collectively these arrangements fulfil the concept of data governance.
3.7 Organisations should be aware that reverting from automated or computerised

systems to paper-based manual systems or vice-versa will not in itself remove the
need for appropriate data integrity controls.
3.8 Where data integrity weaknesses are identified, companies should ensure that
appropriate corrective and preventive actions are implemented across all relevant
activities and systems and not in isolation.
Page 4 of 21
3.9 Appropriate notification to regulatory authorities should be made where significant data
integrity incidents have been identified.
3.10 The guidance refers to the acronym ALCOA rather than ‘ALCOA +’. ALCOA being
Attributable, Legible, Contemporaneous, Original, and Accurate and the ‘+’ referring to
Complete, Consistent, Enduring, and Available. ALCOA was historically regarded as
defining the attributes of data quality that are suitable for regulatory purposes. The ‘+’
has been subsequently added to emphasise the requirements. There is no difference
in expectations regardless of which acronym is used since data governance measures
should ensure that data is complete, consistent, enduring and available throughout the
data lifecycle.
4. Establishing data criticality and inherent integrity risk

4.1 Data has varying importance to quality, safety and efficacy decisions. Data criticality
may be determined by considering how the data is used to influence the decisions
made.
4.2 The risks to data are determined by the potential to be deleted, amended or excluded
without authorisation and the opportunity for detection of those activities and events.
The risks to data may be increased by complex, inconsistent processes with open-
ended and subjective outcomes, compared to simple tasks that are undertaken
consistently, are well defined and have a clear objective.
4.3 Data may be generated by:

(i) Recording on paper, a paper-based record of a manual observation or of an
activity or
(ii) electronically, using equipment that range from simple machines through to
complex highly configurable computerised systems or
(iii) by using a hybrid system where both paper-based and electronic records
constitute the original record or
(iv) by other means such as photography, imagery, chromatography plates, etc.
Paper
Data generated manually on paper may require independent verification if deemed
necessary from the data integrity risk assessment or by another requirement.
Consideration should be given to risk-reducing supervisory measures.
Electronic
The inherent risks to data integrity relating to equipment and computerised systems
may differ depending upon the degree to which the system generating or using the
data can be configured, and the potential for manipulation of data during transfer
between computerised systems during the data lifecycle.
The use of available technology, suitably configured to reduce data integrity risk,
should be considered.
Page 5 of 21
Simple electronic systems with no configurable software and no electronic data
retention (e.g. pH meters, balances and thermometers) may only require calibration,
whereas complex systems require ‘validation for intended purpose’.
Validation effort increases with complexity and risk (determined by software
functionality, configuration, the opportunity for user intervention and data lifecycle
considerations). It is important not to overlook systems of apparent lower complexity.
Within these systems, it may be possible to manipulate data or repeat testing to
achieve the desired outcome with limited opportunity for detection (e.g. stand-alone
systems with a user-configurable output such as ECG machines, FTIR, UV
spectrophotometers).
Hybrid
Where hybrid systems are used, it should be clearly documented what constitutes the
whole data set and all records that are defined by the data set should be reviewed and
retained. Hybrid systems should be designed to ensure they meet the desired
objective.
Other
Where the data generated is captured by a photograph or imagery (or other media),
the requirements for storage of that format throughout its lifecycle should follow the
same considerations as for the other formats, considering any additional controls
required for that format. Where the original format cannot be retained due to
degradation issues, alternative mechanisms for recording (e.g. photography or
digitisation) and subsequent storage may be considered and the selection rationale
documented (e.g. thin layer chromatography).
4.4 Reduced effort and/or frequency of control measures may be justified for data that has
a lesser impact to product, patient or the environment if those data are obtained from a
process that does not provide the opportunity for amendment without high-level system
access or specialist software/knowledge.
4.5 The data integrity risk assessment (or equivalent) should consider factors required to
follow a process or perform a function. It is expected to consider not only a
computerised system but also the supporting people, guidance, training and quality
systems. Therefore, automation or the use of a ‘validated system' (e.g. e-CRF;
analytical equipment) may lower but not eliminate data integrity risk. Where there is
human intervention, particularly influencing how or what data is recorded, reported or
retained, an increased risk may exist from poor organisational controls or data
verification due to an overreliance on the system's validated state.
4.6 Where the data integrity risk assessment has highlighted areas for remediation,
prioritisation of actions (including acceptance of an appropriate level of residual risk)
should be documented, communicated to management, and subject to review. In
situations where long-term remediation actions are identified, risk-reducing short-term
measures should be implemented to provide acceptable data governance in the
interim.
Page 6 of 21
5. Designing systems and processes to assure data integrity; creating the
‘right environment’.
5.1 Systems and processes should be designed in a way that facilitates compliance with
the principles of data integrity. Enablers of the desired behaviour include but are not
limited to:
 At the point of use, having access to appropriately controlled/synchronised clocks
for recording timed events to ensure reconstruction and traceability, knowing and
specifying the time zone where this data is used across multiple sites.
 Accessibility of records at locations where activities take place so that informal data
recording and later transcription to official records does not occur.
 Access to blank paper proformas for raw/source data recording should be
appropriately controlled. Reconciliation, or the use of controlled books with
numbered pages, may be necessary to prevent recreation of a record. There may
be exceptions such as medical records (GCP) where this is not practical.
 User access rights that prevent (or audit trail, if prevention is not possible)
unauthorised data amendments. Use of external devices or system interfacing
methods that eliminate manual data entries and human interaction with the
computerised system, such as barcode scanners, ID card readers, or printers.
 The provision of a work environment (such as adequate space, sufficient time for
tasks, and properly functioning equipment) that permit performance of tasks and
recording of data as required.
 Access to original records for staff performing data review activities.
 Reconciliation of controlled print-outs.
 Sufficient training in data integrity principles provided to all appropriate staff
(including senior management).
 Inclusion of subject matter experts in the risk assessment process.
 Management oversight of quality metrics relevant to data governance.
5.2 The use of scribes to record activity on behalf of another operator can be considered
where justified, for example:
 The act of contemporaneous recording compromises the product or activity e.g.
documenting line interventions by sterile operators.
 Necropsy (GLP)
 To accommodate cultural or literacy/language limitations, for instance where an
activity is performed by an operator but witnessed and recorded by a second
person.
Consideration should be given to ease of access, usability and location whilst ensuring
appropriate control of the activity guided by the criticality of the data.
In these situations, the recording by the second person should be contemporaneous with the
task being performed, and the records should identify both the person performing the task and
the person completing the record. The person performing the task should countersign the
record wherever possible, although it is accepted that this countersigning step will be
retrospective. The process for supervisory (scribe) documentation completion should be
described in an approved procedure that specifies the activities to which the process applies.
Page 7 of 21
6. Definition of terms and interpretation of requirements
In the following section, definitions where applicable, are given in italic text directly below the
term.
6.1. Data
Facts, figures and statistics collected together for reference or analysis. All original records
and true copies of original records, including source data and metadata and all subsequent
transformations and reports of these data, that are generated or recorded at the time of the
GXP activity and allow full and complete reconstruction and evaluation of the GXP activity.
Data should be:
A - attributable to the person generating the data

L – legible and permanent
C – contemporaneous
O – original record (or certified true copy)
A - accurate
Data governance measures should also ensure that data is complete, consistent, enduring
and available throughout the lifecycle, where;
Complete – the data must be whole; a complete set
Consistent - the data must be self-consistent
Enduring – durable; lasting throughout the data lifecycle
Available – readily available for review or inspection purposes
6.2. Raw data (synonymous with ‘source data’ which is defined in ICH GCP)
Raw data is defined as the original record (data) which can be described as the first-capture of
information, whether recorded on paper or electronically. Information that is originally captured
in a dynamic state should remain available in that state.
Raw data must permit full reconstruction of the activities. Where this has been captured in a
dynamic state and generated electronically, paper copies cannot be considered as ‘raw data’.
In the case of basic electronic equipment that does not store electronic data, or provides only
a printed data output (e.g. balances or pH meters), then the printout constitutes the raw data.
Where the basic electronic equipment does store electronic data permanently and only holds a
certain volume before overwriting; this data should be periodically reviewed and where
necessary reconciled against paper records and extracted as electronic data where this is
supported by the equipment itself.
In all definitions, the term 'data' includes raw data.
Page 8 of 21
6.3. Metadata
Metadata are data that describe the attributes of other data and provide context and meaning.
Typically, these are data that describe the structure, data elements, inter-relationships and
other characteristics of data e.g. audit trails. Metadata also permit data to be attributable to an
individual (or if automatically generated, to the original data source).
Metadata form an integral part of the original record. Without the context provided by metadata
the data has no meaning.
Example (i) 3.5
metadata, giving context and meaning, (italic text) are:
sodium chloride batch 1234, 3.5mg. J Smith 01/Jul/14
Example (ii) 3.5
metadata, giving context and meaning, (italic text) are:
Trial subject A123, sample ref X789 taken 30/06/14 at 1456hrs.

3.5mg. Analyst: J Smith 01/Jul/14
6.4. Data Integrity
Data integrity is the degree to which data are complete, consistent, accurate, trustworthy,
reliable and that these characteristics of the data are maintained throughout the data life cycle.
The data should be collected and maintained in a secure manner, so that they are attributable,
legible, contemporaneously recorded, original (or a true copy) and accurate. Assuring data
integrity requires appropriate quality and risk management systems, including adherence to
sound scientific principles and good documentation practices.
6.5. Data Governance
The arrangements to ensure that data, irrespective of the format in which they are generated,
are recorded, processed, retained and used to ensure the record throughout the data lifecycle.
Data governance should address data ownership and accountability throughout the lifecycle,
and consider the design, operation and monitoring of processes/systems to comply with the
principles of data integrity including control over intentional and unintentional changes to data.
Data Governance systems should include staff training in the importance of data integrity
principles and the creation of a working environment that enables visibility, and actively
encourages reporting of errors, omissions and undesirable results.
Senior management should be accountable for the implementation of systems and procedures
to minimise the potential risk to data integrity, and for identifying the residual risk, using risk
management techniques such as the principles of ICH Q9.
Page 9 of 21
Contract Givers should ensure that data ownership, governance and accessibility are included
in any contract/technical agreement with a third party. The Contract Giver should also perform
a data governance review as part of their vendor assurance programme.
Data governance systems should also ensure that data are readily available and directly
accessible on request from national competent authorities. Electronic data should be available
in human-readable form.
6.6. Data Lifecycle
All phases in the life of the data from generation and recording through processing (including
analysis, transformation or migration), use, data retention, archive/retrieval and destruction.
Data governance, as described in the previous section, must be applied across the whole data
lifecycle to provide assurance of data integrity. Data can be retained either in the original
system, subject to suitable controls, or in an appropriate archive.
6.7. Recording and collection of data
No definition required.
Organisations should have an appropriate level of process understanding and technical

knowledge of systems used for data collection and recording, including their capabilities,
limitations and vulnerabilities.
The selected method should ensure that data of appropriate accuracy, completeness, content
and meaning are collected and retained for their intended use. Where the capability of the
electronic system permits dynamic storage, it is not appropriate for static (printed / manual)
data to be retained in preference to dynamic (electronic) data.
As data are required to allow the full reconstruction of activities the amount and the resolution
(degree of detail) of data to be collected should be justified.
When used, blank forms (including, but not limited to, worksheets, laboratory notebooks, and
master production and control records) should be controlled. For example, numbered sets of
blank forms may be issued and reconciled upon completion. Similarly, bound paginated
notebooks, stamped or formally issued by a document control group allow detection of
unofficial notebooks and any gaps in notebook pages.
6.8. Data transfer / migration
Data transfer is the process of transferring data between different data storage types, formats,
or computerised systems.
Data migration is the process of moving stored data from one durable storage location to
another. This may include changing the format of data, but not the content or meaning.
Data transfer is the process of transferring data and metadata between storage media types or
computerised systems. Data migration where required may, if necessary, change the format of
data to make it usable or visible on an alternative computerised system.
Page 10 of 21
Data transfer/migration procedures should include a rationale, and be robustly designed and
validated to ensure that data integrity is maintained during the data lifecycle. Careful
consideration should be given to understanding the data format and the potential for alteration
at each stage of data generation, transfer and subsequent storage. The challenges of
migrating data are often underestimated, particularly regarding maintaining the full meaning of
the migrated records.
Data transfer should be validated. The data should not be altered during or after it is
transferred to the worksheet or other application. There should be an audit trail for this
process. Appropriate Quality procedures should be followed if the data transfer during the
operation has not occurred correctly. Any changes in the middle layer software should be
managed through appropriate Quality Management Systems.
Electronic worksheets used in automation like paper documentation should be version

controlled and any changes in the worksheet should be documented/verified appropriately.
6.9. Data Processing
A sequence of operations performed on data to extract, present or obtain information in a

defined format. Examples might include: statistical analysis of individual patient data to
present trends or conversion of a raw electronic signal to a chromatogram and subsequently a
calculated numerical result
There should be adequate traceability of any user-defined parameters used within data
processing activities to the raw data, including attribution to who performed the activity.
Audit trails and retained records should allow reconstruction of all data processing activities
regardless of whether the output of that processing is subsequently reported or otherwise
used for regulatory or business purposes. If data processing has been repeated with
progressive modification of processing parameters this should be visible to ensure that the
processing parameters are not being manipulated to achieve a more desirable result.
6.10. Excluding Data (not applicable to GPvP):
Note: this is not applicable to GPvP; for GPvP refer to the pharmacovigilance legislation
(including the GVP modules) which provide the necessary requirements and statutory
guidance.
Data may only be excluded where it can be demonstrated through valid scientific justification
that the data are not representative of the quantity measured, sampled or acquired.
In all cases, this justification should be documented and considered during data review and
reporting. All data (even if excluded) should be retained with the original data set, and be
available for review in a format that allows the validity of the decision to exclude the data to be
confirmed.
6.11. Original record and true copy
6. 11.1. Original record
Page 11 of 21
The first or source capture of data or information e.g. original paper record of manual
observation or electronic raw data file from a computerised system, and all subsequent data
required to fully reconstruct the conduct of the GXP activity. Original records can be Static or
Dynamic.
A static record format, such as a paper or electronic record, is one that is fixed and allows little
or no interaction between the user and the record content. For example, once printed or
converted to static electronic format chromatography records lose the capability of being
reprocessed or enabling more detailed viewing of baselines.
Records in dynamic format, such as electronic records, allow an interactive relationship

between the user and the record content. For example, electronic records in database formats
allow the user to track, trend and query data; chromatography records maintained as
electronic records allow the user or reviewer (with appropriate access permissions) to
reprocess the data and expand the baseline to view the integration more clearly.
Where it is not practical or feasibly possible to retain the original copy of source data, (e.g.
MRI scans, where the source machine is not under the study sponsor's control and the
operator can only provide summary statistics) the risks and mitigation should be documented.
Where the data obtained requires manual observation to record (for example results of a
manual titration, visual interpretation of environmental monitoring plates) the process should
be risk assessed and depending on the criticality, justify if a second contemporaneous
verification check is required or investigate if the result could be captured by an alternate
means.
6.11.2. True copy
A copy (irrespective of the type of media used) of the original record that has been verified (i.e.
by a dated signature or by generation through a validated process) to have the same
information, including data that describe the context, content, and structure, as the original.
A true copy may be stored in a different electronic file format to the original record if required,
but must retain the metadata and audit trail required to ensure that the full meaning of the data
are kept and its history may be reconstructed.
Original records and true copies must preserve the integrity of the record. True copies of
original records may be retained in place of the original record (e.g. scan of a paper record), if
a documented system is in place to verify and record the integrity of the copy. Organisations
should consider any risk associated with the destruction of original records.
It should be possible to create a true copy of electronic data, including relevant metadata, for
the purposes of review, backup and archival. Accurate and complete copies for certification of
the copy should include the meaning of the data (e.g. date formats, context, layout, electronic
signatures and authorisations) and the full GXP audit trail. Consideration should be given to
the dynamic functionality of a ‘true copy’ throughout the retention period (see ‘archive’).
Data must be retained in a dynamic form where this is critical to its integrity or later
verification. If the computerised system cannot be maintained e.g., if it is no longer supported,
then records should be archived according to a documented archiving strategy prior to
Page 12 of 21
decommissioning the computerised system. It is conceivable for some data generated by
electronic means to be retained in an acceptable paper or electronic format, where it can be
justified that a static record maintains the integrity of the original data. However, the data
retention process must be shown to include verified copies of all raw data, metadata, relevant
audit trail and result files, any variable software/system configuration settings specific to each
record, and all data processing runs (including methods and audit trails) necessary for
reconstruction of a given raw data set. It would also require a documented means to verify
that the printed records were an accurate representation. To enable a GXP compliant record
this approach is likely to be demanding in its administration.
Where manual transcriptions occur, these should be verified by a second person or validated
system.
6.12. Computerised system transactions:
A computerised system transaction is a single operation or sequence of operations performed

as a single logical ‘unit of work’. The operation(s) that makes a transaction may not be saved
as a permanent record on durable storage until the user commits the transaction through a
deliberate act (e.g. pressing a save button), or until the system forces the saving of data.
The metadata (e.g. username, date, and time) are not captured in the system audit trail until
the user saves the transaction to durable storage. In computerised systems, an electronic
signature may be required for the record to be saved and become permanent.
A critical step is a parameter that must be within an appropriate limit, range, or distribution to
ensure the safety of the subject or quality of the product or data. Computer systems should be
designed to ensure that the execution of critical steps is recorded contemporaneously. Where
transactional systems are used, the combination of multiple unit operations into a combined
single transaction should be avoided, and the time intervals before saving of data should be
minimised. Systems should be designed to require saving data to permanent memory before
prompting users to make changes.
The organisation should define during the development of the system (e.g. via the user
requirements specification) what critical steps are appropriate based on the functionality of the
system and the level of risk associated. Critical steps should be documented with process
controls that consider system design (prevention), together with monitoring and review
processes. Oversight of activities should alert to failures that are not addressed by the process
design.
6.13. Audit Trail
The audit trail is a form of metadata containing information associated with actions that relate
to the creation, modification or deletion of GXP records. An audit trail provides for secure
recording of life-cycle details such as creation, additions, deletions or alterations of information
in a record, either paper or electronic, without obscuring or overwriting the original record. An
audit trail facilitates the reconstruction of the history of such events relating to the record
regardless of its medium, including the “who, what, when and why” of the action.
Where computerised systems are used to capture, process, report, store or archive raw data
electronically, system design should always provide for the retention of audit trails to show all
Page 13 of 21
changes to, or deletion of data while retaining previous and original data. It should be possible
to associate all data and changes to data with the persons making those changes, and
changes should be dated and time stamped (time and time zone where applicable). The
reason for any change, should also be recorded. The items included in the audit trail should be
those of relevance to permit reconstruction of the process or activity.
Audit trails (identified by risk assessment as required) should be switched on. Users should
not be able to amend or switch off the audit trail. Where a system administrator amends, or
switches off the audit trail a record of that action should be retained.
The relevance of data retained in audit trails should be considered by the organisation to
permit robust data review/verification. It is not necessary for audit trail review to include every
system activity (e.g. user log on/off, keystrokes etc.).
Where relevant audit trail functionality does not exist (e.g. within legacy systems) an
alternative control may be achieved for example defining the process in an SOP, and use of
log books. Alternative controls should be proven to be effective.
Where add-on software or a compliant system does not currently exist, continued use of the
legacy system may be justified by documented evidence that a compliant solution is being
sought and that mitigation measures temporarily support the continued use. 1
Routine data review should include a documented audit trail review where this is determined
by a risk assessment. When designing a system for review of audit trails, this may be limited
to those with GXP relevance. Audit trails may be reviewed as a list of relevant data, or by an
‘exception reporting' process. An exception report is a validated search tool that identifies and
documents predetermined ‘abnormal’ data or actions, that require further attention or
investigation by the data reviewer.
Reviewers should have sufficient knowledge and system access to review relevant audit trails,
raw data and metadata (see also ‘data governance’).
Where systems do not meet the audit trail and individual user account expectations,
demonstrated progress should be available to address these shortcomings. This should either
be through add-on software that provides these additional functions or by an upgrade to a
compliant system. Where remediation has not been identified or subsequently implemented in
a timely manner a deficiency may be cited.
6.14. Electronic signatures
A signature in digital form (bio-metric or non-biometric) that represents the signatory. This
should be equivalent in legal terms to the handwritten signature of the signatory.
The use of electronic signatures should be appropriately controlled with consideration given to:
 How the signature is attributable to an individual.
1It is expected that GMP facilities with industrial automation and control equipment/ systems such as programmable logic
controllers should be able to demonstrate working towards system upgrades with individual login and audit trails (reference:
Art 23 of Directive 2001/83/EC).
Page 14 of 21
 How the act of ‘signing’ is recorded within the system so that it cannot be altered or
manipulated without invalidating the signature or status of the entry.
 How the record of the signature will be associated with the entry made and how this
can be verified.
 The security of the electronic signature i.e. so that it can only be applied by the ‘owner’
of that signature.
It is expected that appropriate validation of the signature process associated with a system is
undertaken to demonstrate suitability and that control over signed records is maintained.
Where a paper or pdf copy of an electronically signed document is produced, the metadata
associated with an electronic signature should be maintained with the associated document.
The use of electronic signatures should be compliant with the requirements of international
standards. The use of advanced electronic signatures should be considered where this
method of authentication is required by the risk assessment. Electronic signature or
E-signature systems must provide for “signature manifestations” i.e. a display within the
viewable record that defines who signed it, their title, and the date (and time, if significant) and
the meaning of the signature (e.g. verified or approved).
An inserted image of a signature or a footnote indicating that the document has been
electronically signed (where this has been entered by a means other than the validated
electronic signature process) is not adequate. Where a document is electronically signed then
the metadata associated with the signature should be retained.
For printed copies of electronically signed documents refer to True Copy section.
Expectations for electronic signatures associated with informed consent (GCP) are covered in
alternative guidance (MHRA/HRA DRAFT Guidance on the use of electronic consent).
6.15. Data review and approval
The approach to reviewing specific record content, such as critical data and metadata, cross-
outs (paper records) and audit trails (electronic records) should meet all applicable regulatory
requirements and be risk-based.
There should be a procedure that describes the process for review and approval of data. Data
review should also include a risk-based review of relevant metadata, including relevant audit
trails records. Data review should be documented and the record should include a positive
statement regarding whether issues were found or not, the date that review was performed
and the signature of the reviewer.
A procedure should describe the actions to be taken if data review identifies an error or
omission. This procedure should enable data corrections or clarifications to provide visibility of
the original record, and traceability of the correction, using ALCOA principles (see ‘data’
definition).
Where data review is not conducted by the organisation that generated the data, the
responsibilities for data review must be documented and agreed by both parties. Summary
Page 15 of 21
reports of data are often supplied between organisations (contract givers and acceptors). It
must be acknowledged that summary reports are limited and critical supporting data and
metadata may not be included.
Many software packages allow configuration of customised reports. Key actions may be
incorporated into such reports provided they are validated and locked to prevent changes.
Automated reporting tools and reports may reduce the checks required to assure the integrity
of the data.
Where summary reports are supplied by a different organisation, the organisation receiving
and using the data should evaluate the data provider’s data integrity controls and processes
prior to using the information.
 Routine data review should consider the integrity of an individual data set e.g. is this the
only data generated as part of this activity? Has the data been generated and maintained
correctly? Are there indicators of unauthorised changes?
 Periodic audit of the data generated (encompassing both a review of electronically

generated data and the broader organisational review) might verify the effectiveness of
existing control measures and consider the possibility of unauthorised activity at all
interfaces, e.g. have there been IT requests to amend any data post review? Have there
been any system maintenance activities and has the impact of that activity been
assessed?
6.16. Computerised system user access/system administrator roles
Full use should be made of access controls to ensure that people have access only to
functionality that is appropriate for their job role, and that actions are attributable to a specific
individual. Companies must be able to demonstrate the access levels granted to individual
staff members and ensure that historical information regarding user access level is available.
Where the system does not capture this data, then a record must be maintained outside of the
system. Access controls should be applied to both the operating system and application
levels. Individual login at operating system level may not be required if appropriate controls
are in place to ensure data integrity (e.g. no modification, deletion or creation of data outside
the application is possible).
For systems generating, amending or storing GXP data shared logins or generic user access
should not be used. Where the computerised system design supports individual user access,
this function must be used. This may require the purchase of additional licences. Systems
(such as MRP systems) that are not used in their entirety for GXP purposes but do have
elements within them, such as approved suppliers, stock status, location and transaction
histories that are GXP applicable require appropriate assessment and control.
It is acknowledged that some computerised systems support only a single user login or limited
numbers of user logins. Where no suitable alternative computerised system is available,
equivalent control may be provided by third-party software or a paper-based method of
providing traceability (with version control). The suitability of alternative systems should be
justified and documented. Increased data review is likely to be required for hybrid systems
Page 16 of 21
because they are vulnerable to non-attributable data changes. It is expected that companies
should be implementing systems that comply with current regulatory expectations2.
System administrator access should be restricted to the minimum number of people possible
taking account of the size and nature of the organisation. The generic system administrator
account should not be available for routine use. Personnel with system administrator access
should log in with unique credentials that allow actions in the audit trail(s) to be attributed to a
specific individual. The intent of this is to prevent giving access to users with potentially a
conflict of interest so that they can make unauthorised changes that would not be traceable to
that person.
System Administrator rights (permitting activities such as data deletion, database amendment
or system configuration changes) should not be assigned to individuals with a direct interest in
the data (data generation, data review or approval).
Individuals may require changes in their access rights depending on the status of clinical trial
data. For example, once data management processes are complete, the data is ‘locked’ by
removing editing access rights. This should be able to be demonstrated within the system.
6.17. Data retention
Data retention may be for archiving (protected data for long-term storage) or backup (data for
the purposes of disaster recovery).
Data and document retention arrangements should ensure the protection of records from
deliberate or inadvertent alteration or loss. Secure controls must be in place to ensure the data
integrity of the record throughout the retention period and should be validated where
appropriate (see also data transfer/migration).
Data (or a true copy) generated in paper format may be retained by using a validated scanning
process provided there is a documented process in place to ensure that the outcome is a true
copy.
Procedures for destruction of data should consider data criticality and where applicable
legislative retention requirements.
2It is expected that GMP facilities with industrial automation and control equipment/ systems such as programmable logic
controllers should be able to demonstrate working towards system upgrades with individual login and audit trails (reference:
Art 23 of Directive 2001/83/EC).
Page 17 of 21
6.17.1. Archive
A designated secure area or facility (e.g. cabinet, room, building or computerised system) for
the long term, retention of data and metadata for the purposes of verification of the process or
activity.
Archived records may be the original record or a ‘true copy’ and should be protected so they
cannot be altered or deleted without detection and protected against any accidental damage
such as fire or pest.
Archive arrangements must be designed to permit recovery and readability of the data and
metadata throughout the required retention period. In the case of archiving of electronic data,
this process should be validated, and in the case of legacy systems the ability to review data
periodically verified (i.e. to confirm the continued support of legacy computerised systems).
Where hybrid records are stored, references between physical and electronic records must be
maintained such that full verification of events is possible throughout the retention period.
When legacy systems can no longer be supported, consideration should be given to

maintaining the software for data accessibility purposes (for as long possible depending upon
the specific retention requirements). This may be achieved by maintaining software in a virtual
environment.
Migration to an alternative file format that retains as much as possible of the ‘true copy’
attributes of the data may be necessary with increasing age of the legacy data. Where
migration with full original data functionality is not technically possible, options should be
assessed based on risk and the importance of the data over time. The migration file format
should be selected considering the balance of risk between long-term accessibility versus the
possibility of reduced dynamic data functionality (e.g. data interrogation, trending, re-
processing etc). It is recognised that the need to maintain accessibility may require migration
to a file format that loses some attributes and/or dynamic data functionality (see also ‘Data
Migration’).
6.17.2. Backup
A copy of current (editable) data, metadata and system configuration settings maintained for
recovery including disaster recovery.
Backup and recovery processes should be validated and periodically tested. Each back up
should be verified to ensure that it has functioned correctly e.g. by confirming that the data
size transferred matches that of the original record.
The backup strategies for the data owners should be documented.
Backups for recovery purposes do not replace the need for the long term, retention of data
and metadata in its final form for the purposes of verification of the process or activity.
Page 18 of 21
6.18. File structure
Data Integrity risk assessment requires a clear understanding of file structure. The way data is
structured within the GXP environment will depend on what the data will be used for and the
end user may have this dictated to them by the software/computerised system(s) available.
There are many types of file structure, the most common being flat files and relational
databases.
Different file structures due to their attributes may require different controls and data review
methods and may retain meta data in different ways.
6.19. Validation – for intended purpose (GMP; See also Annex 11, 15)
Computerised systems should comply with regulatory requirements and associated guidance.
These should be validated for their intended purpose which requires an understanding of the
computerised system’s function within a process. For this reason, the acceptance of vendor-
supplied validation data in isolation of system configuration and users intended use is not
acceptable. In isolation from the intended process or end-user IT infrastructure, vendor testing
is likely to be limited to functional verification only and may not fulfil the requirements for
performance qualification.
Functional verification demonstrates that the required information is consistently and

completely presented. Validation for intended purpose ensures that the steps for generating
the custom report accurately reflect those described in the data checking SOP and that the
report output is consistent with the procedural steps for performing the subsequent review.
6.20. IT Suppliers and Service Providers (including Cloud providers and virtual
service/platforms (also referred to as software as a service SaaS/platform as a service
(PaaS) / infrastructure as a service (IaaS)).
Where ‘cloud’ or ‘virtual’ services are used, attention should be paid to understanding the
service provided, ownership, retrieval, retention and security of data.
The physical location where the data is held, including the impact of any laws applicable to
that geographic location, should be considered.
The responsibilities of the contract giver and acceptor should be defined in a technical
agreement or contract. This should ensure timely access to data (including metadata and audit
trails) to the data owner and national competent authorities upon request. Contracts with
providers should define responsibilities for archiving and continued readability of the data
throughout the retention period (see archive).
Appropriate arrangements must exist for the restoration of the software/system as per its
original validated state, including validation and change control information to permit this
restoration.
Business continuity arrangements should be included in the contract, and tested. The need for
an audit of the service provider should be based upon risk.
Page 19 of 21
7. Glossary
Acronym or word or phrase Definition

eCRF Electronic Case Report Form
ECG Electrocardiogram
GXP Good ‘X’ Practice where ‘X’ is used as a collective term for
GDP – Good Distribution Practice,
GCP – Good Clinical practice,
GLP – Good Laboratory Practice
GMP – Good Manufacturing Practice
GPvP – Good Pharmacovigilance Practice
Data Quality The assurance that data produced is exactly what was intended to
be produced and fit for its intended purpose. This incorporates
ALCOA
ALCOA Acronym referring to Attributable, Legible, Contemporaneous,
Original and Accurate.
ALCOA + Acronym referring to Attributable, Legible, Contemporaneous,
Original and Accurate ‘plus’ Complete, Consistent, Enduring, and
Available.
DIRA Data Integrity Risk Assessment
Terminology The body of terms used with a particular technical application in a
subject of study, profession, etc.
Data cleaning The process of detecting and correcting (or removing) corrupt or
inaccurate records from a record set, table, or database and refers to
identifying incomplete, incorrect, inaccurate or irrelevant parts of the
data and then replacing, modifying, or deleting the dirty or coarse
data.
Format The something is arranged or set out
Directly accessible At once; without delay
Procedures Written instructions or other documentation describing process i.e.
standard operating procedures (SOP)
Advanced electronic an electronic signature based upon cryptographic methods of
signatures originator authentication, computed by using a set of rules and a set
of parameters such that the identity of the signer and the integrity of
the data can be verified.
Validated scanning process A process whereby documents / items are scanned as a process
with added controls such as location identifiers and OCR so that
each page duplicated does not have to be further checked by a
human.
Page 20 of 21
8. References
Computerised systems. In: The rules governing medicinal products in the European Union.
Volume 4: Good manufacturing practice (GMP) guidelines: Annex 11. Brussels: European
Commission.
(http://ec.europa.eu/enterprise/pharmaceuticals/eudralex/vol-4/pdfs-en/anx11en.pdf).
OECD series on principles of good laboratory practice (GLP) and compliance monitoring. Paris:
Organisation for Economic Co-operation and Development.
(http://www.oecd.org/chemicalsafety/testing/oecdseriesonprinciplesofgoodlaboratorypracticeglp
andcompliancemonitoring.htm).
Good Clinical Practice (GCP) ICH E6(R2) November 2016

(http://www.ich.org/products/guidelines/efficacy/article/efficacy-guidelines.html).
Guidance on good data and record management practices; World Health Organisation, WHO
Technical Report Series, No.996, Annex 5; 2016.
(http://apps.who.int/medicinedocs/en/m/abstract/Js22402en/).
Good Practices For Data Management And Integrity In Regulated GMP/GDP Environments –
PIC/S; PI041-1(draft 2); August 2016.
(https://picscheme.org/en/news?itemid=33).
MHRA GMP data integrity definitions and guidance for industry. London: Medicines and
Healthcare Products Regulatory Agency; March 2015.
(https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/412735/Data_in
tegrity_definitions_and_guidance_v2.pdf).
MHRA/HRA DRAFT Guidance on the use of electronic consent

(http://www.hra-decisiontools.org.uk/consent/)
EU Pharmacovigilance legislation: http://ec.europa.eu/health/human-use/pharmacovigilance
The Human Medicines Regulations 2012 (Statutory Instrument 2012 No. 1916):
http://www.legislation.gov.uk/uksi/2012/1916/contents/made
EU Good Pharmacovigilance Practice Modules:

http://www.ema.europa.eu/ema/index.jsp?curl=pages/regulation/document_listing/document_li
sting_000345.jsp&mid=WC0b01ac058058f32c
Revision History
Revision Publication Month Reason for changes
Revision 1 March 2018 None. First issue.
Page 21 of 21
Appendices C: FDA Data Integrity and Compliance
with CGMP Guidance
Data Integrity and
Compliance With
CGMP
Guidance for Industry
DRAFT GUIDANCE
This guidance document is being distributed for comment purposes only.
Comments and suggestions regarding this draft document should be submitted within 60 days of
publication in the Federal Register of the notice announcing the availability of the draft
guidance. Submit electronic comments to http://www.regulations.gov. Submit written comments
to the Division of Dockets Management (HFA-305), Food and Drug Administration, 5630
Fishers Lane, rm. 1061, Rockville, MD 20852. All comments should be identified with the
docket number listed in the notice of availability that publishes in the Federal Register.
For questions regarding this draft document, contact (CDER) Karen Takahashi 301-796-3191;
(CBER) Office of Communication, Outreach and Development, 800-835-4709 or 240-402-8010;
or (CVM) Jonathan Bray 240-402-5623.
U.S. Department of Health and Human Services

Food and Drug Administration
Center for Drug Evaluation and Research (CDER)
Center for Biologics Evaluation and Research (CBER)
Center for Veterinary Medicine (CVM)
April 2016
Pharmaceutical Quality/Manufacturing Standards (CGMP)
Data Integrity and
Compliance With
CGMP
Additional copies are available from:
Office of Communications, Division of Drug Information
Center for Drug Evaluation and Research
10001 New Hampshire Ave., Hillandale Bldg., 4th Floor
Silver Spring, MD 20993-0002
Phone: 855-543-3784 or 301-796-3400; Fax: 301-431-6353
Email: druginfo@fda.hhs.gov
http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/default.htm
and/or
Office of Communication, Outreach and Development
Center for Biologics Evaluation and Research
10903 New Hampshire Ave., Bldg. 71, Room 3128
Phone: 800-835-4709 or 240-402-8010
Email: ocod@fda.hhs.gov
http://www.fda.gov/BiologicsBloodVaccines/GuidanceComplianceRegulatoryInformation/Guidances/default.htm
and/or
Policy and Regulations Staff, HFV-6
Center for Veterinary Medicine
7519 Standish Place, Rockville, MD 20855
http://www.fda.gov/AnimalVeterinary/GuidanceComplianceEnforcement/GuidanceforIndustry/default.htm

Center for Veterinary Medicine (CVM)
April 2016
Pharmaceutical Quality/Manufacturing Standards (CGMP)
Contains Nonbinding Recommendations
Draft — Not for Implementation
TABLE OF CONTENTS
I. INTRODUCTION............................................................................................................. 1
II. BACKGROUND ............................................................................................................... 1
III. QUESTIONS AND ANSWERS ....................................................................................... 2
1. Please clarify the following terms as they relate to CGMP records: ......................................... 2
a. What is “data integrity”? ................................................................................................................ 2
b. What is “metadata”? ....................................................................................................................... 3
c. What is an “audit trail”? ................................................................................................................. 3
d. How does FDA use the terms “static” and “dynamic” as they relate to record formats? ............. 3
e. How does FDA use the term “backup” in § 211.68(b)? .................................................................. 4
f. What are the “systems” in “computer or related systems” in § 211.68?........................................ 4
2. When is it permissible to exclude CGMP data from decision making? .................................... 4
3. Does each workflow on our computer system need to be validated? ........................................ 4
4. How should access to CGMP computer systems be restricted? ................................................ 5
5. Why is FDA concerned with the use of shared login accounts for computer systems? ........... 6
6. How should blank forms be controlled? ...................................................................................... 6
7. How often should audit trails be reviewed?................................................................................. 6
8. Who should review audit trails? ................................................................................................... 6
9. Can electronic copies be used as accurate reproductions of paper or electronic records? ..... 7
10. Is it acceptable to retain paper printouts or static records instead of original electronic
records from stand-alone computerized laboratory instruments, such as an FT-IR instrument? . 7
11. Can electronic signatures be used instead of handwritten signatures for master production
and control records?............................................................................................................................... 8
12. When does electronic data become a CGMP record? ................................................................ 8
13. Why has the FDA cited use of actual samples during “system suitability” or test, prep, or
equilibration runs in warning letters? .................................................................................................. 9
14. Is it acceptable to only save the final results from reprocessed laboratory
chromatography? ................................................................................................................................... 9
15. Can an internal tip regarding a quality issue, such as potential data falsification, be handled
informally outside of the documented CGMP quality system? .......................................................... 9
16. Should personnel be trained in detecting data integrity issues as part of a routine CGMP
training program? ................................................................................................................................ 10
17. Is the FDA investigator allowed to look at my electronic records? ......................................... 10
18. How does FDA recommend data integrity problems identified during inspections, in
warning letters, or in other regulatory actions be addressed? ......................................................... 10
1 Data Integrity and Compliance With CGMP

2 Guidance for Industry 1
3
4
5 This draft guidance, when finalized, will represent the current thinking of the Food and Drug
6 Administration (FDA or Agency) on this topic. It does not establish any rights for any person and is not
7 binding on FDA or the public. You can use an alternative approach if it satisfies the requirements of the
8 applicable statutes and regulations. To discuss an alternative approach, contact the FDA staff responsible
9 for this guidance as listed on the title page.
10
11
12
13
14 I. INTRODUCTION
15
16 The purpose of this guidance is to clarify the role of data integrity in current good manufacturing
17 practice (CGMP) for drugs, as required in 21 CFR parts 210, 211, and 212. Part 210 covers
18 Current Good Manufacturing Practice in Manufacturing, Processing, Packing, or Holding of
19 Drugs; General; part 211 covers Current Good Manufacturing Practice for Finished
20 Pharmaceuticals; and part 212 covers Current Good Manufacturing Practice for Positron
21 Emission Tomography Drugs. This guidance provides the Agency’s current thinking on the
22 creation and handling of data in accordance with CGMP requirements.
23
24 FDA expects that data be reliable and accurate (see the “Background” section). CGMP
25 regulations and guidance allow for flexible and risk-based strategies to prevent and detect data
26 integrity issues. Firms should implement meaningful and effective strategies to manage their data
27 integrity risks based upon their process understanding and knowledge management of
28 technologies and business models.
29
30 In general, FDA’s guidance documents do not establish legally enforceable responsibilities.
31 Instead, guidances describe the Agency’s current thinking on a topic and should be viewed only
32 as recommendations, unless specific regulatory or statutory requirements are cited. The use of
33 the word should in Agency guidances means that something is suggested or recommended, but
34 not required.
35
36 II. BACKGROUND
37
38 In recent years, FDA has increasingly observed CGMP violations involving data integrity during
39 CGMP inspections. This is troubling because ensuring data integrity is an important component
40 of industry’s responsibility to ensure the safety, efficacy, and quality of drugs, and of FDA’s
41 ability to protect the public health. These data integrity-related CGMP violations have led to
1
This guidance has been prepared by the Office of Pharmaceutical Quality and the Office of Compliance in the
Center for Drug Evaluation and Research in cooperation with the Center for Biologics Evaluation and Research, the
Center for Veterinary Medicine, and the Office of Regulatory Affairs at the Food and Drug Administration.
1
42 numerous regulatory actions, including warning letters, import alerts, and consent decrees. The
43 underlying premise in §§ 210.1 and 212.2 is that CGMP sets forth minimum requirements to
44 assure that drugs meet the standards of the Federal Food, Drug, and Cosmetic Act (FD&C Act)
45 regarding safety, identity, strength, quality, and purity. 2 Requirements with respect to data
46 integrity in parts 211 and 212 include, among other things:
47
48 • § 211.68 (requiring that “backup data are exact and complete,” and “secure from
49 alteration, inadvertent erasures, or loss”);
50 • § 212.110(b) (requiring that data be “stored to prevent deterioration or loss”);
51 • §§ 211.100 and 211.160 (requiring that certain activities be “documented at the time
52 of performance” and that laboratory controls be “scientifically sound”);
53 • § 211.180 (requiring that records be retained as “original records,” “true copies,” or
54 other “accurate reproductions of the original records”); and
55 • §§ 211.188, 211.194, and 212.60(g) (requiring “complete information,” “complete
56 data derived from all tests,” “complete record of all data,” and “complete records of
57 all tests performed”).
58
59 Electronic signature and record-keeping requirements are laid out in 21 CFR part 11 and apply to
60 certain records subject to records requirements set forth in Agency regulations, including parts
61 210, 211, and 212. For more information, see guidance for industry Part 11, Electronic Records;
62 Electronic Signatures — Scope and Application. 3 The guidance outlines FDA’s current thinking
63 regarding the narrow scope and application of part 11 pending FDA’s reexamination of part 11
64 as it applies to all FDA-regulated products.
65
66 III. QUESTIONS AND ANSWERS
67
68 1. Please clarify the following terms as they relate to CGMP records:
69
70 a. What is “data integrity”?
71
72 For the purposes of this guidance, data integrity refers to the completeness,
73 consistency, and accuracy of data. Complete, consistent, and accurate data should
74 be attributable, legible, contemporaneously recorded, original or a true copy, and
75 accurate (ALCOA). 4
2
FDA’s authority for CGMP comes from FD&C Act section 501(a)(2)(B), which states that a drug shall be deemed
adulterated if “the methods used in, or the facilities or controls used for, its manufacture, processing, packing, or
holding do not conform to or are not operated or administered in conformity with current good manufacturing
practice to assure that such drug meets the requirement of the act as to safety and has the identity and strength, and
meets the quality and purity characteristics, which it purports or is represented to possess.”
3
CDER updates guidances periodically. To make sure you have the most recent version of a guidance, check the
FDA Drugs guidance Web page at
www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/default.htm.
4
For attributable, see §§ 211.101(d), 211.122, 211.186, 211.188(b)(11), and 212.50(c)(10); for legible see §§
211.180(e) and 212.110(b); for contemporaneously recorded (at the time of performance) see §§ 211.100(b) and
211.160(a); for original or a true copy see §§ 211.180 and 211.194(a); and for accurate see §§ 211.22(a), 211.68,
211.188, and 212.60(g).
2
76 b. What is “metadata”?
77
78 Metadata is the contextual information required to understand data. A data value
79 is by itself meaningless without additional information about the data. Metadata is
80 often described as data about data. Metadata is structured information that
81 describes, explains, or otherwise makes it easier to retrieve, use, or manage data.
82 For example, the number “23” is meaningless without metadata, such as an
83 indication of the unit “mg.” Among other things, metadata for a particular piece
84 of data could include a date/time stamp for when the data were acquired, a user ID
85 of the person who conducted the test or analysis that generated the data, the
86 instrument ID used to acquire the data, audit trails, etc.
87
88 Data should be maintained throughout the record’s retention period with all
89 associated metadata required to reconstruct the CGMP activity (e.g., §§ 211.188
90 and 211.194). The relationships between data and their metadata should be
91 preserved in a secure and traceable manner.
92
93 c. What is an “audit trail”?
94
95 For purposes of this guidance, audit trail means a secure, computer-generated,
96 time-stamped electronic record that allows for reconstruction of the course of
97 events relating to the creation, modification, or deletion of an electronic record.
98 An audit trail is a chronology of the “who, what, when, and why” of a record. For
99 example, the audit trail for a high performance liquid chromatography (HPLC)
100 run could include the user name, date/time of the run, the integration parameters
101 used, and details of a reprocessing, if any, including change justification for the
102 reprocessing.
103
104 Electronic audit trails include those that track creation, modification, or deletion
105 of data (such as processing parameters and results) and those that track actions at
106 the record or system level (such as attempts to access the system or rename or
107 delete a file).
108
109 CGMP-compliant record-keeping practices prevent data from being lost or
110 obscured (see §§ 211.160(a), 211.194, and 212.110(b)). Electronic record-keeping
111 systems, which include audit trails, can fulfill these CGMP requirements.
112
113 d. How does FDA use the terms “static” and “dynamic” as they relate to record
114 formats?
115
116 For the purposes of this guidance, static is used to indicate a fixed-data document
117 such as a paper record or an electronic image, and dynamic means that the record
118 format allows interaction between the user and the record content. For example, a
119 dynamic chromatographic record may allow the user to change the baseline and
120 reprocess chromatographic data so that the resulting peaks may appear smaller or
3
121 larger. It also may allow the user to modify formulas or entries in a spreadsheet
122 used to compute test results or other information such as calculated yield.
123
124 e. How does FDA use the term “backup” in § 211.68(b)?
125
126 FDA uses the term backup in § 211.68(b) to refer to a true copy of the original
127 data that is maintained securely throughout the records retention period (for
128 example, § 211.180). The backup file should contain the data (which includes
129 associated metadata) and should be in the original format or in a format
130 compatible with the original format.
131
132 This should not be confused with backup copies that may be created during
133 normal computer use and temporarily maintained for disaster recovery (e.g., in
134 case of a computer crash or other interruption). Such temporary backup copies
135 would not satisfy the requirement in § 211.68(b) to maintain a backup file of data.
136
137 f. What are the “systems” in “computer or related systems” in § 211.68?
138
139 The American National Standards Institute (ANSI) defines systems as people,
140 machines, and methods organized to accomplish a set of specific functions. 5
141 Computer or related systems can refer to computer hardware, software, peripheral
142 devices, networks, cloud infrastructure, operators, and associated documents (e.g.,
143 user manuals and standard operating procedures).
144
145 2. When is it permissible to exclude CGMP data from decision making?
146
147 Any data created as part of a CGMP record must be evaluated by the quality unit as part
148 of release criteria (see §§ 211.22 and 212.70) and maintained for CGMP purposes (e.g., §
149 211.180). Electronic data generated to fulfill CGMP requirements should include relevant
150 metadata. To exclude data from the release criteria decision-making process, there must
151 be a valid, documented, scientific justification for its exclusion (see the guidance for
152 industry Investigating Out-of-Specification (OOS) Test Results for Pharmaceutical
153 Production, and §§ 211.188, 211.192, and 212.71(b)). The requirements for record
154 retention and review do not differ depending on the data format; paper-based and
155 electronic data record-keeping systems are subject to the same requirements.
156
157 3. Does each workflow on our computer system need to be validated?
158
159 Yes, a workflow, such as creation of an electronic master production and control record
160 (MPCR), is an intended use of a computer system to be checked through validation (see
161 §§ 211.63, 211.68(b), and 211.110(a)). If you validate the computer system, but you do
5
American National Standard for Information Systems, Dictionary for Information Systems, American National
Standards Institute, 1991.
4
162 not validate it for its intended use, you cannot know if your workflow runs correctly. 6 For
163 example, qualifying the Manufacturing Execution System (MES) platform, a computer
164 system, ensures that it meets specifications; however, it does not demonstrate that a given
165 MPCR generated by the MES contains the correct calculations. In this example,
166 validating the workflow ensures that the intended steps, specifications, and calculations
167 in the MPCR are accurate. This is similar to reviewing a paper MPCR and ensuring all
168 supporting procedures are in place before the MPCR is implemented in production (see
169 §§ 211.100, 211.186, and 212.50(b), and the guidance for industry PET Drugs — Current
170 Good Manufacturing Practice (CGMP)).
171
172 FDA recommends you implement appropriate controls to manage risks associated with
173 each element of the system. Controls that are appropriately designed to validate a system
174 for its intended use address software, hardware, personnel, and documentation.
175
176 4. How should access to CGMP computer systems be restricted?
177
178 You must exercise appropriate controls to assure that changes to computerized MPCRs,
179 or other records, or input of laboratory data into computerized records, can be made only
180 by authorized personnel (§ 211.68(b)). FDA recommends that you restrict the ability to
181 alter specifications, process parameters, or manufacturing or testing methods by technical
182 means where possible (for example, by limiting permissions to change settings or data).
183 FDA suggests that the system administrator role, including any rights to alter files and
184 settings, be assigned to personnel independent from those responsible for the record
185 content. To assist in controlling access, FDA recommends maintaining a list of
186 authorized individuals and their access privileges for each CGMP computer system in
187 use.
188
189 If these independent security role assignments are not practical for small operations or
190 facilities with few employees, such as PET or medical gas facilities, FDA recommends
191 alternate control strategies be implemented. 7 For example, in the rare instance that the
192 same person is required to hold the system administrator role and to be responsible for
193 the content of the records, FDA suggests having a second person review settings and
194 content. If second-person review is not possible, the Agency recommends that the person
195 recheck settings and his or her own work.
196
6
In computer science, validation refers to ensuring that software meets its specifications. However, this may not
meet the definition of process validation as found in guidance for industry Process Validation: General Principles
and Practices: “The collection and evaluation of data … which establishes scientific evidence that a process is
capable of consistently delivering quality products.” See also ICH guidance for industry Q7A Good Manufacturing
Practice Guide for Active Pharmaceutical Ingredients, which defines validation as providing assurance that a
specific process, method, or system will consistently produce a result meeting predetermined acceptance criteria. For
purposes of this guidance, validation is being used in a manner consistent with the above guidance documents.
7
For further discussion of such alternate control strategies, see the guidance for industry PET Drugs — Current
Good Manufacturing Practice (CGMP).
5
197 5. Why is FDA concerned with the use of shared login accounts for computer
198 systems?
199
200 You must exercise appropriate controls to assure that only authorized personnel make
201 changes to computerized MPCRs, or other records, or input laboratory data into
202 computerized records, and you must implement documentation controls that ensure
203 actions are attributable to a specific individual (see §§ 211.68(b), 211.188(b)(11),
204 211.194(a)(7) and (8), and 212.50(c)(10)). When login credentials are shared, a unique
205 individual cannot be identified through the login and the system would thus not conform
206 to the CGMP requirements in parts 211 and 212. FDA requires that systems controls,
207 including documentation controls, be designed to follow CGMP to assure product quality
208 (for example, §§ 211.100 and 212.50).
209
210 6. How should blank forms be controlled?
211
212 There must be document controls in place to assure product quality (see §§ 211.100,
213 211.160(a), 211.186, 212.20(d), and 212.60(g)). FDA recommends that, if used, blank
214 forms (including, but not limited to, worksheets, laboratory notebooks, and MPCRs) be
215 controlled by the quality unit or by another document control method. For example,
216 numbered sets of blank forms may be issued as appropriate and should be reconciled
217 upon completion of all issued forms. Incomplete or erroneous forms should be kept as
218 part of the permanent record along with written justification for their replacement (for
219 example, see §§ 211.192, 211.194, 212.50(a), and 212.70(f)(1)(vi)).
220
221 Similarly, bound paginated notebooks, stamped for official use by a document control
222 group, allow detection of unofficial notebooks as well as of any gaps in notebook pages.
223
224 7. How often should audit trails be reviewed?
225
226 FDA recommends that audit trails that capture changes to critical data be reviewed with
227 each record and before final approval of the record. Audit trails subject to regular review
228 should include, but are not limited to, the following: the change history of finished
229 product test results, changes to sample run sequences, changes to sample identification,
230 and changes to critical process parameters.
231
232 FDA recommends routine scheduled audit trail review based on the complexity of the
233 system and its intended use.
234
235 See audit trail definition 1.c. above for further information on audit trails.
236
237 8. Who should review audit trails?
238
239 Audit trails are considered part of the associated records. Personnel responsible for record
240 review under CGMP should review the audit trails that capture changes to critical data
241 associated with the record as they review the rest of the record (for example, §§
242 211.22(a), 211.101(c), 211.194(a)(8), and 212.20(d)). For example, all production and
6
243 control records, which includes audit trails, must be reviewed and approved by the
244 quality unit (§ 211.192). This is similar to the expectation that cross-outs on paper be
245 assessed when reviewing data.
246
247 9. Can electronic copies be used as accurate reproductions of paper or
248 electronic records?
249
250 Yes. Electronic copies can be used as true copies of paper or electronic records, provided
251 the copies preserve the content and meaning of the original data, which includes
252 associated metadata and the static or dynamic nature of the original records.
253
254 True copies of dynamic electronic records may be made and maintained in the format of
255 the original records or in a compatible format, provided that the content and meaning of
256 the original records are preserved and that a suitable reader and copying equipment (for
257 example, software and hardware, including media readers) are readily available (§§
258 211.180(d) and 212.110).
259
260 10. Is it acceptable to retain paper printouts or static records instead of original
261 electronic records from stand-alone computerized laboratory instruments,
262 such as an FT-IR instrument?
263
264 A paper printout or static record may satisfy retention requirements if it is a complete
265 copy of the original record (see §§ 211.68(b), 211.188, 211.194, and 212.60). For
266 example, pH meters and balances may create a paper printout or static image during data
267 acquisition as the original record. In this case, the paper printout or static image created
268 during acquisition, or a true copy, should be retained (§ 211.180).
269
270 However, electronic records from certain types of laboratory instruments are dynamic
271 records, and a printout or a static record does not preserve the dynamic format which is
272 part of the complete original record. For example, the spectral file created by FT-IR
273 (Fourier transform infrared spectroscopy) can be reprocessed, but a static record or
274 printout is fixed, which would not satisfy CGMP requirements to retain original records
275 or true copies (§ 211.180(d)). Also, if the full spectrum is not displayed, contaminants
276 may be excluded.
277
278 Control strategies must ensure that original laboratory records, including paper and
279 electronic records, are subject to second-person review (§ 211.194(a)(8)) to make certain
280 that all test results are appropriately reported.
281
282 For PET drugs, see the guidance for industry PET Drugs — Current Good Manufacturing
283 Practice (CGMP) for discussion of equipment and laboratory controls, including
284 regulatory requirements for records.
285
7
286 11. Can electronic signatures be used instead of handwritten signatures for
287 master production and control records?
288
289 Yes, electronic signatures with the appropriate controls can be used instead of
290 handwritten signatures or initials in any CGMP required record. While § 211.186(a)
291 specifies a “full signature, handwritten,” as explained in the Federal Register on
292 September 29, 1978 (43 FR 45069), part of the intent of the full signature requirement is
293 to be able to clearly identify the individual responsible for signing the record. An
294 electronic signature with the appropriate controls to securely link the signature with the
295 associated record fulfills this requirement. This comports with part 11, which establishes
296 criteria for when electronic signatures are considered the legally binding equivalent of
297 handwritten signatures. Firms using electronic signatures should document the controls
298 used to ensure that they are able to identify the specific person who signed the records
299 electronically.
300
301 There is no requirement for a handwritten signature for the MPCR in the PET CGMP
302 regulations (21 CFR part 212).
303
304 12. When does electronic data become a CGMP record?
305
306 When generated to satisfy a CGMP requirement, all data become a CGMP record. You
307 must document, or save, the data at the time of performance to create a record in
308 compliance with CGMP requirements, including, but not limited to, §§ 211.100(b) and
309 211.160(a). FDA expects processes to be designed so that quality data required to be
310 created and maintained cannot be modified. For example, chromatograms should be sent
311 to long-term storage (archiving or a permanent record) upon run completion instead of at
312 the end of a day’s runs.
313
314 It is not acceptable to record data on pieces of paper that will be discarded after the data
315 are transcribed to a permanent laboratory notebook (see §§ 211.100(b), 211.160(a), and
316 211.180(d)). Similarly, it is not acceptable to store data electronically in temporary
317 memory, in a manner that allows for manipulation, before creating a permanent record.
318 Electronic data that are automatically saved into temporary memory do not meet CGMP
319 documentation or retention requirements.
320
321 You may employ a combination of technical and procedural controls to meet CGMP
322 documentation practices for electronic systems. For example, a computer system, such as
323 a Laboratory Information Management System (LIMS) or an Electronic Batch Record
324 (EBR) system, can be designed to automatically save after each separate entry. This
325 would be similar to recording each entry contemporaneously on a paper batch record to
326 satisfy CGMP requirements. The computer system could be combined with a procedure
327 requiring data be entered immediately when generated.
328
329 For PET drugs, see the “Laboratory Controls” section of the guidance for industry PET
330 Drugs — Current Good Manufacturing Practice (CGMP).
331
8
332 13. Why has the FDA cited use of actual samples during “system suitability” or
333 test, prep, or equilibration runs in warning letters?
334
335 FDA prohibits sampling and testing with the goal of achieving a specific result or to
336 overcome an unacceptable result (e.g., testing different samples until the desired passing
337 result is obtained). This practice, also referred to as testing into compliance, is not
338 consistent with CGMP (see the guidance for industry Investigating Out-of-Specification
339 (OOS) Test Results for Pharmaceutical Production). In some situations, use of actual
340 samples to perform system suitability testing has been used as a means of testing into
341 compliance. We would consider it a violative practice to use an actual sample in test,
342 prep, or equilibration runs as a means of disguising testing into compliance.
343
344 According to the United States Pharmacopeia (USP), system suitability tests should
345 include replicate injections of a standard preparation or other standard solutions to
346 determine if requirements for precision are satisfied (see USP General Chapter <621>
347 Chromatography). System suitability tests, including the identity of the preparation to be
348 injected and the rationale for its selection, should be performed according to the firm’s
349 established written procedures and the approved application or applicable compendial
350 monograph (§§ 211.160 and 212.60).
351
352 If an actual sample is to be used for system suitability testing, it should be a properly
353 characterized secondary standard, written procedures should be established and followed,
354 and the sample should be from a different batch than the sample(s) being tested (§§
355 211.160, 211.165, and 212.60). All data should be included in the record that is retained
356 and subject to review unless there is documented scientific justification for its exclusion.
357
358 For more information, see also the ICH guidance for industry Q2(R1) Validation of
359 Analytical Procedures: Text and Methodology.
360
361 14. Is it acceptable to only save the final results from reprocessed laboratory
362 chromatography?
363
364 No. Analytical methods should be capable and stable. For most lab analyses, reprocessing
365 data should not be regularly needed. If chromatography is reprocessed, written
366 procedures must be established and followed and each result retained for review (see §§
367 211.160(a), 211.160(b), 211.165(c), 211.194(a)(4), and 212.60(a)). FDA requires
368 complete data in laboratory records, which includes raw data, graphs, charts, and spectra
369 from laboratory instruments (§§ 211.194(a) and 212.60(g)(3)).
370
371 15. Can an internal tip regarding a quality issue, such as potential data
372 falsification, be handled informally outside of the documented CGMP quality
373 system?
374
375 No. Suspected or known falsification or alteration of records required under parts 210,
376 211, and 212 must be fully investigated under the CGMP quality system to determine the
377 effect of the event on patient safety, product quality, and data reliability; to determine the
9
378 root cause; and to ensure the necessary corrective actions are taken (see §§ 211.22(a),
379 211.125(c), 211.192, 211.198, 211.204, and 212.100).
380
381 FDA invites individuals to report suspected data integrity issues that may affect the
382 safety, identity, strength, quality, or purity of drug products at DrugInfo@fda.hhs.gov.
383 “CGMP data integrity” should be included in the subject line of the email.
384
385 See also Application Integrity Policy, available at
386 http://www.fda.gov/ICECI/EnforcementActions/ApplicationIntegrityPolicy/default.htm.
387
388 16. Should personnel be trained in detecting data integrity issues as part of a
389 routine CGMP training program?
390
391 Yes. Training personnel to detect data integrity issues is consistent with the personnel
392 requirements under §§ 211.25 and 212.10, which state that personnel must have the
393 education, training, and experience, or any combination thereof, to perform their assigned
394 duties.
395
396 17. Is the FDA investigator allowed to look at my electronic records?
397
398 Yes. All records required under CGMP are subject to FDA inspection. You must allow
399 authorized inspection, review, and copying of records, which includes copying of
400 electronic data (§§ 211.180(c) and 212.110(a) and (b)). See also section 704 of the FD&C
401 Act.
402
403 18. How does FDA recommend data integrity problems identified during
404 inspections, in warning letters, or in other regulatory actions be addressed?
405
406 FDA encourages you to demonstrate that you have effectively remedied your problems
407 by: hiring a third party auditor, determining the scope of the problem, implementing a
408 corrective action plan (globally), and removing at all levels individuals responsible for
409 problems from CGMP positions. FDA may conduct an inspection to decide whether
410 CGMP violations involving data integrity have been remedied.
411
412 These expectations mirror those developed for the Application Integrity Policy. For more
413 detailed guidance, see the “Points to Consider for Internal Reviews and Corrective Action
414 Operating Plans” public document available on the FDA Web site, accessible at
415 http://www.fda.gov/ICECI/EnforcementActions/ApplicationIntegrityPolicy/ucm134744.
416 htm.
10
Appendices D: ICH Q10: Pharmaceutical Quality
System
Q10 Pharmaceutical
Quality System
April 2009
ICH
Q10 Pharmaceutical
Quality System
Additional copies are available from:
Office of Communication
Division of Drug Information
Center for Drug Evaluation and Research
10903 New Hampshire Ave.,Bldg. 51, Room 2201
(Tel) 301-796-3400
http://www.fda.gov/cder/guidance/index.htm
Office of Communication, Outreach and
Development, HFM-40
Center for Biologics Evaluation and Research
1401 Rockville Pike, Rockville, MD 20852-1448
(Tel) 1-800-835-4709 or 301-827-1800
http://www.fda.gov/cber/guidelines.htm.
April 2009
ICH
TABLE OF CONTENTS
I. INTRODUCTION (1, 1.1)................................................................................................ 1
II. PHARMACEUTICAL QUALITY MANAGEMENT SYSTEM ................................. 2
A. Scope (1.2)....................................................................................................................................... 2
B. Relationship of ICH Q10 to Regional GMP Requirements, ISO Standards,
and ICH Q7 (1.3)............................................................................................................................ 3
C. Relationship of ICH Q10 to Regulatory Approaches (1.4)......................................................... 3
D. ICH Q10 Objectives (1.5) .............................................................................................................. 3
E. Enablers: Knowledge Management and Quality Risk Management (1.6) ............................... 4
F. Design and Content Considerations (1.7) .................................................................................... 4
G. Quality Manual (1.8)...................................................................................................................... 5
III. MANAGEMENT RESPONSIBILITY (2)...................................................................... 5
A. Management Commitment (2.1)................................................................................................... 5
B. Quality Policy (2.2)......................................................................................................................... 6
C. Quality Planning (2.3).................................................................................................................... 6
D. Resource Management (2.4).......................................................................................................... 7
E. Internal Communication (2.5) ...................................................................................................... 7
F. Management Review (2.6) ............................................................................................................ 7
G. Management of Outsourced Activities and Purchased Materials (2.7).................................... 7
H. Management of Change in Product Ownership (2.8) ................................................................. 8
IV. CONTINUAL IMPROVEMENT OF PROCESS PERFORMANCE AND
PRODUCT QUALITY (3) ............................................................................................... 8
A. Lifecycle Stage Goals (3.1) ............................................................................................................ 8
B. Pharmaceutical Quality System Elements (3.2) .......................................................................... 9
V. CONTINUAL IMPROVEMENT OF THE PHARMACEUTICAL QUALITY
SYSTEM (4) .................................................................................................................... 13
A. Management Review of the Pharmaceutical Quality System (4.1).......................................... 13
B. Monitoring of Internal and External Factors That Can Have an Impacton the
Pharmaceutical Quality System (4.2) ......................................................................................... 13
C. Outcomes of Management Review and Monitoring (4.3)......................................................... 14
VI. GLOSSARY (5)............................................................................................................... 15
Annex I: Potential Opportunities To Enhance Science- and Risk-Based Regulatory
Approaches *............................................................................................................... 18
Annex 2: Diagram of the ICH Q10 Pharmaceutical Quality System Model…..……………19
i
Guidance for Industry1
Q10 Pharmaceutical Quality System
This guidance represents the Food and Drug Administration's (FDA’s) current thinking on this topic. It
does not create or confer any rights for or on any person and does not operate to bind FDA or the public.
You can use an alternative approach if the approach satisfies the requirements of the applicable statutes
and regulations. If you want to discuss an alternative approach, contact the FDA staff responsible for
implementing this guidance. If you cannot identify the appropriate FDA staff, call the appropriate
number listed on the title page of this guidance.
I. INTRODUCTION (1, 1.1)2
This internationally harmonized guidance is intended to assist pharmaceutical manufacturers by

describing a model for an effective quality management system for the pharmaceutical industry,
referred to as the pharmaceutical quality system. Throughout this guidance, the term
pharmaceutical quality system refers to the ICH Q10 model.
ICH Q10 describes one comprehensive model for an effective pharmaceutical quality system that
is based on International Organization for Standardization (ISO) quality concepts, includes
applicable good manufacturing practice (GMP) regulations, and complements ICH “Q8
Pharmaceutical Development” and ICH “Q9 Quality Risk Management.”3 ICH Q10 is a model
for a pharmaceutical quality system that can be implemented throughout the different stages of a
product lifecycle. Much of the content of ICH Q10 applicable to manufacturing sites is currently
specified by regional GMP requirements. ICH Q10 is not intended to create any new
expectations beyond current regulatory requirements. Consequently, the content of ICH Q10 that
is additional to current regional GMP requirements is optional.
1
This guidance was developed within the Expert Working Group (Quality) of the International Conference on
Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) and has been
subject to consultation by the regulatory parties, in accordance with the ICH process. This document has been
endorsed by the ICH Steering Committee at Step 4 of the ICH process, June 2008. At Step 4 of the process, the final
draft is recommended for adoption to the regulatory bodies of the European Union, Japan, and the United States.
2
Arabic numbers reflect the organizational breakdown of the document endorsed by the ICH Steering Committee at
Step 4 of the ICH process, June 2008.
3
These guidances are available on the Internet at http://www.fda.gov/cder/guidance/index.htm. We update
guidances periodically. To make sure you have the most recent version of a guidance, check the CDER guidance
page at http://www.fda.gov/cder/guidance/index.htm
ICH Q10 demonstrates industry and regulatory authorities’ support of an effective

pharmaceutical quality system to enhance the quality and availability of medicines around the
world in the interest of public health. Implementation of ICH Q10 throughout the product
lifecycle should facilitate innovation and continual improvement and strengthen the link between
pharmaceutical development and manufacturing activities.
FDA's guidance documents, including this guidance, do not establish legally enforceable
responsibilities. Instead, guidances describe the Agency’s current thinking on a topic and should
be viewed only as recommendations, unless specific regulatory or statutory requirements are
cited. The use of the word should in Agency guidances means that something is suggested or
recommended, but not required.
II. PHARMACEUTICAL QUALITY MANAGEMENT SYSTEM
A. Scope (1.2)
This guidance applies to the systems supporting the development and manufacture of
pharmaceutical drug substances (i.e., active pharmaceutical ingredients (APIs)) and drug
products, including biotechnology and biological products, throughout the product lifecycle.
The elements of ICH Q10 should be applied in a manner that is appropriate and proportionate to
each of the product lifecycle stages, recognizing the differences among, and the different goals
of, each stage (see section IV (3)).
For the purposes of this guidance, the product lifecycle includes the following technical activities
for new and existing products:
• Pharmaceutical Development
o Drug substance development
o Formulation development (including container/closure system)
o Manufacture of investigational products
o Delivery system development (where relevant)
o Manufacturing process development and scale-up
o Analytical method development
• Technology Transfer
o New product transfers during development through manufacturing
o Transfers within or between manufacturing and testing sites for marketed products
• Commercial Manufacturing
o Acquisition and control of materials
o Provision of facilities, utilities, and equipment
o Production (including packaging and labeling)
o Quality control and assurance
o Release
o Storage
o Distribution (excluding wholesaler activities)

• Product Discontinuation
o Retention of documentation
o Sample retention
o Continued product assessment and reporting
B. Relationship of ICH Q10 to Regional GMP Requirements, ISO Standards,

and ICH Q7 (1.3)
Regional GMP requirements, the ICH guidance “Q7 Good Manufacturing Practice Guidance for
Active Pharmaceutical Ingredients,” and ISO quality management system guidelines form the
foundation for ICH Q10. To meet the objectives described below, ICH Q10 augments GMPs by
describing specific quality system elements and management responsibilities. ICH Q10 provides
a harmonized model for a pharmaceutical quality system throughout the lifecycle of a product
and is intended to be used together with regional GMP requirements.
The regional GMPs do not explicitly address all stages of the product lifecycle (e.g.,
development). The quality system elements and management responsibilities described in this
guidance are intended to encourage the use of science- and risk-based approaches at each
lifecycle stage, thereby promoting continual improvement across the entire product lifecycle.
C. Relationship of ICH Q10 to Regulatory Approaches (1.4)
Regulatory approaches for a specific product or manufacturing facility should be commensurate

with the level of product and process understanding, the results of quality risk management, and
the effectiveness of the pharmaceutical quality system. When implemented, the effectiveness of
the pharmaceutical quality system can normally be evaluated during a regulatory inspection at
the manufacturing site. Potential opportunities to enhance science- and risk-based regulatory
approaches are identified in Annex 1. Regulatory processes will be determined by region.
D. ICH Q10 Objectives (1.5)
Implementation of the Q10 model should result in achievement of three main objectives that
complement or enhance regional GMP requirements.
1. A
chieve Product Realization (1.5.1)
To establish, implement, and maintain a system that allows the delivery of products with the
quality attributes appropriate to meet the needs of patients, health care professionals, regulatory
authorities (including compliance with approved regulatory filings) and other internal and
external customers.
2 Establish and Maintain a State of Control (1.5.2)
To develop and use effective monitoring and control systems for process performance and
product quality, thereby providing assurance of continued suitability and capability of processes.
Quality risk management can be useful in identifying the monitoring and control systems.
3 Facilitate Continual Improvement (1.5.3)
To identify and implement appropriate product quality improvements, process improvements,

variability reduction, innovations, and pharmaceutical quality system enhancements, thereby
increasing the ability to fulfill a pharmaceutical manufacturer’s own quality needs consistently.
Quality risk management can be useful for identifying and prioritizing areas for continual
improvement.
E. Enablers: Knowledge Management and Quality Risk Management (1.6)
Use of knowledge management and quality risk management will enable a company to
implement ICH Q10 effectively and successfully. These enablers will facilitate achievement of
the objectives described in section II.D (1.5) above by providing the means for science- and risk-
based decisions related to product quality.
1. Knowledge Management (1.6.1)
Product and process knowledge should be managed from development through the commercial
life of the product up to and including product discontinuation. For example, development
activities using scientific approaches provide knowledge for product and process understanding.
Knowledge management is a systematic approach to acquiring, analyzing, storing, and
disseminating information related to products, manufacturing processes, and components.
Sources of knowledge include, but are not limited to, prior knowledge (public domain or
internally documented); pharmaceutical development studies; technology transfer activities;
process validation studies over the product lifecycle; manufacturing experience; innovation;
continual improvement; and change management activities.
2. Quality Risk Management (1.6.2)
Quality risk management is integral to an effective pharmaceutical quality system. It can

provide a proactive approach to identifying, scientifically evaluating, and controlling potential
risks to quality. It facilitates continual improvement of process performance and product quality
throughout the product lifecycle. ICH Q9 provides principles and examples of tools for quality
risk management that can be applied to different aspects of pharmaceutical quality.
F. Design and Content Considerations (1.7)
(a) The design, organization, and documentation of the pharmaceutical quality system
should be well structured and clear to facilitate common understanding and consistent
application.
(b) The elements of ICH Q10 should be applied in a manner that is appropriate and
proportionate to each of the product lifecycle stages, recognizing the different goals
and knowledge available for each stage.

(c) The size and complexity of the company’s activities should be taken into consideration
when developing a new pharmaceutical quality system or modifying an existing one.
The design of the pharmaceutical quality system should incorporate appropriate risk
management principles. While some aspects of the pharmaceutical quality system can
be company wide and others site specific, the effectiveness of the pharmaceutical
quality system is normally demonstrated at the site level.
(d) The pharmaceutical quality system should include appropriate processes, resources,
and responsibilities to provide assurance of the quality of outsourced activities and
purchased materials as described in section III.G (2.7).
(e) Management responsibilities, as described in section III (2), should be identified
within the pharmaceutical quality system.
(f) The pharmaceutical quality system should include the following elements, as
described in section IV (3): process performance and product quality monitoring,
corrective and preventive action, change management, and management review.
(g) Performance indicators, as described in section V (4), should be identified and used to
monitor the effectiveness of processes within the pharmaceutical quality system.
G. Quality Manual (1.8)
A Quality Manual or equivalent documentation approach should be established and should

contain the description of the pharmaceutical quality system. The description should include:
(a) The quality policy (see section III (2)).
(b) The scope of the pharmaceutical quality system.
(c) Identification of the pharmaceutical quality system processes, as well as their
sequences, linkages, and interdependencies. Process maps and flow charts can be
useful tools to facilitate depicting pharmaceutical quality system processes in a visual
manner.
(d) Management responsibilities within the pharmaceutical quality system (see
section III (2)).
III. MANAGEMENT RESPONSIBILITY (2)
Leadership is essential to establish and maintain a company-wide commitment to quality and for
the performance of the pharmaceutical quality system.
A. Management Commitment (2.1)
(a) Senior management has the ultimate responsibility to ensure an effective

pharmaceutical quality system is in place to achieve the quality objectives, and that
roles, responsibilities, and authorities are defined, communicated, and implemented

throughout the company.
(b) Management should:
(1) Participate in the design, implementation, monitoring, and maintenance of an
effective pharmaceutical quality system.
(2) Demonstrate strong and visible support for the pharmaceutical quality system
and ensure its implementation throughout their organization.
(3) Ensure a timely and effective communication and escalation process exists to
raise quality issues to the appropriate levels of management.
(4) Define individual and collective roles, responsibilities, authorities, and inter-
relationships of all organizational units related to the pharmaceutical quality
system. Ensure these interactions are communicated and understood at all
levels of the organization. An independent quality unit/structure with authority
to fulfill certain pharmaceutical quality system responsibilities is required by
regional regulations.
(5) Conduct management reviews of process performance and product quality and
of the pharmaceutical quality system.
(6) Advocate continual improvement.
(7) Commit appropriate resources.
B. Quality Policy (2.2)
(a) Senior management should establish a quality policy that describes the overall
intentions and direction of the company related to quality.
(b) The quality policy should include an expectation to comply with applicable regulatory
requirements and should facilitate continual improvement of the pharmaceutical
quality system.
(c) The quality policy should be communicated to and understood by personnel at all
levels in the company.
(d) The quality policy should be reviewed periodically for continuing effectiveness.
C. Quality Planning (2.3)
(a) Senior management should ensure the quality objectives to implement the quality
policy are defined and communicated.
(b) Quality objectives should be supported by all relevant levels of the company.
(c) Quality objectives should align with the company’s strategies and be consistent with
the quality policy.
(d) Management should provide the appropriate resources and training to achieve the
quality objectives.
(e) Performance indicators that measure progress against quality objectives should be
established, monitored, communicated regularly, and acted upon as appropriate as
described in section V.A (4.1) of this document.
D. Resource Management (2.4)
(a) Management should determine and provide adequate and appropriate resources
(human, financial, materials, facilities, and equipment) to implement and maintain the
pharmaceutical quality system and continually improve its effectiveness.
(b) Management should ensure that resources are appropriately applied to a specific
product, process, or site.
E. Internal Communication (2.5)
(a) Management should ensure appropriate communication processes are established and
implemented within the organization.
(b) Communications processes should ensure the flow of appropriate information between
all levels of the company.
(c) Communication processes should ensure the appropriate and timely escalation of
certain product quality and pharmaceutical quality system issues.
F. Management Review (2.6)
(a) Senior management should be responsible for pharmaceutical quality system

governance through management review to ensure its continuing suitability and
effectiveness.
(b) Management should assess the conclusions of periodic reviews of process
performance and product quality and of the pharmaceutical quality system, as
described in sections IV (3) and V (4).
G. Management of Outsourced Activities and Purchased Materials (2.7)
The pharmaceutical quality system, including the management responsibilities described in this
section, extends to the control and review of any outsourced activities and quality of purchased
materials. The pharmaceutical company is ultimately responsible to ensure processes are in place
to assure the control of outsourced activities and quality of purchased materials. These processes
should incorporate quality risk management and include:
(a) Assessing prior to outsourcing operations or selecting material suppliers, the suitability
and competence of the other party to carry out the activity or provide the material
using a defined supply chain (e.g., audits, material evaluations, qualification).

(b) Defining the responsibilities and communication processes for quality-related
activities of the involved parties. For outsourced activities, this should be included in a
written agreement between the contract giver and contract acceptor.
(c) Monitoring and review of the performance of the contract acceptor or the quality of
the material from the provider, and the identification and implementation of any
essential improvements.
(d) Monitoring incoming ingredients and materials to ensure they are from approved
sources using the agreed supply chain.
H. Management of Change in Product Ownership (2.8)
When product ownership changes (e.g., through acquisitions), management should consider the
complexity of this and ensure:
(a) The ongoing responsibilities are defined for each company involved
(b) The essential information is transferred
IV. CONTINUAL IMPROVEMENT OF PROCESS PERFORMANCE AND

PRODUCT QUALITY (3)
This section describes the lifecycle stage goals and the four specific pharmaceutical quality
system elements that augment regional requirements to achieve the ICH Q10 objectives, as
defined in section II.D (1.5). It does not restate all regional GMP requirements.
A. Lifecycle Stage Goals (3.1)
The goals of each product lifecycle stage are described below.
1. Pharmaceutical Development (3.1.1)
The goal of pharmaceutical development activities is to design a product and its manufacturing
process to consistently deliver the intended performance and meet the needs of patients and
healthcare professionals, and regulatory authorities and internal customers’ requirements.
Approaches to pharmaceutical development are described in ICH Q8. The results of exploratory
and clinical development studies, while outside the scope of this guidance, are inputs to
pharmaceutical development.
2. T
echnology Transfer (3.1.2)
The goal of technology transfer activities is to transfer product and process knowledge between
development and manufacturing, and within or between manufacturing sites to achieve product
realization. This knowledge forms the basis for the manufacturing process, control strategy,
process validation approach, and ongoing continual improvement.
3. Commercial Manufacturing (3.1.3)
The goals of manufacturing activities include achieving product realization, establishing and
maintaining a state of control, and facilitating continual improvement. The pharmaceutical
quality system should assure that the desired product quality is routinely met, suitable process
performance is achieved, the set of controls are appropriate, improvement opportunities are
identified and evaluated, and the body of knowledge is continually expanded.
4. Product Discontinuation (3.1.4)
The goal of product discontinuation activities is to manage the terminal stage of the product
lifecycle effectively. For product discontinuation, a predefined approach should be used to
manage activities such as retention of documentation and samples and continued product
assessment (e.g., complaint handling and stability) and reporting in accordance with regulatory
requirements.
B. Pharmaceutical Quality System Elements (3.2)
The elements described below might be required in part under regional GMP regulations.
However, the Q10 model’s intent is to enhance these elements to promote the lifecycle approach
to product quality. These four elements are:
• Process performance and product quality monitoring system

• Corrective action and preventive action (CAPA) system
• Change management system
• Management review of process performance and product quality
These elements should be applied in a manner that is appropriate and proportionate to each of the
product lifecycle stages, recognizing the differences among the stages and the different goals of
each stage. Throughout the product lifecycle, companies are encouraged to evaluate
opportunities for innovative approaches to improve product quality.
Each element is followed by a table of example applications of the element to the stages of the
pharmaceutical lifecycle.
1. Process Performance and Product Quality Monitoring System 3.2.1
Pharmaceutical companies should plan and execute a system for the monitoring of process
performance and product quality to ensure a state of control is maintained. An effective
monitoring system provides assurance of the continued capability of processes and controls to
produce a product of desired quality and to identify areas for continual improvement. The
process performance and product quality monitoring system should:
(a) Use quality risk management to establish the control strategy. This can include
parameters and attributes related to drug substance and drug product materials and
components, facility and equipment operating conditions, in-process controls, finished
product specifications, and the associated methods and frequency of monitoring and
control. The control strategy should facilitate timely feedback/feedforward and
appropriate corrective action and preventive action.
(b) Provide the tools for measurement and analysis of parameters and attributes identified
in the control strategy (e.g., data management and statistical tools).
(c) Analyze parameters and attributes identified in the control strategy to verify continued
operation within a state of control.
(d) Identify sources of variation affecting process performance and product quality for
potential continual improvement activities to reduce or control variation.
(e) Include feedback on product quality from both internal and external sources (e.g.,
complaints, product rejections, nonconformances, recalls, deviations, audits and
regulatory inspections, and findings).
(f) Provide knowledge to enhance process understanding, enrich the design space (where
established), and enable innovative approaches to process validation.
Table I: Application of Process Performance and Product Quality Monitoring System

Throughout the Product Lifecycle
Pharmaceutical Technology Commercial Product

Development Transfer Manufacturing Discontinuation
Process and product Monitoring during A well-defined system Once manufacturing
knowledge generated scale-up activities can for process ceases, monitoring
and process and provide a preliminary performance and such as stability testing
product monitoring indication of process product quality should continue to
conducted throughout performance and the monitoring should be completion of the
development can be successful integration applied to assure studies. Appropriate
used to establish a into manufacturing. performance within a action on marketed
control strategy for Knowledge obtained state of control and to product should
manufacturing. during transfer and identify improvement continue to be executed
scale-up activities can areas. according to regional
be useful in further regulations.
developing the control
strategy.
2. Corrective Action and Preventive Action (CAPA) System (3.2.2)
The pharmaceutical company should have a system for implementing corrective actions and
preventive actions resulting from the investigation of complaints, product rejections,
nonconformances, recalls, deviations, audits, regulatory inspections and findings, and trends
from process performance and product quality monitoring. A structured approach to the
10
investigation process should be used with the objective of determining the root cause. The level
of effort, formality, and documentation of the investigation should be commensurate with the
level of risk, in line with ICH Q9. CAPA methodology should result in product and process
improvements and enhanced product and process understanding.
Table II: Application of Corrective Action and Preventive Action System Throughout the
Product Lifecycle

Product or process CAPA can be used as CAPA should be used, CAPA should continue
variability is explored. an effective system for and the effectiveness of after the product is
CAPA methodology is feedback, feedforward, the actions should be discontinued. The
useful where corrective and continual evaluated. impact on product
actions and preventive improvement. remaining on the
actions are incorporated market should be
into the iterative design considered, as well as
and development other products that
process. might be affected.
3. Change Management System (3.2.3)
Innovation, continual improvement, the outputs of process performance and product quality
monitoring, and CAPA drive change. To evaluate, approve, and implement these changes
properly, a company should have an effective change management system. There is generally a
difference in formality of change management processes prior to the initial regulatory
submission and after submission, where changes to the regulatory filing might be required under
regional requirements.
The change management system ensures continual improvement is undertaken in a timely and
effective manner. It should provide a high degree of assurance there are no unintended
consequences of the change.
The change management system should include the following, as appropriate for the stage of the
lifecycle:
(a) Quality risk management should be utilized to evaluate proposed changes. The level of
effort and formality of the evaluation should be commensurate with the level of risk.
(b) Proposed changes should be evaluated relative to the marketing authorization,
including design space, where established, and/or current product and process
understanding. There should be an assessment to determine whether a change to the
regulatory filing is required under regional requirements. As stated in ICH Q8,
working within the design space is not considered a change (from a regulatory filing
perspective). However, from a pharmaceutical quality system standpoint, all changes
should be evaluated by a company’s change management system.
(c) Proposed changes should be evaluated by expert teams contributing the appropriate
11
expertise and knowledge from relevant areas (e.g., Pharmaceutical Development,

Manufacturing, Quality, Regulatory Affairs, and Medical) to ensure the change is
technically justified. Prospective evaluation criteria for a proposed change should be
set.
(d) After implementation, an evaluation of the change should be undertaken to confirm the
change objectives were achieved and that there was no deleterious impact on product
quality.
Table III: Application of Change Management System Throughout the Product Lifecycle

Change is an inherent The change A formal change Any changes after
part of the development management system management system product discontinuation
process and should be should provide should be in place for should go through an
documented; the management and commercial appropriate change
formality of the change documentation of manufacturing. management system.
management process adjustments made to Oversight by the
should be consistent the process during quality unit should
with the stage of technology transfer provide assurance of
pharmaceutical activities. appropriate science-
development. and risk-based
assessments.
4. Management Review of Process Performance and Product Quality (3.2.4)
Management review should provide assurance that process performance and product quality are
managed over the lifecycle. Depending on the size and complexity of the company, management
review can be a series of reviews at various levels of management and should include a timely
and effective communication and escalation process to raise appropriate quality issues to senior
levels of management for review.
(a) The management review system should include:

(1) The results of regulatory inspections and findings, audits and other
assessments, and commitments made to regulatory authorities
(2) Periodic quality reviews, that can include:
(i) Measures of customer satisfaction such as product quality complaints
and recalls
(ii) Conclusions of process performance and product quality monitoring
(iii) The effectiveness of process and product changes including those
arising from corrective action and preventive actions
(3) Any follow-up actions from previous management reviews
(b) The management review system should identify appropriate actions, such as:
(1) Improvements to manufacturing processes and products
(2) Provision, training, and/or realignment of resources
12
(3) Capture and dissemination of knowledge
Table IV: Application of Management Review of Process Performance and Product

Quality Throughout the Product Lifecycle

Aspects of management Aspects of Management review Management review
review can be management review should be a structured should include such
performed to ensure should be performed to system, as described items as product
adequacy of the product ensure the developed above, and should stability and product
and process design. product and process support continual quality complaints.
can be manufactured at improvement.
commercial scale.
V. ONTINUAL IMPROVEMENT OF THE PHARMACEUTICAL QUALITY

C
SYSTEM (4)
This section describes activities that should be conducted to manage and continually improve the
pharmaceutical quality system.
A. Management Review of the Pharmaceutical Quality System (4.1)
Management should have a formal process for reviewing the pharmaceutical quality system on a
periodic basis. The review should include:
(a) Measurement of achievement of pharmaceutical quality system objectives
(b) Assessment of performance indicators that can be used to monitor the effectiveness of
processes within the pharmaceutical quality system, such as:
(1) Complaint, deviation, CAPA and change management processes
(2) Feedback on outsourced activities
(3) Self-assessment processes including risk assessments, trending, and audits
(4) External assessments such as regulatory inspections and findings and customer
audits
B. Monitoring of Internal and External Factors That Can Have an Impacton

the Pharmaceutical Quality System (4.2)
Factors monitored by management can include:

(a) Emerging regulations, guidance, and quality issues that can have an impact on the
Pharmaceutical Quality System
(b) Innovations that might enhance the pharmaceutical quality system
(c) Changes in business environment and objectives
13
(d) Changes in product ownership
C. Outcomes of Management Review and Monitoring (4.3)
The outcome of management review of the pharmaceutical quality system and monitoring of
internal and external factors can include:
(a) Improvements to the pharmaceutical quality system and related processes
(b) Allocation or reallocation of resources and/or personnel training
(c) Revisions to quality policy and quality objectives
(d) Documentation and timely and effective communication of the results of the
management review and actions, including escalation of appropriate issues to senior
management
14
VI. GLOSSARY (5)
ICH and ISO definitions are used in ICH Q10 where they exist. For the purpose of ICH Q10,
where the words “requirement”, “requirements,” or “necessary” appear in an ISO definition, they
do not necessarily reflect a regulatory requirement. The source of the definition is identified in
parentheses after the definition. Where no appropriate ICH or ISO definition was available, an
ICH Q10 definition was developed.
Capability of a Process: Ability of a process to realize a product that will fulfill the
requirements of that product. The concept of process capability can also be defined in statistical
terms. (ISO 9000:2005)
Change Management: A systematic approach to proposing, evaluating, approving,

implementing, and reviewing changes. (ICH Q10)
Continual Improvement: Recurring activity to increase the ability to fulfill requirements. (ISO
9000:2005)
Control Strategy: A planned set of controls, derived from current product and process
understanding, that assures process performance and product quality. The controls can include
parameters and attributes related to drug substance and drug product materials and components,
facility and equipment operating conditions, in-process controls, finished product specifications,
and the associated methods and frequency of monitoring and control. (ICH Q10)
Corrective Action: Action to eliminate the cause of a detected nonconformity or other

undesirable situation. NOTE: Corrective action is taken to prevent recurrence whereas preventive
action is taken to prevent occurrence. (ISO 9000:2005)
Design Space: The multidimensional combination and interaction of input variables (e.g.,
material attributes) and process parameters that have been demonstrated to provide assurance of
quality. (ICH Q8)
Enabler: A tool or process that provides the means to achieve an objective. (ICH Q10)
Feedback/Feedforward:
Feedback: The modification or control of a process or system by its results or effects.
Feedforward: The modification or control of a process using its anticipated results or

effects. (Oxford Dictionary of English by Oxford University Press, 2003)
Feedback/feedforward can be applied technically in process control strategies and conceptually

in quality management. (ICH Q10)
Innovation: The introduction of new technologies or methodologies. (ICH Q10)
15
Knowledge Management: Systematic approach to acquiring, analyzing, storing, and

disseminating information related to products, manufacturing processes, and components. (ICH
Q10)
Outsourced Activities: Activities conducted by a contract acceptor under a written agreement

with a contract giver. (ICH Q10)
Performance Indicators: Measurable values used to quantify quality objectives to reflect the
performance of an organization, process, or system, also known as performance metrics in some
regions. (ICH Q10)
Pharmaceutical Quality System (PQS): Management system to direct and control a

pharmaceutical company with regard to quality. (ICH Q10 based upon ISO 9000:2005)
Preventive Action: Action to eliminate the cause of a potential nonconformity or other

undesirable potential situation. NOTE: Preventive action is taken to prevent occurrence whereas
corrective action is taken to prevent recurrence. (ISO 9000:2005)
Product Realization: Achievement of a product with the quality attributes appropriate to meet
the needs of patients, health care professionals, and regulatory authorities (including compliance
with marketing authorization) and internal customers requirements. (ICH Q10)
Quality: The degree to which a set of inherent properties of a product, system, or process fulfils
requirements. (ICH Q9)
Quality Manual: Document specifying the quality management system of an organization.

(ISO 9000:2005)
Quality Objectives: A means to translate the quality policy and strategies into measurable
activities. (ICH Q10)
Quality Planning: Part of quality management focused on setting quality objectives and
specifying necessary operational processes and related resources to fulfill the quality objectives.
(ISO 9000:2005)
Quality Policy: Overall intentions and direction of an organization related to quality as formally
expressed by senior management. (ISO 9000:2005)
Quality Risk Management: A systematic process for the assessment, control, communication,
and review of risks to the quality of the drug (medicinal) product across the product lifecycle.
(ICH Q9)
Senior Management: Person(s) who direct and control a company or site at the highest levels
with the authority and responsibility to mobilize resources within the company or site. (ICH Q10
based in part on ISO 9000:2005)
16
State of Control: A condition in which the set of controls consistently provides assurance of
continued process performance and product quality. (ICH Q10)
17
Annex I: Potential Opportunities To Enhance Science- and Risk-Based
Regulatory Approaches *
*Note: This annex reflects potential opportunities to enhance regulatory approaches. The actual
regulatory process will be determined by region.
Scenario Potential Opportunity

1. Comply with GMPs Compliance – status quo
2. Demonstrate effective Opportunity to:
pharmaceutical quality system, • increase use of risk-based
including effective use of quality approaches for regulatory
risk management principles (e.g., inspections
ICH Q9 and ICH Q10).
3. Demonstrate product and process Opportunity to:
understanding, including effective • facilitate science-based
use of quality risk management pharmaceutical quality assessment
principles (e.g., ICH Q8 and ICH • enable innovative approaches to
Q9). process validation
• establish real-time release
mechanisms
4. Demonstrate effective Opportunity to:
pharmaceutical quality system and • increase use of risk-based
product and process understanding, approaches for regulatory
including the use of quality risk inspections;
management principles (e.g., ICH • facilitate science-based
Q8, ICH Q9, and ICH Q10). pharmaceutical quality assessment;
• optimize science- and risk-based
postapproval change processes to
maximize benefits from innovation
and continual improvement;
• enable innovative approaches to
process validation;
• establish real-time release
mechanisms.
18
Annex 2
Diagram of the ICH Q10 Pharmaceutical Quality System Model
ICH Q10 Pharmaceutical Quality System

Investigational products
GMP
Management Responsibilities
Process Performance & Product Quality Monitoring System

PQS Corrective Action / Preventive Action (CAPA) System
elements Change Management System
Management Review
Knowledge Management
Enablers
Quality Risk Management
This diagram illustrates the major features of the ICH Q10 Pharmaceutical Quality System
(PQS) model. The PQS covers the entire lifecycle of a product including pharmaceutical
development, technology transfer, commercial manufacturing, and product discontinuation as
illustrated by the upper portion of the diagram. The PQS augments regional GMPs as illustrated
in the diagram. The diagram also illustrates that regional GMPs apply to the manufacture of
investigational products.
The next horizontal bar illustrates the importance of management responsibilities explained in
section III (2) to all stages of the product lifecycle. The following horizontal bar lists the PQS
elements that serve as the major pillars under the PQS model. These elements should be applied
appropriately and proportionally to each lifecycle stage, recognizing opportunities to identify
areas for continual improvement.
The bottom set of horizontal bars illustrates the enablers: knowledge management and quality
risk management, which are applicable throughout the lifecycle stages. These enablers support
the PQS goals of achieving product realization, establishing and maintaining a state of control,
and facilitating continual improvement.
19
Appendix E: GHTF Quality Management System —

Medical Devices — Guidance on Corrective Action
and Preventive Action and Related QMS Processes
GHTF/SG3/N18:2010
FINAL DOCUMENT
Global Harmonization Task Force
Title: Quality management system –Medical Devices – Guidance on corrective

action and preventive action and related QMS processes
Authoring Group: Study Group 3
Date: 4 November 2010
Dr. Larry Kelly, GHTF Chair
The document herein was produced by the Global Harmonization Task Force, which is comprised of representatives
from medical device regulatory agencies and the regulated industry. The document is intended to provide non-
binding guidance for use in the regulation of medical devices, and has been subject to consultation throughout its
development.
There are no restrictions on the reproduction, distribution or use of this document; however, incorporation of this
document, in part or in whole, into any other document, or its translation into languages other than English, does not
convey or represent an endorsement of any kind by the Global Harmonization Task Force.
Copyright © 2010 by the Global Harmonization Task Force

Guidance on corrective action and preventive action and related QMS processes GHTF/SG3/N18:2010
Table of Contents
Preface ........................................................................................................................................................................3
Introduction.................................................................................................................................................................3
1.0 Scope ...............................................................................................................................................................4
2.0 Definitions.......................................................................................................................................................4
2.1 Correction .........................................................................................................................................4
2.2 Corrective action...............................................................................................................................4
2.3 Data Sources .....................................................................................................................................4
2.4 Concession ........................................................................................................................................4
2.5 Preventive action...............................................................................................................................5
2.6 Nonconformity..................................................................................................................................5
2.7 Verification .......................................................................................................................................5
2.8 Validation..........................................................................................................................................5
3.0 Overview .........................................................................................................................................................5
4.0 Phase I: Planning.............................................................................................................................................7
4.1 Plan for Measurement, Analysis and Improvement Processes..........................................................8
4.2 Establish Data Sources and Criteria ..................................................................................................9
5.0 Phase II: Measurement and Analysis within and across Data Sources..........................................................10
5.1 Measure...........................................................................................................................................10
5.2 Analyze ...........................................................................................................................................10
6.0 Phase III: Improvement .................................................................................................................................14
6.1 Investigate .......................................................................................................................................14
6.2 Identify Root Cause ........................................................................................................................16
6.3 Identify Actions ..............................................................................................................................17
6.4 Verify Identified Actions ................................................................................................................18
6.5 Implement Actions..........................................................................................................................18
6.6 Determine Effectiveness of Implemented Actions..........................................................................19
7.0 Phase IV: Input to Management ....................................................................................................................19
7.1 Report to Management....................................................................................................................19
7.2 Management Review.......................................................................................................................20
Annex A: Examples of Phase Activities ...................................................................................................................21
Annex B: Examples of Data Sources and Data Elements.........................................................................................22
Annex C: Examples of Contributing Factors............................................................................................................24
Annex D: Examples for Documentation of the Improvement Processes ..................................................................25
November 4, 2010 Page 2 of 26

Preface
The document herein was produced by the Global Harmonization Task Force, a voluntary group
of representatives from medical device regulatory agencies and the regulated industry. The doc-
ument is intended to provide non-binding guidance for use in the regulation of medical devices,
and has been subject to consultation throughout its development.
There are no restrictions on the reproduction, distribution or use of this document; however, in-
corporation of this document, in part or in whole, into any other document, or its translation into
languages other than English, does not convey or represent an endorsement of any kind by the
Global Harmonization Task Force.
Introduction
This guidance document is intended for medical device manufacturers and regulatory authorities.
It is intended for educational purposes and is not intended to be used to assess or audit compli-
ance with regulatory requirements. It is expected that the reader is familiar with regulatory Qual-
ity Management System (QMS) requirements within the medical devices sector.
For the purposes of this document it is assumed that the medical device manufacturer has a QMS
which requires the manufacturer to have documented processes to ensure that medical devices
placed on the market are safe and effective. For example ISO13485 Medical Devices – Quality
Management Systems – Requirements for regulatory purposes, Japanese Ministerial Ordinance
on Standards for Manufacturing Control and Quality Control for Medical Devices and in vitro
Diagnostics (MHLW 1 Ministerial Ordinance No. 169), the FDA 2 Quality System Regulation 21
CFR Part 820 or the respective quality system requirements of the European medical Device Di-
rectives.
For this purpose the manufacturer will establish processes and define appropriate controls for
measurement and analysis to identify nonconformities and potential nonconformities. Also, the
manufacturer should establish processes defining when and how corrections, corrective actions,
or preventive actions should be undertaken. These actions should be commensurate with the sig-
nificance or risk of the nonconformity or potential nonconformity.
The terms risk, risk management and related terminology utilized within this document are in
accordance with ISO 14971 “Medical Devices-Application of Risk Management to Medical De-
vices.”
The acronym “CAPA” will not be used in this document because the concept of corrective action
and preventive action has been incorrectly interpreted to assume that a preventive action is re-
quired for every corrective action.
1
Japanese Ministry of Health Labor and Welfare
2
US Food and Drug Administration

This document will discuss the escalation process from different “reactive” sources which will
be corrective in nature and other “proactive” sources which will be preventive in nature. The
manufacturer is required to account for both types of data sources whether they are of a correc-
tive or preventive nature.
Regardless of the nature of the data source, if there is a decision to escalate the information to
further evaluation and investigation, the steps of investigation, identification of root causes and
actions needed, verification, implementation, and effectiveness checks will be similar.
This guidance document will describe measurement, analysis and improvement as complete and
integrated processes.
1.0 Scope
This document provides guidance for establishing adequate processes for measurement, analysis
and improvement within the QMS as related to correction and/or corrective action for noncon-
formities or preventive action for potential nonconformities of systems, processes or products.
2.0 Definitions
The references to clauses in this section refer to ISO 9000:2005.
2.1 Correction
Action to eliminate a detected nonconformity (3.6.2)

Note 1 A correction can be made in conjunction with corrective action (3.6.5)
Note 2 Corrections can be, for example, rework (3.6.7) or re-grade (3.6.8)
2.2 Corrective action
Action to eliminate the cause of a detected nonconformity (3.6.2) or other undesirable situ-
ation
Note 1 There can be more than one cause for nonconformity
Note 2 Corrective action is taken to prevent recurrence whereas preventive ac-
tion (3.6.4) is taken to prevent occurrence
Note 3 There is a distinction between correction (3.6.6) and corrective action
2.3 Data Sources
The processes within a Quality Management System that provide quality information that
could be used to identify nonconformities, or potential nonconformities
2.4 Concession
Permission to use or release a product that does not conform to specified requirements
(3.6.11).

2.5 Preventive action
Action to eliminate the cause of a potential nonconformity (3.6.2) or other undesirable sit-
uation
Note 1 There can be more than one cause for nonconformity
Note 2 Preventive action is taken to prevent occurrence whereas corrective ac-
tion (3.6.5) is taken to prevent recurrence
2.6 Nonconformity
Non fulfillment of a requirement (3.1.2)
2.7 Verification
Confirmation through provision of objective evidence (3.8.1) that specified requirements

(3.1.2) have been fulfilled
Note 1 The term “verified” is used to designate the corresponding status.
Note 2 Confirmation can comprise activities such as:
- performing alternative calculations,
- comparing a new design specification (3.7.3) with a similar proven
design specification, undertaking tests (3.8.3), performing demonstra-
tions, and reviewing and approving documents prior to issue.
2.8 Validation
Confirmation through provision of objective evidence (3.8.1) that the requirements for a
specific intended use or application have been fulfilled
Note 1 The term “validated” is used to designate the corresponding status.
Note 2 The use conditions for validation can be real or simulated.
3.0 Overview
The manufacturer is responsible for the implementation and maintenance of a QMS which en-
ables their organization to provide safe and effective medical devices meeting customer and
regulatory requirements.
A nonconformity as defined in 2.6 is a non fulfillment of a requirement. It is important to under-

stand that requirements may relate to product, process or the QMS.
When a nonconformity is identified, the manufacturer will determine the significance, the asso-
ciated risk and the potential for recurrence.
Once these have been determined the manufacturer may decide the nonconformity has little as-
sociated risk or is unlikely to recur. In such cases the manufacturer may decide only to carry out
a correction.

Should the nonconformity recur within the QMS, during manufacture or after the medical device
has been delivered to a customer, it is an indication that improvement action(s) may be needed.
In either case the QMS requires that a corrective action should be carried out with the aim to
prevent recurrence. The corrective action may be as simple as retraining, or as complex as redes-
igning the manufacturing process.
The manufacturer may encounter situations that have not actually caused a nonconformity, but
may do so in the future. Such situations may call for preventive action. For example, production
or acceptance testing trend data indicates that control limits are being approached and revision of
product or production (process, equipment or facilities) requirements may be necessary. These
revisions could constitute a preventive action. Preventive action would not include planned
process adjustments intended to return process performance to nominal values from the edges of
the process control range.
Actions taken to eliminate observed nonconformities within the scope of a single QMS (regard-
less of whether the actions are taken at more than one site or facility operating within that QMS)
would be considered corrective actions. However, similar actions applied within another QMS
(regardless of whether it is the same site, facility, or organization) that has not yet experienced
these nonconformities, would be considered preventive actions.
Figure 1 illustrates typical Phases to be considered when planning, implementing and maintain-
ing effective processes for measurement, analysis, improvement and providing input to manage-
ment. See Annex A for a list of possible activities corresponding to the phases in Figure 1.
As a check on the effectiveness of the processes defined, management should regularly review
the outputs of processes and make adjustments as needed.
Documented procedures, requirements and records should be maintained by the manufacturer to

ensure and demonstrate the effective planning, operation and control of the processes. Docu-
mented evidence of decisions and actions taken will be a part of the QMS.

Figure 1: Processes for measurement, analysis and improvement
4.0 Phase I: Planning

Planning involves specifying processes and associated resources in order to meet specific objec-
tives. Factors to consider during the planning phase should be aligned with the manufacturer’s
overall business planning and include the device’s intended use, markets and users, as well as
regulatory requirements.
The involvement of management at appropriate levels (e.g. review, approval) in actions taken in
response to nonconformities or potential nonconformities should be established. Management
should ensure that measurement criteria are defined for identified data sources and communi-
cated across the organization.

4.1 Plan for Measurement, Analysis and Improvement Processes

Factors to consider during this planning phase should be aligned with the manufacturer’s overall
business planning and as a minimum include the type of device being manufactured, intended
markets and users, and regulatory requirements. As part of planning, management should review
the processes critical to the operations with regard to quality and regulatory requirements and
select relevant data sources to measure, analyze and facilitate improvement as necessary.
In the process of planning measurement and analysis, a manufacturer needs to take into account
data sources, the measurement of the data elements within each data source, the frequency of
monitoring, and the analysis to be performed within a data source, or across data sources.
The measurement of data elements should be done in a way that ensures the manufacturer is ef-
fective in managing the operations and maintain an effective QMS. Each of the data elements
should be planned and established with specific requirements for measurement that are moni-
tored routinely.
The scope of the QMS and the scope of the measurement, analysis and improvement processes
will provide the boundaries as to whether the data source is reactive/corrective or proac-
tive/preventive.
The planning phase should ensure the following:

Identification of relevant internal and external data sources that are indicators of process
and product performance.
Provision for adequate resources and establish responsibilities and authorities to enable
the necessary actions. Resources may include technical experts, testing laboratories, data
management, infrastructure, training, etc.
Definition of requirements for each identified data source, including limits, acceptance
criteria, escalation criteria and mechanisms for reporting of nonconformities or potential
nonconformities.
Analysis of data elements within data sources.
Coordination and analysis of data across data sources.
For each data element individual criteria should be defined; however, criteria may be defined for
a combination of data elements. Criteria should be quantitative whenever possible in order to
maximize consistency and reproducibility for subsequent analysis. If the criteria and data are qu-
alitative, subjectivity should be eliminated or minimized.
Acceptance criteria should be based on system, product and process specifications or require-
ments which are typically identified during design and development activities. This includes the
design of the Quality Management System, development and maintenance of assembly proc-
esses, delivery processes, servicing and installation processes.
Escalation criteria used for the purpose of initiating the improvement process (see 6.0) may often
be called action levels, trigger points, thresholds, etc. These escalation criteria should be proce-
duralized and would likely include certain generic action levels as well as specific action levels
resulting from risk management activities. In particular, criteria should be established for imme-

diate escalation. For example, an incident alleging a death or serious injury should be escalated
to the improvement phase (see 6.0) for immediate action.
For new technology and existing technologies with new intended uses/applications, initial escala-
tion criteria may be difficult to define for the monitoring process. Therefore a manufacturer
should plan for resources to analyze information in order to confirm initial assumptions and es-
tablish or revise escalation criteria.
Planning should provide for confirmation that the defined limits, acceptance criteria, escalation
criteria and mechanisms for reporting of nonconformities or potential nonconformities for the
original data sources and data elements are still appropriate. Where new data sources need to be
established, confirm that they have been identified and their criteria defined.
4.2 Establish Data Sources and Criteria

The manufacturer should identify and document relevant data sources and their data elements,
both internal and external to the organization. Data elements provide information regarding non-
conformities, potential nonconformities and the effectiveness of the established processes within
the data sources.
Examples of data sources can be, but are not restricted to:
Regulatory Requirements
Management Review
Supplier (performance/controls)
Complaint Handling
Adverse Event Reporting
Process Controls
Finished Product
Quality Audits (internal/external)
Product Recall
Spare Parts Usage
Service Reports
Returned Product
Market/Customer Surveys
Scientific Literature
Media Sources
Product Realization (design, purchasing, production and service and customer informa-
tion)
Risk Management
For further examples of data elements see Annex B.
When an issue is identified in one of the data sources, it is also important that the manufacturer
identify and review related information from other data sources across the organization. Fur-
thermore a review of information from external data sources should also be considered. The ag-
gregation of information from more than the original data source may lead to more comprehen-

sive knowledge. With this knowledge base a manufacturer will be positioned to better determine
appropriate action.
5.0 Phase II: Measurement and Analysis within and across Data Sources
Once data sources, data elements and acceptance criteria have been specified, as part of the plan-
ning process, the manufacturer is required to perform measurement, monitoring and analysis
processes to determine conformity or nonconformity.
Software used in measurement, monitoring and analysis, whether purchased (Off-The-Shelf) or

custom developed, should be validated for its intended use.
For example, a customer survey conducted by the marketing department, indicated that there was
a general dissatisfaction with the packaging of product X. When investigated further (within and
across other data sources) and reviewed with other data from complaints, returned product and
service reports, it became evident that there was a potential for misuse, unsafe use, or damage to
the device as a result of the current packaging design. As the result of this analysis, escalation to
Phase III (see 6.0) for preventive action may be appropriate.
5.1 Measure
For the purpose of this guidance, measurement is a set of operations to determine a value of a
data element (i.e. quantity, quality).
Data collected from the measurement of product, process and QMS are acquired throughout the
life-cycle of the product. The manufacturer should define for example frequency of the meas-
urement, precision and accuracy of the data. The manufacturer should also ensure that the data
collected is current and relevant.
Measurement data should be retained as a quality record. The manufacturer should maintain the
data in a form that is retrievable, suitable for analysis and meets both QMS and regulatory re-
quirements.
Monitoring is the systematic and regular collection of a measurement. The manufacturer should
define during the planning phase what, when and how data should be monitored. The data should
be defined such that it can be analyzed for further action. The monitoring of data may be con-
tinuous or periodic, depending on the type of data source and elements. The monitoring proc-
esses should be periodically reviewed for their continued suitability.
5.2 Analyze
For the purpose of this guidance, analysis is a systematic review and evaluation of data from
measurements to derive a conclusion.

The manufacturer should have documented procedures for the analysis of data against the estab-
lished criteria (see 4.1). Analysis is performed to identify nonconformity or potential noncon-
formity or identify areas where further investigation should be initiated. In addition analysis is
used to demonstrate the suitability and effectiveness of product, process and QMS. Analysis can
be performed utilizing analytical tools, a team of experts, process owners or independent review-
ers. The results of the analysis should be documented.
After it is determined what will be measured, statistical techniques should be identified to help
understand variability and thereby help the manufacturer to maintain or improve effectiveness
and efficiency. These techniques also facilitate better use of available data to assist in decision
making. Statistical techniques assist in identifying, measuring, analyzing, interpreting and mod-
eling variability.
For the analysis of nonconformity, appropriate statistical and non-statistical techniques can be
applied. Examples for statistical techniques are:
Statistical Process Control (SPC) charts
Pareto analysis
Data trending
Linear and non-linear regression analysis
Experimental design (DOE – Design of Experiments) and analysis of variance
Graphical methods (histograms, scatter plots, etc.)
Non-statistical techniques are for example:

Management reviews
Results from quality meetings
Safety committees (internal/external)
Failure Mode and Effect Analysis (FMEA)
Fault Tree Analysis (FTA)
Analysis will likely occur at several different points (time and/or organizational level). For ex-
ample, a certain amount of analysis and possible failure investigation (where there is evidence of
a nonconformity) will occur for each data source.
In addition to the analysis within the data sources there should also be a level of analysis across
data sources to determine the extent and significance of nonconformity or potential nonconform-
ity. The linkage of data from different data sources will be referred to as “horizontal analysis”.
The horizontal analysis may:
determine that the action proposed from the data source analysis is appropriate without
further progress into Phase III (see 6.0); or,
provide additional information warranting progress into Phase III (see 6.0), regardless of
whether the data source analysis escalated the nonconformity or potential nonconformity.
The outcome of measurement and analysis leads to different scenarios as shown in Figure 2.

Figure 2: Outcomes of measurement and analysis
The following tables provide more details to support the use of Figure 2. Each scenario is de-
scribed with an example showing the different outcome of measurement and analysis.
Basic The documentation requirements in a research design and development procedure

Example were not followed. The missing documentation involves changing to a different
supplier of an electronic board. The requirement is to document the supplier name
and supplier number in the research report.
Scenario A No correction required, continue measurement and monitoring

The decision is made not to take any correction nor escalate the handling of the nonconformity to
Phase III (see 6.0).
Example
Nonconformity The supplier number was not included in the research report. (however, the
supplier name is documented).
Key Results of Analysis indicates that the procedure is adequate and well known to the us-
Measurement ers of the research procedure.
and Analysis Following a review of the issue this appears to be a one time oversight.
The intent of the requirement is for convenience only.
Conclusion No initial correction - It is not necessary to update the research report, as the
supplier is documented by name, hence traceability is maintained.
Do not escalate to Phase III.
Scenario B Correction required, continue measurement and monitoring

The decision is made to perform a correction but not to escalate the handling of the nonconformity
to Phase III (see 6.0).
Example
Nonconformity The supplier name and number was not included in the research report.
Key Results of Analysis indicates that the procedure is adequate and well known to the us-
Measurement ers of the research procedure.
and Analysis Following a review of the issue this appears to be a one time oversight.
The intent of the requirement is to ensure traceability to the supplier and this
could be lost if the research report is not updated.
Conclusion Take an initial correction to update the research report with the supplier
name and number.
Do not escalate to Phase III.

Scenario C Correction and escalation to further investigation under the improvement phase.
The decision is made to perform an initial correction. However, there is a need for escalation to
Phase III (see 6.0) to further investigate as a result of the analysis performed in order to determine
the appropriate corrective action.
Example
Key Results of Analysis indicates that the procedure may not be adequate and it is not well
Measurement know to the users of the research procedure. The issue has been identified in
and Analysis multiple reports.
In some cases, traceability to the supplier could be established via other
means, and in other cases it could not.
Conclusion Take an initial correction to update the research report with the supplier
name and number (in the cases where the supplier could be identified).
Escalate to Phase III for corrective action.
Scenario D Escalation for further investigation under the improvement phase.

The decision is made that there is not enough information at this time to determine the required
action. Therefore the investigation is escalated to Phase III.
Example
Key Results of Analysis indicates that the procedure may not be adequate and it is not well
Measurement know to the users of the research procedure. The issue has been identified in
and Analysis multiple reports.
Traceability to the supplier could not be established via other means in any
of the cases.
Conclusion No initial correction - The supplier is not known so an initial correction can-
not be taken at this time.
Escalate to Phase III for corrective action.
Documented procedures should clearly delineate and define when escalation to Phase III is re-
quired.
Typically manufacturers have organizational groups or processes surrounding some of their main
data sources (e.g. complaint handling, handling of nonconformities, material review boards,
change management process). Within these groups or processes certain activities described in
Figure 2 can be implemented without escalation.
There may be predefined events that due to the significance of the risk will be escalated to Phase
III without any delay that can not be justified. In the event a potential nonconformity is identi-
fied, it may be escalated into Phase III (see 6.0) for consideration of actions to prevent the occur-
rence of the potential nonconformity.
When no correction or only corrections within these groups or processes are taken, there needs to
be data source monitoring and analysis (e.g. trending) to determine if escalation to Phase III may
be necessary from accumulated information. Whenever an issue is escalated to Phase III, any in-
formation gained within the defined activities of these groups or processes should be an input to
the Phase III activities such as Investigation (see 6.1) or Identified Actions (see 6.3).

6.0 Phase III: Improvement

The improvement phase of a corrective action process or preventive action process is designed to
eliminate or mitigate a nonconformity or potential nonconformity.
The improvement activities are dependant on the specific nonconformity or potential noncon-
formity. Any previous data from Phase II should be utilized as input to the Phase III process.
The improvement phase and the activities described in Figure 3 needs to be documented. Im-
provement generally involves the following activities that the manufacturer would take sequen-
tially or sometimes simultaneously:
A thorough investigation of the reported nonconformity
An in-depth root cause analysis
Identification of appropriate actions
Verification of identified actions
Implementation of actions
Effectiveness check of implemented actions
6.1 Investigate 6.2 Identify Root Cause
6.6 Determine
6.3 Identify Actions
Effectiveness of Improvement
Implemented Actions
6.4 Verification of
6.5 Implement Actions identified Actions
Figure 3: Phase III – Improvement
6.1 Investigate
The purpose of investigation is to determine the root cause of existing or potential non –
conformities, whenever possible, and to provide recommendations of solutions. The magni-
tude/scope of the investigation should be commensurate with the determined risk of the noncon-
formity.
Good practice shows that a documented plan should be in place prior to conducting the investi-
gation (see Annex D for examples). The plan should include:
Description of the nonconformity expressed as a problem statement
Scope of the investigation
Investigation team and their responsibilities
Description of activities to be performed
Resources
Methods and tools
Timeframe
From the information obtained throughout the process the problem statement should be reviewed
and refined as appropriate.
The investigation should:

Determine the extent of the nonconformity or potential nonconformity
Acknowledge that there are likely to be several causes of an event; hence, the investiga-
tion should not cease prematurely
Require that symptoms be distinguished from root causes and advocate the treatment of
root causes rather than just the symptoms
Require that an end point be defined for the investigation. An overly exhaustive investi-
gation may unduly delay the correction of non-conformity or unnecessarily incur addi-
tional cost. (For example, if removal of the causes identified so far will correct 80% of
the effects then it is likely that the significant causes have been identified (Pareto rule))
Take into account the output of relevant risk management activities
Agree on the form of evidence. For example, evidence should support:
- the seriousness of the event
- the likelihood of occurrence of the event
- the significance of the consequences flowing from the event
The investigation should include the collection of data to facilitate analysis and should build
upon any analysis, evaluation and investigation that were previously performed (see 5.0). This
will require the investigator to identify, define and further document the observed effects/non-
conformity, or already determined causes, to ensure that the investigator understands the context
and extent of the investigation. It may be necessary to:
Review and clarify the information provided
Review any additional information available from an horizontal analysis
Consider whether this is a systemic issue/non-systemic issue
Gather additional evidence, if required
Interview process owners/operators or other parties involved
Review documents
Inspect facilities, or the environment of the event
Previous investigations should be reviewed in order to determine if the event is a new problem or
the recurrence of a previous problem where, for example, an ineffective solution was imple-
mented. The following questions will assist in making the determination:
Is the nonconformity from a single data source?
Does the current nonconformity correlate with nonconformities from other data sources?
Are multiple data sources identifying the same nonconformity?
Do other nonconformities have an effect on the problem investigated here?
Many of the tools used in investigations rely upon a cause and effect relationship between an
event and a symptom of that event. To ensure that causes are identified, not symptoms, the fol-
lowing should be considered:
There must be a clear description of a cause and its effect. The link between the cause
and the undesirable outcome needs to be described.
Each description of a cause must also describe the combined conditions that contribute to
the undesired effect

A failure to act is only considered a cause if there was a pre-existing requirement to act. The re-
quirement to act may arise from a procedure, or may also arise from regulations, standards or
guidelines for practice, or other reasonably expected actions.
Some of the more common tools and techniques include:

Cause and effect diagrams
5 Why’s analysis
Pareto charting
Fishbone/Ishikawa cause and effect diagrams
Change analysis
Risk analysis techniques
Is/Is Not
The outcome of an investigation should include:

Clearly defined problem statement
What information was gathered, reviewed and/or evaluated
Results of the reviews/evaluations of the information
Identification of cause(s) or contributing factors
Solutions to address the cause(s) or contributing factor(s)
6.2 Identify Root Cause

Causes or contributing factors of detected nonconformity or potential nonconformity should
promptly be identified so that corrective action can be taken to prevent recurrence, or preventive
action taken to prevent occurrence. The process to identify the root cause should start with the
output(s) of the investigation (see 6.1).
When assessing relevant data, the following should be considered:

Systematic generation of cause and effect conclusions supported by documented evidence
Evaluate significant or underlying causes and their relationship to the problem
Ensure that causes are identified, not the symptoms
Check for more than one root cause (above processes if necessary)
Causes or contributing factors of nonconformities or potential nonconformities may include the

following:
Failure of, or malfunction of, incoming materials, processes, tools, equipment or facilities
in which products are processed, stored or handled, including the equipment and systems
therein
Inadequate or non-existent procedures and documentation
Non-compliance with procedures
Inadequate process control
Inadequate scheduling
Lack of training
Inadequate working conditions
Inadequate resources (human or material)
(Inherent) process variability

For further details on aspects to be considered when doing the root cause analysis see Annex C.
The output of the root cause analysis should be a clear statement of the most fundamental
cause(s) resulting in the nonconformity (see Annex D for examples).
6.3 Identify Actions

When the root cause(s) has been determined, the manufacturer should identify and document the
necessary corrections and/or corrective actions or preventive actions. These actions should be
reviewed to ensure that all necessary actions are identified. The review may benefit from a cross
functional approach. Where applicable, product disposition decisions should also be docu-
mented.
The following outcomes are possible and should be documented:
No further action necessary

(provided that no safety issue exists and regulatory requirements are met)
- With continuous monitoring
- Acceptance under concession and continuance of monitoring
Correction
It may be necessary to take initial corrections (e.g. containment, stop of shipment/supply,
issuance of advisory notice) in order to address an immediate risk or safety issue. This
may be necessary before investigation has been completed and root cause has been de-
termined. However, after investigation and root cause determination, additional and/or
possibly different corrections may become necessary.
Corrective action
Corrective action should address systemic problems. For example, changing the proce-
dure and training of personnel to the revised procedure may not, by itself, be appropriate
or sufficient to address the systemic cause(s).
Preventive action
By its very nature preventive action can not follow a nonconformity.
As a result of this step, a list of action items should be documented. These may include:
A detailed description of the implementation
Review regulatory requirements (e.g. submissions, licensing, certifications)
Roles and responsibilities for execution of action items
Identification of the necessary resources (e.g. IT, infrastructure, work environment)
Verification and/or validation protocols of the action(s) with acceptance criteria
Implementation schedule, including timelines
Method or data for the determination of effectiveness with acceptance criteria
Identify the starting point of monitoring, and end point of correction and/or corrective ac-
tion or preventive action as described above

6.4 Verify Identified Actions

Before the implementation of action(s), a manufacturer should verify the identified action(s) and
approve their implementation. In addition validation may be required where process validation or
re-validation may be necessary, or where user needs or intended uses are changed and design va-
lidation will be required.
Verification activities are to ensure that all the elements of the proposed action (documentation,
training etc) will satisfy the requirements of the proposed action. These activities should be per-
formed by persons who are knowledgeable in the design or use of the product or process that is
the subject of corrective or preventive action. Verification of a preventive action can be accom-
plished by introducing the conditions that would induce a nonconformity and confirming that the
nonconformity does not occur.
Validation activities generate data and information that confirm the likelihood of the effective-
ness of the corrective action to eliminate the nonconformity or proposed nonconformity.
Examples of items to be considered when planning the verification/validation activities include:

Does the action(s) eliminate the identified root cause(s)?
Does the action(s) cover all affected products/processes?
Does the action(s) adversely affect the final products?
Is it possible to finalize the actions timely in planned schedule
(resources, materials/kits, logistics, communications, etc.)?
Is the execution of the action commensurate with the degree
of risk previously established?
Are new risks or nonconformities derived from the action?
6.5 Implement Actions

The following items that may be considered at implementation should be documented:
Parties involved
Materials
Processes
Training
Communications
Tools
Timelines for the implementation of the approved action
Verify that the implementation has been completed.

6.6 Determine Effectiveness of Implemented Actions
The manufacturer should gather data over a period of time related to the effectiveness of the im-
plemented action (see Annex D for examples).
Management should ensure and be involved in a review and confirmation that actions taken were
effective and did not introduce new issues or concerns. The following questions should be con-
sidered at appropriate times throughout the process and be revisited in the final review:
Has the problem been comprehensively identified?
Has the extent of the problem been identified (e.g. range of affected devices, patient out-
come, process, production lines, operator)?
Have the root cause/contributing factors of the problem been identified and addressed?
Has the improvement action(s) been defined, planned, documented, verified and imple-
mented?
If the manufacturer finds the actions are not effective, the manufacturer should re-initiate Phase
III activities (see 6.0). If the manufacturer finds the actions create a new issue or a new noncon-
formity then the manufacturer needs to initiate Phase II (see 5.0) activities.
7.0 Phase IV: Input to Management

Management at different levels in the organization should be involved in each improvement ac-
tion either through approval of the improvement steps or reporting.
The Management Review is the overall mechanism for management to ensure that the Quality
Management System as a whole is effective.
7.1 Report to Management

The manufacturer should have a mechanism/procedure that expeditiously raises safety related
issues or other high risk issues to management. These issues can be identified in the data sources,
the improvement phase (see 6.0), or originate from other sources external to the Quality Man-
agement System. In addition to this expeditious escalation mechanism, the manufacturer should
define management and personnel responsibilities (i.e. process owner) for the measurement,
analysis and improvement processes, to ensure that the processes and the actions being imple-
mented are effective. For this purpose there needs to be a mechanism for management at differ-
ent levels to stay informed of the information or data from:
The measurement and analysis activities from the individual data sources
The investigations, actions, implementations, etc. from the improvement processes

7.2 Management Review

The manufacturer has procedures for what is provided as input for the management review, in-
cluding relevant information from the improvement processes, such as improvement actions
(corrective actions, and/or preventive actions) as well as important corrections.
The manufacturer needs to define what meaningful data is to be reported for a management re-
view. Data should be specific to the quality objectives of the manufacturer and be reported regu-
larly. Merely providing the number of improvement actions or the number of how many im-
provement actions are opened or closed to the management review process are not sufficient in
assessing the effectiveness of the processes.
Included in this review would be an assessment of any opportunities for improvement of the de-
vice, manufacturing process, QMS or the organization itself.
An outcome of the review could be the allocation of funding or personnel to a particular area,
project or device that the review has identified as not meeting customer and regulatory safety and
effectiveness expectations.

Annex A: Examples of Phase Activities

List of possible activities corresponding to the phases in Figure 1.
The following is an outline/aid memoir of the main points described in this document. It is not
intended as a “box ticking” exercise and should not be used as such, but used purely to summa-
rize and align the steps in the process described in this document. The activity numbers do not
imply sequential steps – some steps may take place in parallel.
The references in this Annex refer to the sections in this document.
Phase Activities
Planning 1. Identify all data sources (internal/external) by product type (4.1)
2. Identify resources required and individual personnel responsibilities for
measuring each data source (4.1)
3. Define the requirements for each data source and the data elements
within each data source that will be measured and analysed (4.1)
4. Define requirements for escalation to the improvement phase (4.1)
5. Define requirements for monitoring the measurements in the data
sources (5.1)
6. Establish data sources (4.2)
Measurement and Analysis 7. Measure and analyse all data sources for nonconformities and potential
within and across Data Sources nonconformities (5.0, 5.1 and 5.2)
8. Have reports of nonconformity or potential nonconformity come from
more than one data source?
9. Is the nonconformity or potential nonconformity systemic?
Improvement 10. Determine scope and required outcome of investigation (6.1)

11. Investigate nonconformity or potential nonconformity (6.1)
12. Analyse nonconformity or potential nonconformity for root cause(s)
(6.2)
13. Identify actions ( correction, corrective action or preventive action)
(6.3)
14. Verify proposed actions before implementation (6.4)
15. Implement proposed actions (6.5)
16. Determine effectiveness of actions (validate if possible) (6.6)
Input to Management 17. Report investigation and outcome to management (7.1)

18. Review investigation, analysis and outcome (6.6, 7.2)
19. If not satisfied return to step 10
20. If required, report to regulator (note: reporting may be required earlier
depending on severity)*
21. Audit system at determined intervals*
22. If numbers of nonconformities or potential nonconformities exceeds
targets, review all QMS processes*
*Steps 20 to 22 are not described in this document but are added as reminders of general management responsibilities
in this area of the QMS.

Annex B: Examples of Data Sources and Data Elements

Examples of data sources and their data elements can be, but are not restricted to:
Data Sources Data Elements
Regulatory Requirements Result of a regulatory inspection
New or revised regulatory requirements
Management Review Management review output
Supplier Number of batches received
Performance/Controls Batch and/or shipment
Inspection and test records
Quantity of rejects or deviations
Reason for rejection
By supplier, if more than one supplier
Use in which product or service
Supplier problems
Complaint Handling Quantity
By product family
By customer (physician, healthcare facility, patient, etc.)
Reason for complaint
Complaint codes
Severity
Component involved
Adverse Event Reporting Event
Quantity
By product family
By customer (physician, healthcare facility, patient, etc.)
Type of event (death or serious injury, etc.)
Component involved
Process Controls By product
Operator
Work shift
Equipment and/or instruments used
Inspection and test records
In-process control results
Process control parameters
Inspection process
Final acceptance
Rejects
Special process
Validation study results
Process monitoring observations
Finished Product Inspection and test records
Quality Audits Observations (number, category, corporate policy, regulatory requirements,
(internal/external) significance, etc.)
Repeat observations (indicative of effectiveness)
Closure times
Overall acceptability of contractor or supplier
Compliance to audit schedule
Audit personnel

Data Sources Data Elements

Product Recall Timeliness of recall communication
Classification of recall
Recall effectiveness checks
Spare Parts Usage Frequency of replacement
Batch number of spare part
By supplier of spare part, if more than one supplier
By customer
By location or area of customer
Service Reports Installation
First use of equipment
Frequency of maintenance visits
Types of repairs
Frequency of repairs
Usage frequency
Parts replaced
Service personnel
Returned Product Quantity
Reason for returning product
By customer
Types of defects identified on returned product
Market/Customer Sur- Customer preferences
veys Customer service response time
Solicited information on new or modified products
Scientific Literature Research papers
Media Sources Articles in trade journals
Product Realization Design and development review results
( Design, Purchasing, Design and development verification results
Production and Service Design and development validation results
and Customer informa- Design and development changes
tion) (reason or cause for change, effectiveness of change, etc.)
Controls on purchased products or services
(See above Supplier Performance/Controls)
Verification results of purchased product
Inspection and testing data of purchased product
Production and Service processes-
Cleaning operations of product and facilities
Sterilization
Installation results
Servicing and Maintenance if required (See also: Service Reports)
Verification and Validation results of processes used in production and
service. Including approval of equipment and qualification of personnel
Traceability Data
Controls of monitoring and measuring devices
Calibration and maintenance of equipment
Customer Information- New or repeat customer
Customer feedback maybe in other forms than complaints or returned
product (Customer Service call data, repeat sales , delivery/distribution data)
Risk Management Published reports/literature of failures of similar products
Stakeholder concerns and generally accepted state of the art
Risk acceptability criteria

Annex C: Examples of Contributing Factors

Examples of possible contributing factors to be considered when doing the root cause analysis:
Materials
Defective raw material (does material meet specification?)
Batch related problem
Design problem (wrong material for product, wrong specifications)
Supplier problem (lack of control at supplier, alternative supplier)
Lack of raw material.
Machine / Equipment
Incorrect tool selection – suitability
Inadequate maintenance or design – calibration?
Equipment used as intended by the manufacturer?
Defective equipment or tool
End of life?
Human error – inadequate training?
Environment
Orderly workplace
Properly controlled – temperature, humidity, pressure, cleanliness
Job design/layout of work
Management
Inadequate management involvement
Stress demands
Human factors
Hazards not properly guarded
Were management informed / did they take action?
Methods
Procedures not adequately defined
Practice does not follow prescribed methodology
Poor communications
Management system
Training or education lacking
Poor employee involvement
Poor recognition of hazard
Previous hazards not eliminated
Measurement, monitoring and improvement

Inadequate measuring and improvement

Annex D: Examples for Documentation of the Improvement Processes

The table below includes guidance for documenting various requirements of the improvement
processes.
Guidance Example Documentation

Problem Clearly defined problem statement. During in-process testing of Product A finished product on
Statement State how the issue was discov- [date], two devices out of 30 were found to be noncon-
ered. The process/procedure that forming per Design Document 123456, revision A. Note
was not followed. 2.1 in Design Document 123456 requires that the surface
finish be 32 µinch maximum on all exterior surfaces. The
Provide evidence two nonconforming devices had a surface finish above the
What, When, Who, Where and maximum 32 µinch finish as follows:
How much (as applicable) Serial Number 54321 had a surface finish of 67 µinch
Serial Number 65432 had a surface finish of up to 38
µinch
Correction General Examples The supplier was notified of the issue on [date].
Containment, The supplier conducted an operator awareness training of
Stop of shipment/ supply the incident on [date].
Issuance of advisory notice
Incident awareness / training Initial extent of the issue is restricted to supplier lot #678.
Change or suspend production All unused components and product built with components
process from this lot were controlled on [date]. No product built
with this lot had been distributed.
Investigate • Clearly defined problem statement See initial problem statement. Subsequent investigation
(update/refine if new information confirmed that the issue was limited to lot #678. All addi-
is determined) tional available lots of this component were inspected with
What information was gathered, a 95/95 inspection plan and no additional lots were con-
reviewed and/or evaluated firmed to have the issue.
Results of the reviews/evaluations The incoming inspection process and component FMEA
of the information were reviewed and determined to be adequate and accu-
Identification of cause(s) or con- rate, respectively.
tributing factors
Review of finished product reject data over the past year
revealed no other rejects for surface finish of this compo-
nent.
The following problem-solving tools and methods were

used during the course of the investigation of the surface
finish issue.
Fishbone analysis – see the attached file labeled ‘Sur-
face Finish Analysis’.
Conference calls and documentation reviews with the
Supplier – see attached file which contains the min-
utes from the conference calls.
Results of the investigation were the following. Two dif-

ferent raw tubing lots were mixed at the Supplier’s finish-
ing process. One raw tubing lot was intended for customer
A’s products (Lot number 10000-100 requiring a surface
finish of 32 µinch maximum) and the other was intended
for customer B’s product which had a surface finish above
the 32 µinch maximum.

Identify The output of the root cause analy- It has been concluded that the root cause of the tubing sur-
Root Cause sis should be a clear statement of face finish issue is inadequate line clearance procedures
the most fundamental cause(s) re- established at the supplier.
sulting in the nonconformity
Planned Specify: Corrective action: Supplier to add line clearance require-

actions What the action is ments to documented procedures by [date].
Who will do it
When it should be done Preventive action: Not applicable.
Verification Verification activities are to ensure General examples are included below. Actual documenta-
of actions that all the elements of the pro- tion would need to be more specific.
posed action (documentation, Review and approval of the procedural changes prior
training etc) will satisfy the re- to use
quirements of the proposed action Conduct a pilot of new procedure on a specific pro-
ject/department/time frame prior to full scale imple-
Validation activities generate data mentation
and information that confirm the Verification that the updated supplier procedure ad-
likelihood of the effectiveness of dresses the process that caused the nonconformity
the corrective action to eliminate Verification that the training materials address the
the nonconformity or proposed specific process that caused the nonconformity
nonconformity. Comparing a new design specification with a similar
proven design specification
Performing calculations using an alternative method
Perform validation of equipment, software, production
processes, test method, component, etc.
Specific example:
Review and approval of supplier procedure XXX by the
supplier and the customer to ensure adequacy of the up-
dated line clearance process.
Verification Method or data for the determination of X months after implementation:

of effective- effectiveness with acceptance criteria.
ness The improvement goal Conduct a query of the electronic manufacturing data
The evidence (data sources) that system to verify there are zero surface finish rejects
will be used to support effective- for this component at finished Product A final inspec-
ness (e.g., a data source could be tion.
where the problem was initially
found) Supplier Quality Engineer to conduct on site review at
The time frame that effectiveness the supplier of the action to confirm the procedures
will be monitored (e.g., upon com- are in place, are known to the operators, and there is
pletion of actions or three months, evidence that the procedures are being followed.
six months as appropriate)
OR
Sample size required to demon-
strate effectiveness
Winterhufen 1.0

300 N. Washington St., Suite 200, Falls Church, VA 22046, USA

Auditing For Quality Manufacturing

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Auditing For Quality Manufacturing

Uploaded by

Copyright:

Available Formats

Auditing for Quality

The Importance of Data Integrity

Figure 1. Data Integrity Principles and Regulatory Reference

Data Integrity Element Description 21 CFR Reference

Source: Regulatory Compliance Associates Inc., 2019

º Complete: Information that is critical to recreating and understanding an event, includ-

º Data integrity violations could include:

A Brief History of Data Integrity Issues

º The generic industry;

º Draft Pharmaceutical Inspection Co-operation Scheme (PIC/S) Guidance: Good Prac-

º The PDA’s Elements of a Code of Conduct for Data Integrity, 2015.

Creating a Culture of Quality

º Managers throughout the organization either fail to consistently emphasize quality or

º Training and development do not emphasize quality;

3. Communicating that quality is everyone’s responsibility at the management level;

Mitigating the Risk of Aging Facilities

Auditing an Aging Facility

º Wall, ceiling and floor composition and layout;

Making the Case for Modernization

Keeping Abreast of Changing Standards

Slowing the Aging Process

º Appropriate corrective actions and/or preventive actions (CAPAs) should be identified

º ISO 13485: Quality Management for Medical Devices;

Troubleshooting the CAPA Process

º Identifying issues: production and process nonconformances, complaints, audit obser-

º Performing risk assessment: investigating to the root cause;

Evaluate Need for Perform Root Issue CAR

Source: Regulatory Compliance Associates Inc., 2019

º Using attribute scales that are neither specific nor appropriate;

a process, a method, an environment, personnel, or technologies,

■■ training of personnel who use computerized systems and review

2. Aims and objectives of this guidance

disaster recovery and may be periodically overwritten. Such temporary back-up

contemporaneously recorded, original or a true copy and accurate. Assuring data

preclinical, clinical, manufacturing, testing, storage, distribution and post-market

documents, such as templates, forms and master documents, that may be

■■ application of modern QRM principles and good data management

4.12 Examples include, but are not restricted to:

5. Quality risk management to ensure

6. Management governance and quality audits

political, financial and other pressures or conflicts of interest that may

might reasonably be expected to inhibit the active and complete reporting

of data for all of these operations. Another example relates to identifying

7. Contracted organizations, suppliers

governance systems of the contract acceptor, through an audit or other

8. Training in good data and record management

8.2 In addition, key personnel, including managers, supervisors and quality

9. Good documentation practices

9.2 Attributable. Attributable means information is captured in the record so

10. Designing and validating systems to

Validation to assure good documentation

■■ documenting configuration specifications for commercial off-the-

Currently under review.

For systems to be used in clinical trials, configuration and

11. Managing data and records

■■ simple and streamlined;

11.6 Example consideration.

11.12 Whenever out-of-trend or atypical results are obtained they should be

11.14 Example considerations

11.16 Example consideration

12. Addressing data reliability issues

References and further reading

■■ PA/PH/OMCL (08) 89 R – Annex 3: Validation of computers as part of test equipment