US Food and Drug Administration: 06n-0464-tr00001

GOVERNMENT
OF
THE UNITED STATES OF AMERICA
+ + + + +
FOOD AND DRUG ADMINISTRATION
+ + + + +
PUBLIC HEARING
+ + + + +
ELECTRONIC SUBMISSION OF REGULATORY

INFORMATION AND CREATING AN ELECTRONIC
PLATFORM FOR ENHANCED INFORMATION MANAGEMENT
+ + + + +
Monday
December 18, 2006
+ + + + +
9:00 a.m.
PANELISTS PRESENT:
JANET WOODCOCK Deputy Commissioner

RANDALL LUTTER Panelist
RANDY LEVIN Panelist
ARMANDO OLIVA Panelist
KEVIN FAIN Panelist
LANA SKIRBOLL Panelist
KEN BUETOW Panelist
BARBARA MITTLEMAN Panelist
CLARK NARDINELLI Panelist
The transcript constitutes the

minutes from the public hearing held on
NEAL R. GROSS
COURT REPORTERS AND TRANSCRIBERS
1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com
December 18, 2006.
A-G-E-N-D-A
Presentations to the Panel
S. Albert Edwards
TAP Pharmaceutical Products...............10
Thomas W. Littlejohn
Piedmont Medical Group....................34
Ron Celeste, ThinSpring...................55
Tom Klaff, Surety.........................97
Nancy Smerkanich
Octagon Research Solutions...............113
Mark Scheineson, Alston & Bird, LLP......138
Debra Bremer, Pfizer.....................150
Diana McKenzie, Amgen....................157
Sue Dubman, Theravance...................161
Bob Beck, Fox Chase Cancer Center........167
Diane Paul, CRIX . . . . . . . . . . . 173
Dan Ruggles, Liaison Technologies . . . 178
Mark Adams, Booz Allen Hamilton . . . . 183
Ed Tripp, Abbott Laboratories . . . . . 211
Bill Rosen, Pfizer . . . . . . . . . . 225
John Rapoza, JRRapoza Associates . . . 253
Ari Kaliannan
Newtech Global Solutions . . . . . . . 255
Mark Rutkiewicz, AGA Medical. . . . . . 263

Presentations to the Panel (cont.)
Jason Rock, Global Submit . . . . . . . 287
Laurie Rose, SAS . . . . . . . . . . . 307
Harry Fisher, Northrop Grumman . . . . 328
Robert Cothran, Northrop Grumman . . . 335
Terence Zagar, Northrop Grumman . . . . 347
Public Comments
Bob Bard, Aastrom Biosciences . . . . . 363
Michael Brennan, Centocor . . . . . . . 367
Bill Kubeck, Steeple Laboratories . . . 368
Antoinette Azevedo
e-Submissions Solutions . . . . . . . . 372
Madeline Palla
Animal Health Institute . . . . . . . . 375
Stephanie Vatleit
Animal Drug Alliance . . . . . . . . . 376
NEAL R. GROSS
P-R-O-C-E-E-D-I-N-G-S
9:44 a.m.
DR. WOODCOCK: Good morning, I'm
Janet Woodcock. I'm Deputy Commissioner for
Operations at the Food and Drug
Administration and I'll be serving as the
presiding officer at this hearing. On behalf
of the Commissioner of Food and Drugs, Andrew
von Eschenbach, I'd like to welcome you to
this public hearing on electronic submission
of regulatory information and also on
creating an electronic platform for enhanced
information management.
With me today on the panel that
will be listening to the presentations are:
Dr. Ken Buetow who is Associate Director for
Bioinformatics and Information Technology at
the National Cancer Institute; Mr. Kevin Fain
who is Associate Chief Counsel, Food and Drug
Administration; Dr. Randy Levin, Director for
Health and Regulatory Data Standards at Food
and Drug Administration; Dr. Randy Lutter,
Associate Commissioner for Policy and

Planning at FDA; Dr. Armando Oliva, Deputy
Director for Bioinformatics in the Office of
Critical Path Programs at FDA; and Dr. Lana
Skirboll, Director of Policy and Planning at
NIH. And it's no accident that NIH and FDA
are together on this panel. We have a great
deal of both agencies' interest in these
various topics.
So first let me describe briefly
the issues we're going to be talking about
today and then I'll review the format for
this hearing. FDA and NIH is interested in
hearing about issues concerning the
feasibility and effect of an all-electronic
submission environment as well as issues
related to the electronic regulatory
information exchange platform. Over the last
decade FDA has been moving toward
transforming all regulatory submissions from
paper to electronic means. For example,
we've issued regulations related to voluntary
NEAL R. GROSS
electronic submissions of regulatory
information which many of you are familiar
with. We've also issued numerous guidance
documents to assist in the submission of
these various regulatory documents in
electronic format. We've also collaborated
with manufacturers, with standard development
organizations, with healthcare information
suppliers and other government agencies to
develop data standards and build databases
for sharing certain clinical trial
information. NIH has also in the last five
years been extremely interested in sharing
clinical trial information on behalf of its
investigators and the research it pursues.
To help us answer our questions
related to these two issues, we published in
the Federal Register on November 21, 2006, an
invitation to interested organizations and
individuals to participate in making
presentations at today's meeting specifically
addressing the questions that were listed in
the notice. These questions were divided

into the two topics. The first set of
questions addressed the transition from paper
submissions to electronic submissions,
related costs, interested in time spent and
how to implement this transition. Now we
recognize there are a broad range of FDA-
regulated products and that they're on a
spectrum of experiencing electronic
submissions right now. Some product areas
such as pharmaceuticals, there's a great deal
of electronic submission. In other areas,
much less to no electronic submission and few
associated standards. And therefore, we
recognize that whatever we do we're going to
have to take a step-wise approach to
implementation depending on the particular
area on where we're starting and that will
probably influence how long it would take to
get there.
Now the second set of questions
relate to the concept and feasibility of an
NEAL R. GROSS
electronic platform that would facilitate the
exchange of clinical research information and
regulatory product information and the role
of the public-private partnership in the
creation of such a platform. Our experience
to date has shown that, although we may have
the standards in certain areas for electronic
submission, we don't perhaps quite have the
network.
Fourteen people signed up today to
help answer those questions and we'll hear
from those folks first. When that is done
and if time permits we'll open the floor to
anyone else who may wish to address those
questions. If you're a scheduled speaker, we
request you stay within the allotted time and
I will make sure you do.
Before we go on, let me stress
this is a listening exercise for FDA and the
NIH. We really want to hear what you have to
say on these issues. We are having this
meeting transcribed, we hope, and the members
of this panel and their staff will pay

careful attention to what is said today as we
decide our future course. And I expect that
you can look forward to prompt action from us
on these matters. The direction will depend
in part upon the input we receive today. The
docket will continue to stay open until
February 16, 2007 to receive additional
comments.
Now there are several housekeeping
items I'd like to address. Number one,
please restrict yourselves to this room, the
hallways, the stairs if you like and the
lavatories, and do not move otherwise about
the building because it's an FDA facility and
there are other people working. At lunch we
have only an hour. There is across Fisher's
Lane a cafeteria-style public restaurant that
you can go to and there are some other
restaurants down. You go out this door and
go to your left. So B but we have talked to
the folks across the way and told them they
NEAL R. GROSS
may expect a larger group today, so hopefully
they will be ready and so we can all
reconvene promptly at the time for the
afternoon. So we have a very busy day and
I'd like to begin with our first speaker.
Okay, our first speaker is Dr. S.
Albert Edwards, who is the Director of
Regulatory Affairs at TAP Pharmaceutical
Products, Inc. He will speak for 30 minutes.
Dr. Edwards.
DR. EDWARDS: Okay, thank you very
much. I'm here today in my current role as
the Director of Regulatory Affairs and
Operations. If I were speaking to you 20-
some years ago, and I'm not going to divulge
the exact number of years, I could say I'm Al
Edwards and I'm here talking to you from the
FDA. So I've kind of sat on both sides of
things and I would say to you initially I
wish in my lifetime early on I had access to
electronic submissions to have done my job.
The greatest enemy I guess that I faced at
FDA in my time was the fact that I would get

to that very, very last volume in the NDA and
then I'd have to go into the documents room
and I'd have to try and find it. Documents
rooms then got closed and secured which was a
good idea at FDA at the time, but then it put
all the documents in the control of the
document clerk and if you wanted your volume
soon, you had to come up with an incentive
plan. And my incentive plan was to always
turn in my volume request with a suitable
incentive, okay? I'll talk more about more
formal incentives later, but those kind of
things were needed.
Back to present day. I want to
share with you eight observations that I have
about electronic submissions. Those span
about the 10 years of experience that I've
had and this next slide will show you what
TAP's experience has been completing about
700 ECTD submissions to date. We did a
complete conversion in September of `05.
NEAL R. GROSS
That conversion included all original INDs,
supplements and amendments. The conversion
also included all original INDs and
amendments. And the last thing we did was
convert all of our existing applications. So
we are a 100 percent ECTD shop.
FDA is our customer. I can
reinforce that because I sat on the other
side of the fence many years ago. I'm not
going to take time to read this entire slide,
and Dr. Woodcock, when I get to my allotted
time I know you're going to cut me off. So
everybody has my handout. Overall it has
taken TAP some time to put the ECTD process
in place. I want to emphasize that that's
process and technology on our side of the
fence. I suspect it will be on FDA's as
well. We have some improvements along the
way and that is we intend to control content
and meta-data from authoring through
submissions to archival. We believe that the
electronic tool sets are currently available
for others to manage information and complete

ECTD submissions, as well.
Let me take you on my travels
through these general observations. A couple
of footnotes along the way. Number one, I
recently learned in an important meeting that
adult audiences really won't listen to you
unless you give them about five why's for
everything you tell them. So you will see
"why" pop up on the screen here very often.
That's to convince you or at least open your
mind to my points. Some of my points will
be, `Oh yes, Al, we already know that.
Please go on.' Some of them will be rather
controversial and so that's kind of the span
of where we're going today.
Number one. We, meaning FDA and
industry, should be a partnership, must work
closely together and listen attentively to
each other's concerns. Why? Computers are
fussy beasts and the information must not be
changed when it's moved from one electronic
NEAL R. GROSS
environment to another. Two, second why,
again my former life as a reviewer.
Reviewers depend on this information. It's
their day-to-day feeding and care that
happens, to perform their job functions.
Also, the industry depends on this
information submitted to support and maintain
healthcare product approvals. Why again? If
electronic systems cannot talk to each other
successfully and share unchanged information,
the process can be compromised. I'm not
saying it's compromised every time, I'm
saying it can be.
Observation number two. We must
carefully select and closely supervise any
third party that may handle electronic
systems and any information stored, either by
FDA or by us in industry. This is contract
organizations that produce, transmit and
otherwise interface, touch e-submission
information. Why? The lifecycle of a
healthcare product, the development,
marketing of a prescription drug for

instance, and I know there are other products
at issue here, is a long-term process that
could last decades. In contrast to that why,
the lifecycle of electronic systems and
software is much shorter than a healthcare
product lifecycle. Only third parties with
extensive appreciation for and commitment to
managing the disconnect between this long
lifecycle that we have for our healthcare
products and the short cycle for systems and
software should be considered suitable
candidates for handling e-submission
information. For the good of patients that
we serve we need to be in this for the long
haul, folks. It's not just something you can
do on Sunday afternoon. Forget about it.
Again, still on observation two,
why? Disconnects resulting in information
migration and retrofits are costly endeavors.
Migration and data retrofits may be likely
outcomes when inexperienced, low-bid, third
NEAL R. GROSS
party contractors are used. Why? This is
not only a dollar cost, it is a time cost.
Migration and retrofits take time. Systems
are down. Delays disadvantage patients or
consumers, whether the delays originate in
FDA or from us in the industry.
Finally, confidentiality issues
may arise due to incidental or inappropriate
release of information. I would let you know
that most of these releases are not due to
the system, they're actually due to humans.
And so therefore staff at third party
organizations have to be of suitable quality
and they must be invested in the security of
the information that we entrust to them.
Very, very important.
Observation number three. We must
jointly learn how to handle and access
repetitive e-submission information.,
Investigator Forms 1572s, and updates more
efficiently and find ways to reduce that
workload. Now I'm speaking about
prescription drugs. That's been my life for

the last, well, let's say 20-plus years.
Let's not go there. But it's been a long
time. And we must find ways to more
efficiently handle that information. I would
say current industry submission systems are
clogged with this information and it easily
accounts for 20-50 percent of an average
month's submission activities.
The current trained labor force,
and I think this was in the Federal Register
as well, from e-submissions available within
the U.S. is not sufficient today to handle
the upward spiral of this workload if we add
more to it. Why? I think we must change and
we must turn our attention to and use our
limited resources to handle more difficult
and challenging repetitive submission types.
My example here is advertising and promotion
submissions. This will require at a minimum
the integration of three-dimensional scanning
and viewing. We all watch TV every day. We
NEAL R. GROSS
have camera phones. There are other forms of
video. I think that you know - one of the
folks in my office always asks me, `What
you're saying, can it pass the Washington
Post test?' And I guess I would challenge
all of us, whether we have FDA jobs or NIH
jobs, NCI jobs, industry jobs, if we said, we
do this number of submissions electronically,
but yet we almost have a whole group of
submissions over here, advertising and
promotion, other things, and we don't do
anything for those. We don't enable those in
any way. Would that fly? And I'll just
leave that kind of as an open question.
This brings me to a point I want
to put in capital letters obviously and that
is an e-submission paradigm should be an all-
inclusive one. eSubmissions should be the
Club Med. That's where we're going. We must
learn to use our systems, to use them
repetitively for all submission types, if we
are to maintain credibility with our external
audiences. Commissioners' offices, Congress,

consumers we serve, corporate entities that
we have to report into.
Why again? Repetitive use of the
same tool set for multiple submission types
will drive down the costs - cost is a key
factor in this meeting - and provide some
positive business cases and incentives for
all to participate.
Observation number four. We must
find a common ground to move forward with
making e-submissions a requirement and not an
option. We must provide incentives -
remember that Snickers bar, only now we're
going to go for something bigger - to move
from the current state of approximately 3-5
percent of all submissions sent to the FDA
electronically to a 95 percent level very
soon. 2009 is my example.
Here I'm going to depart from the
why's. I'm going to give you a few proposed
incentives. Incentive number one, reduce the
NEAL R. GROSS
PDUFA fee for those submissions requiring
fees by some percentage or amount when a
submission comes in in eCTD format.
Incentive number two, reduce the PDUFA review
clock by some number of days for applicable
submissions in eCTD format. Proposed
incentive number three, grant a tax credit to
those in industry, industry entities that
have ongoing e-submissions programs. These
incentives are my opinions based on 10 years
of working experience with e-submissions.
They do not represent my prior experience at
FDA, nor do they represent my current
thinking of my employer, just so you know
that. But I think we need to consider these
seriously and we need to be creative about
what it is that will move us from - I think
there are 200 people in this room. I think
the next time we meet we should have about
2,000 to hear this.
We must also find ways to enhance
and support FDA offices germane to e-
submissions. FDA Office of Business Process

Support and the FDA staff that currently
handle the electronic gateway. This has got
to be a requirement on the way to the 95
percent goal. There are great people out
there. They do tremendous jobs. We at TAP
would not be able to be where we are if it
wasn't for these offices. And I want to
commend people that put these folks in place
and the work they do every day. They have
clarified many things for us so that we can
move ahead and convert to all-eCTD.
Observation number five. Common
ground must include a master plan in which we
can all invest. This plan should include
what? Things like what are we going to do
about E2B, SPL and the acronyms can go on,
folks. I could spend another half hour on
acronyms if I could remember, okay? If I had
enough RAM up here to remember them all.
When? We need to get serious
about the order of adoption of new standards
NEAL R. GROSS
for e-submissions. The investment requires a
significant buy-in on the management chain on
both FDA's side and within industry.
(Sound system interruption)
DR. EDWARDS: Stay tuned for
further interstellar travel. We may be
leaving at any moment.
Why? Currently, electronic
systems deployed for the most part are highly
customized, configured to meet individual
needs and expectations. We can't continue to
afford this diversity. Taxpayers can't
afford it, we in industry can't afford it,
it's just too expensive. The electronic
silos that we've created by our current
approaches will soon be obsolete due to
evolving technology. There's no better time
to make a master plan than right now, today.
There is no better time.
Why? And you'll see, I sort of
got tired as I got to observation five
because there are fewer and fewer why's. The
time interval grew much shorter so I couldn't

get all my why's in here. Standards
development organizations are and will
continue to produce more and different types
of proposed standards for our consideration.
We need to select those standards which will
yield maximum benefit in an orderly manner.
Again, a master plan is not something that
can't change, it just needs to have an
orderly process we can all invest in.
Observation number six. The
master plan must include a schedule to
control lifecycle development process.
Before I put this next piece up, I am going
to say that SPL is a very good thing that
happened and I don't want what I've put up
here to be in any way negatively interpreted
about SPL itself. I'm only using that as an
example and as you can tell I now have some
nice background music to lead into this
controversial statement that I'm about to
make. Thank you for that intro.
NEAL R. GROSS
Why? One merely needs to look at
the impact of structured product labeling
around October of 2005 for a justification
for this. At least in our shop there were
two additional releases of software
corrections and changes that were made in
November and December. For those with
ongoing eCTD programs, this triggered
multiple change requests and re-validations.
This took some other people out of the work
queue in our shop to deal with this. These
were unscheduled.
Why? The inclusion of SPL was not
compliant with ICH eCTD requirements. Thus
the eCTD software providers had to go outside
the confines of ICH to produce a fix. So we
had to violate one standard in order to make
another one work. I think we can do better.
The disconnect between the SPL and the ICH
eCTD requirements to me would have become
evident had there been adequate pilot testing
and we could have considered some appropriate
alternative measures to move that forward.

Observation number seven. We must
include a Plan B, not to be confused with the
other Plan B, okay? This is a computer
process now, not another process that also
was Plan B, in our master plan. Why? We
must have an alternative in this master plan,
a Plan B when the electricity goes out or a
disaster strikes. I have with me in this bag
a CD-ROM. I also have my little thumb drive.
If the presentation wasn't present here,
maybe I could go to Dr. Levin's computer, we
could rig this up and I could give my
presentation on a - you know, continue.
Because I have backup. And so for any plan
we have, we've got to have backup. The best
example I can give here are prescription 15-
day safety reports that we need to keep
turning in, whether the lights are on in my
shop or whether they're on in Washington or
not. I tell this story often, that during,
not an eCTD NDA, but an eNDA, about five or
NEAL R. GROSS
six years ago a little squirrel gave up his
life and took us completely out of publishing
because he decided to jump across this
generator grid about two miles from our
facility in Lake Forest, Illinois, and that
took out the entire grid for about four
hours. So you need to have a Plan B. We all
need to have one.
Why? We must continue to perform
our jobs when electronic systems are not
available. When an outage occurs, we need to
go on, get things done. Why? It's just good
business to have a disaster recovery plan for
these critical electronic systems. Each
party, again, industry and FDA should know
when the other party is down. We shouldn't
be trying to send electronic systems in if
FDA's systems are down. If our systems are
down, we should be notifying FDA that they're
down and why they're down and how long
they'll be down.
Observation number eight. And now
you can tell I really got tired because there

are no why's to this one, okay? But we must
bring the correct focus and attention to the
required infrastructure surrounding e-
submissions. That's not only technology, but
it's process. We must find a way to get the
same level of attention and appreciation for
e-submissions as is currently accorded the
content of a new drug application. Again, my
former life as a reviewer, I know how
important NDAs are and there has to be a
match now between meta-data and how it gets
transmitted and the content. If you don't
get all the content because the meta-data is
screwed up, it doesn't matter and that
reviewer needs access to everything.
So far, Dr. Woodcock, you have not
cut me off. Okay. I'm going to quickly go
through the conclusions. We must quickly
work - I'm sorry, we must work together. I
must work quickly. Carefully select third
parties. We must learn how to efficiently
NEAL R. GROSS
handle and access repetitive information. We
must find common ground to move forward in
making e-submissions the requirement. The
common ground must include a master plan.
That master plan must include a controlled
lifecycle development process. The master
plan must include Plan B, and meta-data and
infrastructure must equal the content of e-
submissions in terms of importance.
To whom much is entrusted, much is
expected. I don't know who the author was,
but it's a great thing. Or, we must all hang
together lest we all be hung separately.
Benjamin Franklin said that. As far as I'm
considered it's truly time to put away our
perceived differences, move e-submissions
from a 3-5 percent level of participation to
the 95 percent level in the next two years.
I want to thank FDA, thank you Dr. Woodcock
for allowing me to speak. Thank you audience
for your kind attention.
DR. WOODCOCK: Thank you, and do
the panelists have any questions for Dr.

Edwards? Randy.
DR. LEVIN: Yes, I have two
questions. One regards the incentives. Is
the candy incentive out of the question?
(Laughter)
DR. EDWARDS: No, we just need to
talk to Mars, the candy company.
DR. LEVIN: My question is about
your comment about our labor force is not
sufficient. What kind of skills are not
sufficient? Can you expand on that a little
bit?
DR. EDWARDS: I think that - and
Randy, I'll just give you the experience in
my shop. People need to be able to pay
direct attention to a computer screen for 8-
plus hours. That's the first thing they need
to do. They need to be extraordinarily
vigilant in terms of what they see on that
screen to see if there are any errors or
omissions so that we can produce a viable,
NEAL R. GROSS
quality product that you or your reviewers
want to look at. Those people are not
necessarily PhD's. Some of them are high
school graduates. For every, let's say, four
people that I bring into my shop, usually two
of them survive because it is a unique skill
set of someone who will pay that much
attention to a computer continuously. I'll
admit I can't do it personally. I can't
spend eight hours of my day staring at a
computer screen, but that's what it takes to
put an eCTD together. And so we just need to
be conscious of the fact that as we move
forward, we don't have good parameters to
pre-test people to understand who can do a
good job and who can't.
DR. WOODCOCK: Ken?
DR. BUETOW: Implicit in your
presentation, which I should thank you is
very useful, is the need for this
infrastructure to be comprehensive, but also
the need to actually have a master plan for
its deployment, if that's a fair

interpretation. But I was curious. Can you
give us any insight - this is obviously huge
- where would be the sweet spots to hit some
of these repetitive tasks that you've
mentioned? What would be high-priority
activity?
DR. EDWARDS: High-priority
activities, I can't remember which slide it
is, but I think we have to go to advertising
and promotion submissions. I know that's
very controversial, I know there's a lot of
resistance there. Before I came to the FDA I
worked at MD Anderson Hospital in Houston on
the wards. The way the hospital
administrator let me on the wards to do drug
monitoring as a pharmacist because of course
they believe a pharmacist couldn't do it. He
said I'll give you the most difficult ward
first. You prove to me you can do it there,
you can go anywhere in the hospital after
that. So I guess, and it reflects a little
NEAL R. GROSS
bit of my personal opinion. I think we need
to do the difficult stuff first to prove to
people that we can move on.
DR. WOODCOCK: Any other questions
from the panel? Armando.
DR. OLIVA: How long did it take
your company to move to the all-eCTD
environment once you made that decision?
DR. EDWARDS: We carefully
monitored the eCTD both from the point of
view of guidance documents and available
software providers from July of 2000 through
July 2003. We made a commitment at that time
on paper to move. We produced our first test
disk for Ken Edmunds and all these other
wonderful people that work in business
process support. May of `04 we produced our
first NDA in eCTD format in `04. And my
boss, who's also a very big supporter of
this, came into my office and said so you've
done the NDA. So now show me you can do
everything. So that's what we had to do in
2005. So that's kind of our timeline. I

think there are probably other experiences in
this room that I'd refer you to.
DR. WOODCOCK: Other questions?
Thank you very much, Dr. Edwards. Our next
speakers are Dr. Thomas W. Littlejohn,
Medical Director of the Piedmont Medical
Group and Mr. Doug Pierce, President of
PeerMed, Inc.
DR. LITTLEJOHN: Thank you, Dr.
Woodcock. Good morning. Thank you for
allowing my colleague and I the time to
address this panel. My name is Tom
Littlejohn. I am the Medical Director of
Piedmont Medical Group which is an
organization of independent clinical research
sites in the two Carolinas and Tennessee. I
have been involved in clinical research for
20 years and have functioned as a principal
investigator on over 300 trials. My
colleague is Doug Pierce, President of
PeerMed, Inc., a healthcare-formed company
NEAL R. GROSS
located in Virginia and North Carolina.
We are here to address the issue
of electronic data capture, storage and
transfer as it applies to clinical research
sites. We applaud the efforts of those who
are working to convert the traditional paper
model into an electronic model. This
transition is picking up momentum and has
been clearly shown to reduce costs, improve
quality and accelerate the process. All of
the activity to date, however, assumes that
with the exception of large academic centers
with EMR, that the original data, the
wellspring, if you will, at clinical research
sites will be captured on paper and then
transferred to a case report form, whether it
be paper or electronic. These source
documents are sacred to the integrity of the
trial and must be accurate, accessible and
protected for many years. They must remain
under the control of the principal
investigator and free from manipulation or
tampering of any kind. They cannot be

reproduced. For this reason they must remain
at the investigative site and review of them
requires that the reviewer travel to the site
to verify the data and its integrity.
We are proposing a technology
whereby the initial data can be captured
electronically in the same manner as it is
currently on paper and that the data can be
validated, verified and protected more
effectively. Additionally, this digital
source data could be reviewed by monitors,
regulators or any other authorized party from
a remote location. Adoption of this
technology would close the final gap that
otherwise would continue to exist between a
completely electronic environment and a paper
one. I'll turn it over to Doug.
MR. PIERCE: I'm in Dr. Edwards'
presentation. Which I'm going to do it
again, just in case you didn't get it.
(Laughter)
NEAL R. GROSS
MR. PIERCE: Bear with me here one
second. While I find this, I'm going to let
you know that I did not attend the why
meeting and I have no why's. Thank you. Can
you all see? I can't see, I'm at the podium.
I didn't hit the why meeting. However, I was
at a meeting that said that when you start
these kind of addresses, you should blaspheme
famous authors. So for those of you who have
read Shakespeare's Julius Caesar, I'm sorry.
We have gathered here not to praise paper,
but to bury paper.
(Sound system interference)
(Laughter)
MR. PIERCE: But this isn't the
first time we've gathered together to bury
paper and paper is proving to be puzzling. I
want to talk about the puzzling persistence
of paper. And I think the photograph below
the title says a lot. And it says, it shows
what we do maybe every day. We sit at a
desk.
DR. LITTLEJOHN: We've got a

keyboard, a mouse and we shove them aside to
grab a legal pad and a pen. And you notice
that the person is starting a list. We have
three very different ways to input data. And
it seems to me that we have to examine the
power of paper and the paper interface and
understand it if we're going to in fact bury
it. So we'll be talking about the puzzling
persistence of paper today.
Benefits of electronic data
capture we all know very well, especially in
this room. We have rapid access to data. We
have higher quality of data. We have fewer
queries and faster resolution of the queries
that we do get. We have better tracking of
adverse events, less time to data log and of
course substantial cost savings. And
obviously those are the reasons why EDC is
gaining momentum in the market and that's a
wonderful thing.
However, there are barriers to
NEAL R. GROSS
adoption. The rate of adoption is not as
fast as we would like. It's not as fast as
many people thought. There are technological
constraints that are holding us back. For
example, if it's a web-based EDC we have
internet access or do we not have internet
access. Do we have broadband? Is it dial-
up? Is it every other day? You know, that
sort of things. We have connectivity issues
within a research site. There are hardware
and software deployment issues involved. All
those things limit and slow the growth of
EDC.
There's also personnel constraints
and I almost said behavioral constraints
because I think that's probably a better
word. We have attitudes toward work and work
flow and what traditional responsibilities
have been and which will change when EDC
systems are brought onboard. There's a
certain level of technological
sophistication, of savvy, of comfort that EDC
systems seem to demand. But the interesting

question to us is even when the technological
constraints and the personnel constraints
have been met, in the very best setting why
is it the paper source docs are still being
used? Why is it that even with some of the
best EDC systems you can get the coordinator
initially captures the data on paper? Well,
to understand that we have to understand the
power of pen and paper. If it was a computer
system, paper and pen, we would say the
modern paper and pen interface is the result
of 5,000 years open source global
collaboration. It's the most well developed,
shared, open source way to gather data ever.
The early data wasn't good. You've seen it
on the cave walls, but we quickly moved from
the rock and the chisels to paper and we've
been developing it forever. And almost every
human being on the planet knows what to do
when you hand them paper and a pen or pencil.
So it's been around a long time and it's part
NEAL R. GROSS
of us. It's the original
graphical user interface. It's flexible. If
you forget something on your list, you can
add it. I could have added "wise" to my
thing here, but I can't now. It's familiar.
It's friendly. No one's intimidated. When
you hand a legal pad to somebody, they don't
go oh, I don't do legal pads.
You don't have somebody looking
around trying to find their legal pad like I
was doing. So it's very friendly, familiar
and it works very, very well. Now, let's
take some of these ideas and look at paper
source documents. Paper source documents
allow for structured and free-form entry of
data. In other words, the paper source docs
that you've made following the protocol have
blanks and check boxes for structured data,
data that you know the protocol asks for.
But the beauty of paper is if you need to add
something on the fly, you just write a
marginal note. You circle. You sign. You
can add data on the fly. They're simple.

They require minimal training. Data capture
on paper source docs is a very unintrusive
thing. Coordinator and patient are there in
a room. The coordinator is taking data down.
No one has to turn around and type in data.
There's no machines to wheel in, et cetera.
And the most important reason we think paper
source documents have been so hard to kill is
that there's just a lot of very important
data, important for the case history,
important for the trial that is not a data
point to be entered into the data management.
It's not even and it shouldn't be on the
electronic case report form, but you've got
to have it. And when we've talked to
coordinators and monitors, they insist on
seeing it. So there is lots of data, what we
call data, and by "data" it may be marginal
notes that have to be captured, but that
aren't passed on downstream that force people
to use paper source docs which keeps us sort
NEAL R. GROSS
of you know tied to paper.
So what can be done about that?
Nothing. No, here we go. We think one
solution is to recreate paper source
documents keeping the 5,000-year-old
interface, the 5,000-year-old data entry
methods that we know and love, but change the
paper. It's not the interface, it's not the
way we enter data that's the problem, it's
paper. So what we propose doing is replacing
paper with a tablet PC which of course you
are very familiar with. Recreating the
source documents that look and feel just like
paper source documents that contain
structured data, check boxes, pulldown lists,
et cetera, but that also allow you to write
marginal notes, to ink comments about blood
pressures, et cetera. The data itself is
then - the data points extracted from the
forms and sent through a secure web portal
straight into the ECRF that the sponsor is
using.
In addition to the data being

passed through, images of the form itself in
PDF version are also captured and sent to the
secure web portal. These are the actual
forms. These are the actual forms with the
checks and the marginal notes, the forms that
have been assigned, and they reside there on
the website where they can be reviewed from
anywhere in the world by the sponsor's
monitor, by regulators from the FDA, anyone
who has authorized access can look at the
source documents, the actual ones. If they
have any questions about data downstream they
can go back up to the source, the wellspring
of the data and look at what was done. In
addition, if it's an adapted trial, for
example, and the source documentation needs
to be changed, it can be changed in one place
and pushed down to the local devices deployed
in the field.
When the study is over, research
sites have the problem of keeping up with
NEAL R. GROSS
their paper source docs. There are, as you
well know, warehouses scattered around filled
with paper source docs that supposedly are
kept in a, you know, secure way. One good
storm does those in. So since the source
documentation are digital they can be moved
to a long-term storage where again the forms
and the audit trail can be reviewed at any
later date. The benefits of this sort of
digital source documents are data is entered
only once. Now, most of the time, even when
EDC is in place it's actually entered twice.
It's written on paper first and then it's
translated into the ECRF.
Point of entry protocol-specific
data validation. Before the patient leaves,
data is validated. If there's a problem, you
can correct it there on the spot. Simple,
intuitive, familiar interface. Unobtrusive
data capture. As you all know, tablets are
very unobtrusive. You can work on the fly.
You can set them down. You can take a vital.
You can pick it up. Nothing gets in the way

of the coordinator-patient interaction. A
secure audit trail is of course provided and
that audit trail can be viewed from anywhere.
And you can see exactly what was changed,
when it was changed, who changed it and why.
We believe a system like this
would produce significant barriers to
fraudulent data entry. All the research
sites nationwide globally would all be
looking at the exact same source docs and you
can trace not just the data that's passed
downstream but what they actually did and who
did it. Who signed what. You can see where
they signed it. It's mobile. It works in a
disconnected state. That means if you're not
connected to the internet, you're walking
around your office, you take your data. When
you get back to your internet connection you
sync up and the data is automatically loaded
up to the website.
If data standards are adopted
NEAL R. GROSS
freely, it plugs into any ED system. This
system is only concerned about capturing
initial data and then passing it on. This
would allow monitors, data managers,
regulators to verify source documents from
wherever they are. The savings to the
sponsor would be huge. It would solve the
sort of dilemma of things like fixed unit
pricing versus 100 percent source data
verification. How can we verify enough data
and yet save money and meet the budget? So
we think that a system such as this that
focuses on just the initial capture of data
and that does it in a way that mirrors this
old, old paper interface that we're used to
using, but does it in a way that's modern and
that allows us to verify data and securely
transmit data would go a long way toward
finally burying paper.
Now, the FDA's role. Clearly, a
system such as this and any EDC system
requires clear, consistent guidelines for
data validation. We all have to be

validating data in the same way. Things like
the eSDI initiative are crucial. We have to
have compliance market-wide. Right now as
you all know of course there's islands of EDC
systems. Some can communicate with each
other. Some can communicate with EHRs and
EMRs, others can't. There's got to be from
the top down a consistent set of data
transmission standards and we all have to use
them.
We believe that a system such as
this supported by well established shared
data transmission standards would go a long
way toward helping investigators at the site
level. It would make it easier for a new
investigator to enter this line of work. It
would be clearer. What was called upon for
him or her would be clearer. It would also
make post-market surveillance easier, safer
and more affordable. So for all those
reasons we would like to see us move in that
NEAL R. GROSS
direction and really address the power of
paper and then bury it. I have time left.
Questions?
DR. WOODCOCK: Thank you. Are
there questions from the panel? Armando.
DR. OLIVA: Can you be more
specific? Exactly what exchange standards
are missing or are needed in this area?
DR. LITTLEJOHN: Well, I think the
standards that the CDISC group is working on
now where data is gathered in the same format
and then can be transmitted into whatever
back end system is being used, XML standards.
You know, it doesn't really matter on what
the standard is as long as we're using the
same one and it's an appropriate data format
to transmit via the net and share. So I
think the work's being done now, I just think
we need to keep pushing it. We need to make
sure - and at some point there's got to be
teeth. I mean someone has to say this is
what we are doing. This is the standard that
we are going to use. Otherwise I think we're

going to go a long time in having partially
compatible data standards in data
transmission standards.
DR. WOODCOCK: Ken?
DR. BUETOW: Yes. The
longstanding debate of whether one should
have structured data at point-of-collection
or later in translation after collecting
unstructured data. Do you want to come down
on that, on what side of the argument you
come down on on that?
DR. LITTLEJOHN: Well, both. I
hate to straddle the fence here. I think the
beauty of digital source docs is that you can
in fact have both. You should have both.
There's a lot of data that the protocol calls
for that is structured data. Data has to be
captured and captured in a certain way. At
the same time there's got to be room to enter
free-form data. I was talking to a monitor
last week and a very simple but clear
NEAL R. GROSS
example. She was talking about blood
pressure. And we take a patient's blood
pressure, it's really, really high. Well,
what you always do is you just wait. Maybe
they just ran in, maybe they're late, maybe
they're anxious, six cups of coffee. You
just wait and you take it again. Well, if
there's not room to write a note, literally a
note saying, you know, I gave her five more
minutes, she seemed anxious or nervous. I
took it again and this is what I got. Then
you're left, what you do is you grab paper.
I mean, you aren't going to not record that.
Open up a chart, there's sticky notes
everywhere. And as long as we have paper and
sticky notes we're still chained to paper
source docs and this trail. So I think it
has to have both. I think the only solution
is we have a system that can accommodate
both.
DR. WOODCOCK: Randy.
DR. LUTTER: Can you elaborate a
little bit more on what you mean by

validation guidelines?
DR. LITTLEJOHN: Absolutely.
MR. PIERCE: Can I address that?
What needs to happen for something like this
to be adopted by industry. For this to be
adopted by industry they need to know that
the device, the technology, the software that
we're developing is CFR 21 Part 11 compliant
and that the FDA feels confident that these
devices if constructed in the proper way
would be acceptable, would pass an audit, for
example. Some sort of collaboration,
cooperation from the agency to look at these
type devices as they're being developed to
reassure the potential customers which would
be any particular sponsor would want to adopt
the technology. So we would make a plea that
this sort of sharing of information and you
know working together would help move this
along greatly.
DR. WOODCOCK: Is there an issue
NEAL R. GROSS
with the current definition of "source
documentation" that exists within the FDA?
MR. PIERCE: I don't think there's
- a source documentation is pretty clear.
It's the initial capture of any information,
whether it's an EKG, a blood pressure, you
know and wherever it exists is the original
source. So I don't think there's a problem.
Something that happens with electronic
medical records, for example, because it
doesn't lend itself easily to a clinical
trial protocol, it's not uncommon for data to
be written down on a piece of paper and then
entered into the electronic health record.
Well, what was on the paper is the actual
source document according to your definition.
So I think we eliminate that problem with
this interface.
DR. WOODCOCK: So what you're
saying is that the Part 11 issue is there
needs to be clarification there, but
conceptually there's not a problem with the
definition of source documentation, it's more

the compliance, Part 11 compliance.
MR. PIERCE: Right. We would just
like to see the guidelines to be a little
more clear, concise, unambiguous and that
there not be significant cost constraints to
adopting them.
DR. WOODCOCK: Other questions
from the panel? Ken.
DR. BUETOW: While we're on Part
11.
MR. PIERCE: Yes.
DR. BUETOW: So, would you
actually argue partially in answer to the
question right before Dr. Woodcock's, you're
arguing that we need additional specification
a la Part 11 for the specific systems, for
their definitions and validation over and
above what's already in Part 11?
MR. PIERCE: Well, I would submit
that the guidelines in their present form are
rather vague and rather ambiguous and don't
NEAL R. GROSS
give I think industry comfort that spending a
lot of money developing a system like this
without knowing ahead of time that it would
be acceptable is a problem. So there aren't
- it's not that the guidelines are wrong, we
just, we need more clear and concise
guidelines I think. More in depth.
from the panel? Thank you very much.
MR. PIERCE: Thank you.
DR. WOODCOCK: Our next speaker
will be Mr. Ron Celeste from ThinSpring.
MR. CELESTE: Good morning and
thank you very much. I'd like to thank the
FDA very much for the opportunity to present
this morning. I am going to, having only
really 15 minutes and five minutes for
questions, I'm going to go through these
slides some of them very rapidly. What I'd
like to do is start with a historical
perspective of these submissions and get into
what the current industry standards landscape
looks like, then talk primarily at ThinSpring

servicing over 60 medical products
manufacturers. Primarily going to address
the issues that our clients see in adopting
electronic submission technologies across the
board. Talk about submission lifecycle
management, the concept of a third party
facilitation for these submissions and then
again highlight some specific challenges we
see in industry.
You all remember computer-aided
new drug application guidelines in the `80s.
When I was at Baxter Healthcare the
technology we had at that point in time
really was not very conducive, was not
conducive at all to real-time discussions
between the reviewer and the applicant or
sponsor manufacturer. They did enable
asynchronous communications where files could
be sent back and forth and questions be
answered and then files updated and sent back
using graphics programs like Harvard
NEAL R. GROSS
Graphics, the old blue screen WordPerfect
word processor and databases on the desktop,
286 processing power desktops. We didn't
think we'd need anything more. DB3+. Those
tools were not integrated.
We didn't get integrated desktops
until the `90s with things like Microsoft
Office. The portable document format, file
format from Adobe didn't come out until 1994.
Then Al Gore invented the internet and
everything changed. We really view
extensible markup language. My second key
point is that extensible markup language as a
file format is the key technology that can
enable efficient and effective electronic
submissions in any e-submission data
platform.
We've seen an explosion - this is
my alphabet soup slide - an explosion of
initiatives since the internet came to be
with the browsers and the desktop
capabilities that are available through a
browser. I don't have time to go through all

of these acronyms. It's full of TLAs and
FLAs, 3-letter acronyms and 4-letter
acronyms. It would take me all 15 minutes to
get through all these, but key point at the
end of this. Moving into the future you see
RPS. RPS stands for regulatory product
submissions and RPS is under the auspices of
- in a standard controlled by HL7. We
believe that in all XML ECTD will be an
efficient file format to deliver effective
electronic submissions throughout industry.
Key challenge for industry. These
purple 3-letter acronyms and 4-letter
acronyms on the screen are existing
enterprise systems that commercial
manufacturers have in-house today.
Enterprise resource planning systems,
electronic document management systems,
enterprise content management systems. The
key is that their electronic submissions
tools and the tools that they use to
NEAL R. GROSS
communicate with the FDA need to integrate
tightly with existing legacy information
systems and solutions that are in-house
already at these manufacturers.
There is a roadmap that we've been
following. We've talked about guidelines
that have been published and mandates that
have come out of the FDA starting with the
electronic labeling rule in 2004. This is a
timeline that we're following at ThinSpring
in supporting our clients moving forward. We
do see ECTDs eventually being mandated, a
potential proposed rule coming out of this
meeting, becoming a final rule sometime
possibly in 2007, late 2007. We also see
RPS, the regulatory product submission,
regulated product submission standard being
adopted industry-wide. One of the challenges
we'll get to later though is harmonization of
standards between the U.S., between the E.U.,
between Canada. XML is a data file format
that we believe is going to enable effective
e-submissions. However, there does need to

be an agreement on standards and there does
need to be more harmonization than we've seen
today between ICH, HL7, CDISC and a variety
of other standards bodies out there.
Key challenge for the
manufacturers, the sponsor applicants is that
not all of them, especially the smaller and
mid-sized manufacturing organizations have
today the ability to manage XML data-centric
and document-centric information. So an XML
infrastructure in-house that integrates with
their existing enterprise solutions is a
challenge that a number of the manufacturers
are facing today. XML is a great technology
for improving data portability and data
archiving. Dr. Edwards mentioned we need a
technology that has a long enough lifecycle
to at least match the lifecycle of a medical
product. XML technology and its ability for
archiving, I can open 20 years from now an
XML file with a simple text editor. We have
NEAL R. GROSS
one client who is literally storing a deck
vax with a clinical data management system
from 30 years ago in Iron Mountain so that
they can get access to that data when they
need it. With an XML file, you can open that
file 20 years from now with a simple text
editor. So from an archiving standpoint,
again, we firmly believe XML is the -
extensible markup language is the technology,
the file format technology that enables an
all-electronic e-submission platform.
Hear a lot about what extensible
markup language is. I wanted to talk a
little bit about what XML is not. XML is not
a single software package that you can buy
and install and start using like Microsoft
Word. It's a structured content markup
language that facilitates data exchange.
It's not a single electronic submission
standard. There are multiple standards.
There are XML standards for clinical data
that are managed through CDISC. HL7 has the
RCRIM structure for defining medical product

data. ICH has a number of standards. The
electronic common technical document today,
the backbone for the ECTD is XML, defined in
extensible markup language. A key point for
the manufacturers and the people who are
hands-on as Dr. Edwards mentioned, looking at
that screen for eight hours a day is XML is
not a single file. XML is not a Microsoft
Word document, it is not a .pdf file, and
today again most manufacturing organizations
have document management systems that are
geared towards managing the lifecycle of a
single flat file. XML needs to - you need to
have configuration management version status
accounting tools that enable you to manage
the structure, the content and the style or
formatting of your submissions. Another key
point is XML content does not replace printed
literature and that is an interesting
scenario when you start talking about
advertising and marketing materials and how
NEAL R. GROSS
content that is approved in extensible markup
language file formats translates into printed
material through different formatting and
style sheets.
This is a little difficult to see,
but this is making the point again that XML
is not a single file format. This is an
example of a structured product label file
folder which contains content files,
structural files and presentation or style
sheet files. So again the industry challenge
is for the manufacturers is that we need to
have tools that allow us the configuration
management control for structure versus
content versus style. We can have that same
structure - I'm sorry, the same content that
gets approved again using the SPL example,
the boxed warning structured product content
gets approved by the FDA and posted on the
National Library of Medicine. We can have
multiple formats or style sheets for
presenting that approved content back at the
sponsor manufacturer's location. They may

have a PDF style sheet to publish that
information on their website. They send it
to perhaps an encapsulated postscript file to
print that boxed warning on a physical
container label when they do a product run on
the shop floor. So the key is we need to be
able to maintain configuration management
status accounting version control of the
content, the structure and the presentation
of our XML-based submissions.
Full lifecycle management. Dr.
Edwards also mentioned that in their internal
system at TAP they're looking towards
improving the management of the meta data.
Meta data is data about data. The meta data
and the content contained in their
submissions throughout the entire product
lifecycle. Again using the structured
product labeling example, most manufacturers
are through the path of - the challenge of
getting existing Word documents, existing PDF
NEAL R. GROSS
converted into XML and now are looking
towards full lifecycle control. What do we
do after the submission is approved?
The third party electronic
submission facilitation, an important slide
here. This is just another version of a
slide I presented back in 2004 at the
Structured Product Labeling Challenge that
the FDA published. The technology exists
today now much different from the early `80s
and the mid-`80s to allow both real-time
and/or asynchronous communication between the
sponsor applicant and the FDA. Web-based
technologies and XML data file formats are
those enabling technologies.
Industry adoption challenges. A
key is minimizing system design complexity.
We've heard earlier and we've had a bit of
conversation about validation and having to
re-validate every time there's a change in
the tool set. Minimizing system design
complexity minimizes the validation
challenge. And I have a final slide to

discuss that in more detail. Component-based
content creation training and education.
Again, remember XML is segregating structure
from content from format. That is a
different paradigm for the users at the
sponsors, the authors of these submissions at
the manufacturing sponsor organizations to
get used to. So there is a good deal of
education and training required.
Intellectual property protection
is a big concern of the manufacturers,
especially when we're dealing with a new drug
application and proprietary drug technology.
Technology such as SAFE, Signatures and
Authentication for Everyone. It's a
consortium that was formed to address
electronic signatures and authentication
within industry can be applied to help
address this concern. Regulatory agency
standards harmonization I mentioned earlier.
Very important that we see a higher level of
NEAL R. GROSS
harmonization between ICH, HL7 and CDISC.
And then communication, communication,
communication. That's what we're doing here
today. I think this is an outstanding forum
and I think we need to be doing more of this.
My final point on system
complexity. Here's an equation we've been
using for a number of years but you do need
to see that complexity and the validation
challenge of any system design increases
significantly and exponentially as the total
number of software packages and technologies
used to create the solution are employed.
And that's all I had. We have about two
minutes for questions.
DR. WOODCOCK: Thank you very
much. Are there questions from the panel?
Anyone? All right.
MR. CELESTE: Thank you for your
time.
DR. WOODCOCK: We will now take a
break. We will take a break until 10:35 by
that clock right there and also by my watch.

So you have a few minutes, but we will resume
promptly at 10:35. Thank you.
(Whereupon, the foregoing matter
went off the record at 10:17 a.m. and went
back on the record at 10:35 a.m.)
DR. WOODCOCK: Joining us this
morning on the panel sitting in for Dr.
Skirboll will be Dr. Barbara Mittleman also
from NIH. Our next speakers are Dr. David
Hardison who's Vice President of Life
Sciences at SAIC and Wayne Kubick who is
Senior Vice President, Lincoln Technologies.
Ready to go.
DR. HARDISON: Good morning. I'd
like to thank Dr. Woodcock and the panel for
this opportunity to speak today. Wayne
Kubick will be joining me for the Q&A after I
give my remarks.
Clinical Data Interchange
Standards Consortium, better known as CDISC,
wishes to commend FDA for the regulations
NEAL R. GROSS
guidance documents and specifications and
their overall support for standards as part
of the efforts to move towards electronic
regulatory submissions. CDISC heartily
endorses the FDA proposal to move to an all-
electronic submission environment for
regulatory information and the creation of an
electronic platform for enhanced information
management across departments and divisions
of the agency.
CDISC is a global, open,
multidisciplinary non-profit organization
that has developed standards to support the
acquisition, exchange, submission and archive
of clinical trial data and meta data. CDISC
is the leader in the development of data
standards for clinical research. CDISC
participants and stakeholders include more
than 175 organizations representing academia,
pharmaceutical companies, technology and
service providers, institutional review
boards and others interested in streamlining
biopharmaceutical product development and

improving clinical data quality in
healthcare. CDISC also has joint memberships
with HL7, PIMS, EMEA and CPATH Institute.
Please check out CDISC.org for additional
information.
The mission of CDISC is to develop
and support global platform independent data
standards that enable information system
interoperability to improve medical research
and related areas of healthcare. The CDISC
standards are in different stages of their
lifecycle. Several of the standards are
written as specifications in FDA guidance and
we continue to work towards finalizing and
improving these standards with your help.
CDISC depends heavily on its volunteers.
Data flow using CDISC standards
including its link with healthcare. After
the presentation of Dr. Littlejohn and Mr.
Pierce today we'll have to modify this to
make sure we better illustrate capture at the
NEAL R. GROSS
point of care in our diagram which is where
everything begins from the data standards
world. CDISC supports the belief that
submissions in a standard electronic format
will facilitate regulatory review processes
at FDA and enable the use of common review
tools that improve reviews. I underscore we
support the belief that these standards will
improve regulatory review. Because of the
lack of a robust infrastructure in which our
sponsor companies can support their
electronic submissions, especially with
respect to the CDISC standards. It makes it
very difficult to have a test bid - I knew
the gremlins were going to come in at some
point in time. It makes it very difficult to
test and ensure that an implementation of the
standards can actually be received at FDA.
So therefore having an infrastructure in
place will accelerate and improve the
standards that we are working on so hard
right now. In addition, standards for
electronic submissions enable data

aggregation and the population of cross-study
and cross-product databases that will vastly
enhance the FDA's ability to perform safety
assessments, identify trends and conduct
product evaluations. Such knowledge
repositories would not be feasible without
the submission of logically and semantically
consistent data and information by
manufacturers in a standard electronic
format.
The value of standards accrues to
manufacturers as well. When applied from the
startup stage of a clinical study or program,
standards have been shown to improve data
quality and substantially reduce cost and
time in the product development process for
sponsors and other stakeholders in the
industry. We will speak to these specifics
later. Most importantly, CDISC believes the
transition from paper to standard electronic
submissions will lead to safer and more
NEAL R. GROSS
effective drug products to improve public
health.
One of the impediments to
transitioning to an electronic submission
environment is the requirement that this data
or information must be in a standard format.
Fortunately industry has been actively
involved with the agency over the past decade
to help develop and implement industry
standards for the purpose of supporting
electronic submissions. CDISC is the leader
in the development of data standards related
to clinical research. Our members have
contributed collaboratively to the consensus-
based CDISC standards development process.
Most stakeholders in the industry are either
in the process of implementing or making
plans to implement these standards. Broad-
based technical knowledge and experience in
the application of these standards are needed
among all stakeholders, including industry
and FDA personnel. CDISC is proud of the
fact that we're working with FDA to provide

adequate training in these standards in the
coming year. Fortunately, both XML and
expertise and knowledge of the CDISC
standards are increasingly prominent within
the workforce. Education and training
courses are readily available at a reasonable
cost.
While the implementation of
standards-based electronic submissions
requires an investment of time and money, it
also reduces operating costs and time-to-
market once the standards are in place.
CDISC recently participated and partnered
with the Gartner Group with support from
PhRMA to develop a business case analysis
that estimates the costs and benefits of
standards implementation. While these costs
are very difficult to estimate particularly
because of the absence of baseline
information, the business case metrics
indicate that there are substantial
NEAL R. GROSS
reductions in cost and time related to
capturing, cleaning, analyzing and reporting
clinical trial data, especially when
standards are used at the startup stage of a
clinical trial or program. Similarly,
regulatory reviewers will be able to spend
less time on data manipulation and more time
on the science when data are submitted in a
standard format. Communication will also
improve between the agency and manufacturers
such as reducing time for follow-up queries
once a submission occurs.
Over 50 percent of clinical trials
involve a contract research organization,
CRO, and an increasing number of trials
involve electronic data capture vendors.
Communications and data exchange among
sponsors, CROs, EDC vendors and even project
team members become much more cost-effective
and easier when standards and electronic
processes are employed. This is especially
true as we've seen in earlier presentations
if we can harmonize standards across clinical

research and with healthcare so that data are
entered only once at the site. This will
help relieve a lot of the burden of clinical
investigators and create more incentives for
their participation in clinical research.
Therefore, CDISC heartily supports the 2-year
industry transition time that is stated in
the proposed rule for electronic regulatory
submissions.
Making the transition from paper
to electronic submissions is hard work.
CDISC would like to offer several
recommendations to ease the transition. FDA
must take the strong position to drive
manufacturers to make all electronic
submissions in XML containing data in
specified standard formats. The majority of
industry organizations is prepared or is in
the process of preparing to utilize CDISC
standards for submission of clinical data for
regulated biopharmaceutical products. This
NEAL R. GROSS
includes the CDISC study data tabulation
model, the operational data model and
define.xml. SDTM can be submitted to FDA in
the ECTD or the HL7 regulated product
submission format.
We would like to see the FDA adopt
the ODM for electronic CRF data submissions
in addition to using it as a transport format
for define.xml meta data and SDTM and
analysis data. This transport standard is
already familiar to most industry
stakeholders. It can also facilitate auditor
reviews of electronic data capture
environments at investigative sites.
CDISC is driven by a mission that
supports electronic submissions and data
exchange that will enable translational
research. To that end CDISC is working with
industry and FDA on standards to support
electronic submissions for devices, genomics
data and animal data. CDISC has already
published the SEND standard, the Standards
Exchange of Nonclinical Data. Both SEND and

STTM are cited in study data specifications
for the final ECTD guidance.
Many of you - translational
research is more rapidly bringing innovations
from bench to bedside. For over five years
CDISC has formal relationship with HL7 to
enable a link between clinical research and
healthcare through standards. We strongly
encourage the FDA to continue its support of
these standards development efforts which are
critical success factors for translational
research and electronic submissions and data
exchange.
Variation in terminology is also a
barrier to effective collaboration, data
aggregation and multidisciplinary research.
CDISC collaborates with the National Cancer
Institute in a number of ways, including the
development of standards relevant for
oncology. CDISC through NIH roadmap grants
in collaboration with Duke Clinical Research
NEAL R. GROSS
Institute has work under way to develop
therapeutic area terminology standards with
the initial focus on cardiovascular and
infectious diseases. FDA should continue its
support of terminology standards.
CDISC is leading a collaborative
group devoted to achieving Critical Path
Opportunity Number 45 to standardize case
report form data collection consistent with
the STTM format. FDA should provide
additional support to complete this work
which will provide tremendous value to
investigators, clinical research associates
and monitors, project leaders and others who
work at the front end of the clinical trial
process. As mentioned earlier, these
benefits will cascade to provide significant
downstream benefit for electronic
submissions.
It goes without saying that the
infrastructure to receive standardized
electronic submissions is critical. So are
standards. However, the lack of an

infrastructure, its absence, is a barrier to
standards development. We need a test bed.
We need to be able to demonstrate that these
standards, once developed and implemented in
our sponsor companies, in our member
companies, will actually be acceptable to
FDA. Therefore CDISC supports clinical
research information exchange as it's
currently being proposed.
Additional standards. Before
specifying additional standards, CDISC
believes it is important to first apply the
existing standards effectively and to work to
harmonize standards to support semantic
interoperability within clinical research and
between research and healthcare. Significant
progress has been made in this area with the
Biomedical Research Integrated Domain Group,
or the BRIDG modeling efforts. This was
initiated by CDISC. It's now an open source
model governed by FDA, CDISC, NCI and HL7.
NEAL R. GROSS
Further harmonization support is
needed in specific areas, including the
following. To harmonize the multiple
standards related to adverse event reporting.
At last count there were six. Clearly a
requirement of biopharmaceutical companies to
report globally, having to do that in
multiple formats is a problem. Doing
government-sponsored research and have to
provide adverse event data in one form, why
does it have to be different than industry?
And I could go on and on. This is a
significant area of concern given the
importance of drug safety and surveillance.
We also need to complete - ensure
compatibility with implementations of the
CDISC HL7 protocol representation standard,
including trial design. Further, we need to
harmonize the protocol representation
standard and SDTM with the needs of other
regulatory agencies and global organizations.
For example, EMEA's EudraCT and the WHO's
International Clinical Trial Registry

Platform and various results reporting
projects, all of these need to be harmonized
so that we can adopt the standards to be more
robust to meet the multiple needs of these
different stakeholders.
Global pharmaceutical companies
will appreciate the efficiencies they will
gain if there is harmonization of
requirements and a single consistent set of
standards used across multiple groups such as
FDA and I might add within FDA. It's
important to - if there were clarity and
consistency within the agency and I know that
is a priority. Also with EMEA, WHO and NIH.
These groups are requesting trial registry
information and results reporting subsets for
multiple purposes. If these data formats are
designed to support semantic interoperability
there will be cost and time savings for all
stakeholders globally. This is very
important. We ask that FDA recognize and
NEAL R. GROSS
support CDISC's continued leadership of these
standards enhancements and harmonization
efforts.
CDISC supports the use of a third
party entity, especially if that accelerates
the decision by FDA to require electronic
submissions in a standard format. We believe
that the benefits to patient care and all
stakeholders far exceed the initial cost.
CDISC asks that FDA recognize and support
CDISC's continued leadership in these areas
for the enhancement and harmonization of
standards for clinical research. Thank you
very much for this opportunity to add our
comments today on behalf of CDISC and its
board and all of its members. We're very
grateful and now we'll take questions, and
Wayne if you'll join me.
Randy.
DR. LEVIN: First, I'd like to
thank the CDISC organization for their

efforts on moving the data standards forward.
They've done a lot of good work and I
appreciate all the work that you've been
doing on this. One of the things that you
noted on your presentation were that some
companies are being proactive but others are
waiting for a mandate. Can you elaborate on
that, why they're doing that and what are the
issues?
DR. HARDISON: I'll make a general
comment and, Wayne, you can add some
specifics. Implementing standards requires
an investment. And I think that industry and
especially this industry who's risk averse in
its nature are waiting for an indication
that, (a) this is an absolute requirement,
and (b) the agency is actually ready to
receive electronic submissions. So that's
the general comment. Wayne?
MR. KUBICK: I think there are a
couple of other points as well that are
NEAL R. GROSS
relevant. One is companies are concerned
that maybe the standards environment isn't
terribly stable yet, that things will be
changing, they'll be making investments and
then they'll be forced to change again. And
so we need to have a uniform message that
we're going to hold to these standards,
they're going to be backward compatible and
that work you put in now won't be lost in the
future.
I think the other issue which is
pretty important is that it's a very complex
standards environment. A lot of people are
looking out there and concerned that there's
competition between some of the standards,
the HL7 versus the CDISC standards for
example, and we're working very closely to
keep those harmonized. We're working very
tightly within the RCRIM environment. But
the perception out there is not quite as
clear. And so there are concerns that again
something else will come along right after
I've gone, put the effort into implementing

what's there now. And that's one of the
messages we need to get across is a uniform
stable standard that we're all supporting,
we're all working on the same direction.
DR. LUTTER: Thank you. Could I
just follow up on that mandate question that
the other Randy asked? And that is do you
see that as necessary or would alternatives
such as incentives, think reductions in user
fees or maybe changes in performance goals or
options like that might also be an
appropriate substitute for mandates? That's
the first question.
The second one is in that same
slide on key messages for management from the
PhRMA-Gartner-CDISC project, could you
elaborate more on the nature of the cost
savings? In that slide you had an average
per study cost savings of $9 million. What
type of study is one where one might expect
savings of that nature? Thank you.
NEAL R. GROSS
DR. HARDISON: Okay. I'll try to
address the first question around incentives
versus mandate. Basically I think that a
number of companies who see the benefits of
implementing the standards upstream in the
whole lifecycle of clinical research at the
beginning around - even with the protocol and
especially when they're designing their data
capture approach, the case report form
annotation, data management plans,
statistical analysis plans. Companies just
see the benefit of doing that according to
consistent standards globally. And then they
say well which standards? Well, there's the
CDISC standards. It just makes good business
sense. So a number of companies no matter
whether FDA ever requires or mandates
electronic submissions in the standard
format, you just see the business benefit of
moving to the standards. So a lot of
companies do not need a mandate for that.
Then there's whatever proportion,
I can't give you an exact calculation of

this. A number of companies are - like to
sit on the fence. They're aware of the
standards activity, but they're waiting for a
clear signal from FDA that there will be
benefit from them making investments in
implementing these standards. So what
signals do they have that this is going to be
something that is worthwhile? One, clearly
having reference to the standards and
specifications in guidance documents is a
good step.
Mandate versus incentives, I can't
answer that since I'm not a sponsor company,
but when I used to work for a sponsor company
clearly incentives are good because the cost
of drug-to-development is so high. So
therefore I would think that it would be a
driver. But there's still nothing short of
being able to point to a rule that says, you
know, you must do this if you want to get
your new drug application approved. The
NEAL R. GROSS
strength behind that I think outweighs
whatever you might be able to do with
incentives.
Now, the other question, second
question. Do you have anything to add to
that? Okay. Second question, around the
Gartner-PhRMA-CDISC business case. The
savings - problem with doing any kind of
business case industry level and also being
trained as a statistician early in my career,
you have problems with averages. The
average, that number, the $9 million savings,
was based on average IND clinical trials as
reported by - I think we used CMR data as one
of the primary sources for this. Then we
also used Tufts data who estimate that the
average - once again, average cost of drug
development is I think $37,000 a day. So
when you look at the - and the folks that
participate in the business case study, major
biopharmaceutical companies who have adopted
CDISC standards. We were able to extrapolate
the savings that they accrued down to the

average investment required for clinical
trials to come up with that $9 million on
average savings. So typically these would be
the large multi-center clinical trials,
pivotal studies, Phase III is where you would
start to see that amount. Smaller studies
you'd accrue less savings, bigger studies
more savings. Hope that helps. Ken.
DR. BUETOW: So I think several
speakers - and again, you all have
acknowledged this challenge of both ambiguity
and inconsistency of the current collection
of standards. And in a certain sense we find
ourselves maybe caught in this sort of
chicken and egg sort of universe right here
as acknowledged by you all as people don't
want to adopt the standards until they're
stable and harmonized, yet we don't have
harmonized standards. So can you help us see
a path forward that we could very rapidly get
to harmonized standards that seems like a
NEAL R. GROSS
prerequisite for this?
DR. HARDISON: I'll let Wayne
answer this after I just make one general
comment. It's that we've got to clarify
first, when we say "harmonizing standards" be
clear on which standards we're talking about.
And the second comment is within those
standards that CDISC are producing, it is a
number one priority to make sure our
standards are harmonized internally. And
from the board's standpoint we've invested
significant dollars to put, you know, put
something behind our words. And Wayne is
leading that effort and can say more about
that.
MR. KUBICK: There's a couple of
different points. I think first of all
different standards represent different
constituencies and so the more broader you
make the scope the tougher the problem
becomes. Within CDISC when our original
scope of supporting standards that support
regulated clinical research among the

commercial pharmaceutical industry, I think
we feel our standards are pretty solid.
There's work to be done to tie them together,
but we think they're usable and that we're
committed to making sure that there's going
to be a clean, upward migration path for any
work done forward.
The second part of the issue is
harmonizing with the broader world and we're
working together with NCI and with HL7 RCRIM
on the BRIDG model, the bioresearch
information development group, domain group,
the BRIDG model. And we're making sure that
we're representing all the standards and the
semantics for all the standards and the
semantics for all the standards in this
common domain analysis model. We've been
very successful to date without having to
really make any structural changes to the
existing standards, but making sure that we
make the interconnections between them. That
NEAL R. GROSS
really is a critical centerpiece moving
forward to make sure these activities stay
harmonized.
DR. BUETOW: Maybe a riff on that
same thread then and it goes back to the
early question of mandate versus consensus or
incentivize. Would you argue then there's a
strong role for mandating specific standards
for the deployment and then other things will
harmonize or lock in around those?
DR. HARDISON: Well, selfishly,
mandates are always good for standards
development organizations.
MR. KUBICK: But a mandate doesn't
necessarily have to take the form of
legislation. A mandate that, you know, we
need this in order to achieve our principal
mission of delivering better healthcare to
the world and ensuring patient safety is a
strong mandate. If we keep that consistent
message and make sure that the right things
to do in support of that mandate I think we
can make a lot of progress.

DR. HARDISON: And if I may add,
that's why having an organization of
public/private partnership that provides an
environment for actually demonstrating the
efficacy of these standards if you will is so
critical. Because that then would allow us
to more rapidly pilot and move from the
actual development of the standards to
implementation of the standards in an
accelerated way.
DR. LEVIN: I have another
question if I could, going back to when we
were talking about the timelines on
investment. Can you expand on that a little
bit? How would that work? Can you say a
little more, like if you're going to now go
in to adopt a CDISC standards, move it into
your system just at a company level, what are
the timelines that are associated with that?
Or how is that?
DR. HARDISON: Well, it depends
NEAL R. GROSS
very much on the size and complexity of your
organization, your legacy environment versus
you know a small startup biotech where you
have a green field. So it's going to vary
tremendously depending on your starting
point. Now, Wayne, you've been involved with
several implementations. Do you want to
comment specifically on that?
MR. KUBICK: Well, I think again
partially as a consequence for some of the
other issues we said earlier, a lot of
companies have been adopting standards pretty
much at the end stage process. You know,
taking what you get out and mapping it and
that's seldom an effective way to do things.
The critical issue is moving things upstream.
And it really isn't necessarily a huge issue
if you start out with the standards embedded.
I think the problems we have is sort of
unlearning the years of legacy processes and
individual standards that have been done.
Some companies make the transition very
quickly in a matter of months. Others,

larger companies certainly take longer times.
You know, that's the transition
implementation path that has to be followed.
DR. LEVIN: If you're starting off
with a new study, I mean that would be the
easiest transition, I guess.
MR. KUBICK: Absolutely.
DR. HARDISON: Yes. I have
clients that I've worked with who, once they
made the commitment to the standards, they
required a mandate in their organization that
said all new studies starting January 1, 2007
will use the standards. And then you have to
make a decision well what about ongoing
studies. Given where they are in their
lifecycle, is it cost-effective to convert
them or not? And you know, that's just a
business decision that you have to make on a
study-by-study company-by-company basis.
much. Our next speaker will be Mr. Tom Klaff
NEAL R. GROSS
from Surety.
MR. KLAFF: I want to thank you
all, especially the panel for letting us
speak today on the electronic submission
standard and what Surety's perspective is on
this. My name is Tom Klaff. I'm the CEO of
Surety and we are a trusted third party time-
stamping company. Some call us a digital
notary. And I stand here to represent not
just my company, but also an industry, a
burgeoning industry called the trusted time-
stamping market. And only about three years
ago it really didn't exist, but as many folks
who are familiar with record management and
migrating from paper records to electronic
records and also through what's happening
with the Federal Rules of Evidence and the
Federal Rules of Civil Procedure, the need to
prove integrity of data is really very
important and especially when organizations
rely on this electronic information that
constitutes roughly 70 to 80 percent of their
asset value, it's very, very important.

What I want to talk about today in
a very short time is just give you a brief
introduction about who we are and then go
into some of the obstacles faced by going all
electronic. We have a lot of experience with
this. We have several customers who have
made this decision or are making this
decision and it required the need to prove
independently of their people, processes and
systems that all of their electronic records
and forms and meta data and audit logs had
never been altered either maliciously or
inadvertently. I want to talk about also
from the perspective of a third party
provider. I know part of the submission
standard is who would be able to provide this
kind of service from an archive standpoint.
Surety has a lot of experience working with
third party archive service providers because
it attacks a wholly different problem. When
you're a custodial data provider, you're
NEAL R. GROSS
managing other people's electronic records
and there's a need to mitigate that risk and
to prove legally that during its chain of
custody those records were never maliciously
altered. I want to talk about the industry
standards surrounding trusted third party
time-stamping and as I indicated before, I am
also a representative of what's called the
Information Assurance Consortium which is a
body of time-stamp vendors who have come
together to form a standard working with the
American National Standards Institute, ANSI,
in what's called the ANSI X9.95 which is
endorsed by many of the financial service
companies that you know today including also
NIST and NSA. And then lastly I want to just
give a brief implementation scenario about
how this would work inside of a live
customer.
So Surety is an IT security
software and services company. We're based
in Herndon, Virginia and we provide software
and we operate a third party trusted time-

stamping service called AbsoluteProof.
AbsoluteProof enables our customers to
protect and defend their intellectual
property by independently proving the
integrity of their electronic records from
the point that they were created ongoing for
permanent preservation. The issue that we're
addressing as part of third party trusted
time-stamping is legal defensibility. And
underneath the legal defensibility angle
there's intellectual property protection and
litigation readiness. Many companies we work
with in the life sciences market have made a
conscious decision to move from a paper-based
record management system to an all-electronic
records management system. Some have
deployed electronic lab notebooks. Some are
going with electronic records management
systems and building their own custom
applications but in each instance they've all
recognized the need to integrate the ability
NEAL R. GROSS
to prove that their documents, their
electronic IP was pristine. And especially
when it comes into litigation, and especially
considering the recently amended Federal
Rules of Civil Procedure. During meet-and-
confer sessions those files could be turned
over in electronic record discovery and there
is a need also from the electronic file
standpoint to prove that those haven't been
altered. So authenticity is very, very
important.
A number of reasons why data
integrity is a specific issue. First and
foremost about 93 percent of all business
records today are electronic. And everybody
considers the outside attacks being a clear
threat to infrastructure, and clearly it is,
but what people don't realize for the most
part is that for the most part, 60 percent of
IT security breaches happened inside the
enterprise with trusted insiders. And those
are the databases and the record management
systems and the electronic assets that are

really very, very important. Also with the
advent of more publishing software it's very,
very easy to tamper with electronic records
if you are an insider, but very, very
difficult to detect the manipulation of those
records. And lastly as I indicated, data
integrity is really a key component as part
of anybody's electronic vital legal defense
if they're going electronic, and that's going
to be a very, very important issue to
consider going forward.
So what does time-stamping
provide? Well, it's really an added layer of
assurance to what companies are doing in the
electronic world. Clearly we're not talking
badly of electronic signatures or digital
signatures. We're looking at this as really
an addition or an additional layer of
assurance because at the end of the day
digital signatures could expire, they could
be compromised. There needs to be a
NEAL R. GROSS
cryptographic binding of an electronic record
with a secured audible time source and that's
what we're advocating. So really from the
standpoint of trusted time-stamping it's the
what and the when in addition to the who that
makes a very compelling argument to defend
legally your electronic records.
Lastly, there's a need to prove
irrefutably that those documents that you
possess, those forms that you are archiving
and managing are trustworthy. Sandra Lambert
who was the editor of the X9.95 standard,
quote, says, "Trusted time stamps will
replace mechanical, manual or system-
generated time stamps and provide a tool that
will offer an evidentiary trail of data
authenticity." Anybody that has a laptop
knows how easy it is to turn back the date
and the time on your laptop. And we have
several examples at Surety where we can
demonstrate that even with a PDF file with a
backdated time clock and a digitally signed
document for that matter can actually show

that a record that existed - even an article
from the Wall Street Journal that was
published on January 17 could be signed by
somebody with a digital signature on January
5 and then validated. So from a legal
standpoint that is a clear piece of evidence
that could be challenged and could be thrown
out.
So why do customers choose our
service? One, and first and foremost really
from the life sciences standpoint and from
the semiconductor industry and the
manufacturing industry and the aerospace
industry and even the government is to be
able to prove that your intellectual property
is yours. It's an effective deterrent with a
time-stamping service integrated into your
business process to deter others from
actually going in and manipulating electronic
IP. Secondly, it's the ability to really
monetize the value of going fully electronic.
NEAL R. GROSS
A lot of organizations go halfway. We call
them hybrid. They actually want to go
electronic but also use paper printouts as a
way, as their redundant record management
backstop. And really that is a very costly
endeavor indeed. So the ability to prove
non-repudiation is important and with having
the ability to attest the integrity of that
document similar to a wet signature is
really, really important.
We talked about legal
defensibility and authenticity and then the
ANSI X9.95 standard is what I want to get
through next. AbsoluteProof application
areas, other time-stamping providers can
actually seal any sort of data. It could be
Word documents, PowerPoints, it could be
audit logs, meta data, it could be audio,
video, it really doesn't matter. At the end
of the day it's asset data that is important
to the end user. And if it's important to
the end user, it needs to be protected.
From a third party service

standpoint if I am an archive service I need
the ability of three key things. One, I need
to have the ability to have long-lasting
protection because if I'm managing a lot of
infrastructure, there's no question that my
infrastructure will obsolesce over time.
It's the ability to constantly upgrade that
environment and know that the records that
are being managed and stored will not lose
their ability to be proven authentic.
Secondly, it's the ability to be
independently verifiable. And when I talk
about independent verifiable, I mean
independent of your own customer's people,
processes and systems and of yours. So these
all could be challenged legally.
And lastly, it's the need to go
with standard solutions, and that's what I
want to talk about now. The ANSI X9.95
standard is a standard form for trusted time
stamp management specifically endorsed by the
NEAL R. GROSS
financial services community. It was
unanimously approved back in 2005 and it
allows for choice. There are a number of
technologies that at one point were
proprietary and now they are part of a
standard so that there is vendor neutrality
and the ability for customers to know that
their investment is protected in time stamps.
Under the standard there are four specific
technologies that are proved and endorsed.
One is a digital signature type of time-
stamping, two is a message authentication
code, three is link token, which is what we
do, and the fourth is transient key. And
within the standard there's a comprehensive
set of processes, procedures that would allow
organizations who wish to deploy these time
stamps to be complicit.
We are often compared to an RFC
3161 standard which really technically isn't
a standard. It's just a request for comment.
But the X9.95 trusted time stamp standard
does include digital signatures which RFC

3161 supports, plus the other three
techniques. What's also key is that there is
policy and practice statements involved in
the standard. There's audit evaluation
criteria and the process is well spelled out,
including time calibration and auditability
of those time stamps that come from secure
sources.
With the last remaining seconds I
have left I just want to go through an actual
implementation scenario where an organization
has migrated from a paper-based record
management system to an electronic record
management system and has used a third party
time-stamping service provider like Surety to
seal and validate the integrity of their
electronic IP. And this happened to be
integrated into an electronic lab notebook
infrastructure also using electronic
signature. So from really the concept of the
idea until the preservation of the idea there
NEAL R. GROSS
is a seal that is assigned to that specific
file. That file can be stored in any record
management or archive module. And then at
some point in the future if that file or if a
patent or a trade secret is being litigated,
that piece of evidence can be sent along with
the trusted seal to a third party for
validation. So the key point here is that
it's very, very important when considering
electronic submission systems and management
systems that you consider the legal
defensibility angle and know that down the
road 10 - 20 years from now when challenges
do occur and the people who were originally
there aren't there anymore that those records
have the ability to stand up in court.
In conclusion, there are a number
of ways that you can go about doing this. We
believe that as organizations go through the
electronic record management process and
decide for themselves that paper isn't
necessarily the way they want to go, that
there needs to be a mechanism in place for

being able to prove the integrity of their
electronic records. And Surety's
AbsoluteProof time-stamping service is one of
those mechanisms. And we've been able to
work with organizations who wish to protect
their IP, but also organizations who from a
third party standpoint wish to mitigate their
risk and be able to prove the chain of
custody has never been altered. Thank you.
Yes, Kevin.
DR. FAIN: Yes, I have one
question. I was interested in you mentioned
that the time stamps have been used in the
context of private litigation for
authenticity of documents. I was wondering
in terms of FDA and FDA inspections, are you
aware of those time stamps being considered
or used by companies, especially when FDA is
inspecting records? And I guess a follow-up
NEAL R. GROSS
question to that would be are there any
unique issues for FDA to take into account in
considering these time stamps?
MR. KLAFF: Well, I think that I
mean from our company standpoint we haven't
engaged with the FDA in any sort of
commercial endeavor. But I would add that
the time stamp technology that we use really
is - if you're familiar with secure hash
algorithms, they're NIST-approved so there
isn't really anything proprietary about them.
From a litigation standpoint typically
technologies aren't challenged as much as the
processes are challenged. So from that
standpoint all we're doing is we're helping
organizations who have specific processes in
place know that in a litigation standpoint
you can surely challenge their processes, but
when it comes to the actual data there is
irrefutable proof that nothing has been
changed. And we're advocating that if you're
looking at standards from an electronic
submissions standpoint, you might want to

consider the ANSI X9.95 standard.
from the panel? Thank you very much.
MR. KLAFF: Thank you.
DR. WOODCOCK: Our next speaker is
Nancy Smerkanich and she is Vice President,
Regulatory Affairs, Octagon Research
Solutions.
MS. SMERKANICH: Thank you Dr.
Woodcock, members of the panel for first of
all for pronouncing my name right. I don't
get that very often. I appreciate it. Thank
you for allowing me to speak today. A little
background about myself and why I wanted to
present today. I've been a regulatory
affairs professional for 20-some odd years.
I started off my career in big pharma. I
worked as an independent consultant and for
the last six years I've been at Octagon. My
primary responsibility in all that time was
compiling and maintaining both
NEAL R. GROSS
investigational and new drug applications. I
am certainly not only a believer in
electronic submissions, but probably some
people would consider me a bit of an
evangelist about them. I spent the first 10
years of my career saying there's got to be a
better way to do this and then I found the
better way and I want everybody else to do it
that way as well.
For purposes of full disclosure I
should tell you that I am working at Octagon.
I'm not only an employee but a stockholder.
And we are a software and services provider
to the pharmaceutical and biotech industry,
primarily focused on electronic submissions
but also the data management and data
integration that's involved in creating those
deliverables for electronic submissions. So
the benefits may seem obvious to you if
electronic submissions become mandated, but
to me it means I get a whole lot busier. And
I'm already very busy.
But the most compelling reason

that I asked to present today has to do with
the fact that we sent an email blast out of
all of my regulatory contacts which numbers
between 200 and 300 people about this meeting
because a lot of people have trouble keeping
up to date on the docket and so we sent out
the email blast and the response to some very
simple questions that were in that email was
overwhelming. In addition to the responses
to what I'm calling an informal survey, I
received numerous voice mails, emails, people
even tracked me down on my cell phone to ask
me to present information on their behalf
today because especially for small to mid-
sized pharma there's a lot of regulatory
people who wear multiple hats who knew they
wouldn't be able to attend today. So while
there were many great questions I think in
the docket I'm going to confine my remarks to
transitioning to and implementing electronic
submissions from the point of view from these
NEAL R. GROSS
responders. Hopefully I'll be able to answer
some questions from my own experience. And
then there was also a number of questions in
the docket about cost and time of doing
electronic submissions so I have a couple of
scenarios that I can present to you.
Hopefully you'll find them useful. They
involve not only the cost of the production
to do an electronic submission but also the
time involved.
So we took the questions in the
docket and kind of boiled them down to a
little bit leaner, meaner set of questions
basically asking folks have they implemented
electronic submissions within their
organization and if the answer to that was
no, why no. And then asking those who had
what were the major challenges of
implementing electronic submissions and what
were the major benefits. And then
specifically trying to get at some of the
concerns around cost we asked if those costs
associated with doing this was burdensome.

And then we left the field open for
additional comments. And I have to tell you,
you'll see in the next few slides where there
were some people who didn't answer any
questions but wildly used the additional
comment field. So I've tried to summarize
those as well. So I received 77 responses in
seven days plus all the emails and the voice
mails. So this is obviously a hot button
issue for a lot of folks out there. The
companies represented were either tier one
big pharma, tier two you know kind of mid-
sized pharma or small pharma as well as some
additional participants who represented CROs,
medical devices and vet meds.
And then the answers were kind of
split amongst have you implemented electronic
submissions within your organization. These
are overall percentages. The tier one
companies, 10 yeses, they equated to 13
percent, six nos. The tier twos were split.
NEAL R. GROSS
The tier threes had 26 companies that had and
28 that had not, and the others were split
equally amongst them as well.
So when we asked the nos why no,
why haven't you done electronic submissions,
some kind of interesting information perked
out of this. For tier one companies they
cited lack of internal resources and
expertise, lack of upper management support
which in big pharma is an endless chain at
times. They cited costs and they cited
ingrained processes. And I think a number of
people today have addressed the fact that big
pharma can be very hard to move to a new way
of doing things. It has a lot to do with
change management. Tier two companies were
both undergoing pending acquisitions. I
don't know if that was of their companies or
of their drugs, but it certainly puts you
into a holding pattern if you don't know what
your future is going to be. As for the
little guys, the tier threes, they cited
time, costs, expertise and resources. And

again, we'll get into a little bit more about
what I think time, cost, expertise and
resources mean, but in fact a number of them
were in the stage of assessing how to go to
electronic submissions and a number of them
were also implementing. They too had
management constraints. The fact that
electronic submissions aren't mandated was
definitely a barrier for some of them. Some
of them it was just an issue of timing. They
didn't actually have a major submission that
would warrant it. All of the companies that
were in the ancillary area were in the
implementation stage so I think that's
encouraging. And as I mentioned, you'll see
34 no responses. It just means they went
directly to the comments field.
So for the folks that had gone to
electronic submission we asked what the major
challenges were. And you'll notice that tier
one and tier three have identical responses,
NEAL R. GROSS
but I think it has a lot to do with what time
means to big pharma and what time means to a
smaller company. Time to change existing
process to integrate with numerous other
tools for big pharma and lead time for small
pharma. Just having that built into their
development process. They both cited cost.
Again, cost if one of those things, for big
pharma it's a lot like renovating an old
house, but for new pharma it's like new
construction, and there are a lot of costs
associated with both. The fact that both
cited expertise I think is interesting and it
kind of speaks to where the workforce is and
where the experience level is with electronic
submission.
Training, obviously very important
and resources. And I think a lot of time
resources for a small company can mean
everything from an IT infrastructure to an IT
support person as opposed to dedicated
resources in a bigger pharmaceutical company.
Both cited process change, but again I think

they mean different things. I think for tier
one pharma you truly are talking about
changing existing process, but for smaller
pharmaceutical companies you're talking about
no process most of the time.
There were some technology issues
cited by the tier two companies and some
additional things for the tier threes which
was they just didn't even know how to create
compliant documents and didn't have document
standards. They dealt with a lot of legacy
study information and again, probably most of
it outside of their control. And that they
couldn't communicate to the management that
these challenges could actually be time-
consuming and costly.
The major benefits. Really they
validate everything I've been saying for the
last six years so I really am happy to
present this slide. Both - actually across
all of the companies the major benefits of
NEAL R. GROSS
electronic submissions were seen to be the
ease of use within the organization, the
speed of compilation. As one of the end
people it is faster, but it isn't a push of a
button. Compatibility with other systems was
very important for big pharma as were
harmonization mostly across sites and across
subsidiary companies, reviewer friendliness
and document reuse. I love the fact that all
three tier companies said no paper because
that certainly was recognized as a benefit.
And it was interesting to me that the tier
one companies perceived that there was a
quicker agency approval with electronic
submissions although we've never been given
any metrics to that effect. I'm looking at
Dr. Levin when I say that. For tier three
companies again they stated that the same
benefits and so I think there's a real
consistent message there in terms of what are
the benefits.
In response to the question about
costs it was a very interesting split here.

Tier two companies you'll remember from the
last slide were the ones that had technology
issues and they clearly felt that there was a
burden to the cost of implementing electronic
solutions. But tier one and tier three
companies were relatively split amongst them
as were the others and the others were
actually more favorable in terms of there not
being additional cost burden on creating
electronic submission deliverables, but I
suspect that's because they were from CROs
and so they basically have electronic
databases and electronic documents so they
weren't implementing a whole enterprise-wide
solution.
Fifty-one of the respondents
provided additional comments, some of which
were very lengthy. It was a good venting
session for some people I have to say. And
so to try to summarize them was actually a
little bit difficult. But since everyone hit
NEAL R. GROSS
on cost I did make a point around summarizing
that, that most if not all of the companies
in some way or another cited costs as an
issue but felt that the cost of implementing
electronic submissions would be worthwhile in
the long run. And I have to say I take this
quote and I'm probably going to reuse it
again because it's a good one from a small
company. "We build and collect things
electronically. It makes sense to submit
them electronically. It's the cost of doing
business." Some companies felt that the
impact of the costs actually hurt them in a
competitive environment. And I think that
this comment also related to some issues that
especially small pharmaceutical companies
face when they have limited resources in
terms of head count and they have to choose
someone to place on the clinical team or
someone to place in regulatory operations.
Guess who usually wins? During the
development phase clinical usually wins.
Some companies felt that finding the correct

technology that's reliable and current is not
only an issue but will continue to be an
issue. I remember one of the comments
actually said getting the right people and
the right tools at the right time was
probably their greatest challenge. And there
were at least seven if not more companies
that responded that they really wanted to see
ECTD mandated because it's the only way they
felt they were going to get buy-in from their
upper management and to move their
contributors away from a paper paradigm. So
again, that's just a sampling of the survey
results.
Since everybody talked about time
and cost I thought it would be maybe
beneficial for the panel to see some
time/cost information. It's based on an
outsourcing model because that's what I have
access to and it's very important that the
outsourcing model be considered as part of
NEAL R. GROSS
this because I think it is where all of the
time, the costs, the resources and the
expertise lie. You're basing these costs on
having a validated model, validated software,
a pressure-tested process and emphasis on
pressure, and a trained staff. And so making
sure that that's your baseline that you're
using created a couple of scenarios around
this, one of which is that you get all
submission-ready documents. For people who
live in this world they know that's a bit of
a dream, but content and format having been
reviewed and approved upstream of you. It's
also assuming that documents and data are
provided on a rolling basis. It's not this
giant bolus of work at the end. And for
Scenario 1 I used a smallish submission,
SNDA, a small 505(b)(2), even an IND around
150 volumes with about half of it coming from
an electronic source versus a paper source.
And you can see that assuming that
infrastructure is in place, electronic
submissions can not only be cheaper, about 15

percent, but you can also do them faster.
I created another scenario based
just on a bigger submission but again on the
same model and you can see that the total
cost again for electronic ended up being
cheaper, not as big a difference, about 11
percent. And also easier to compile, faster
to compile. I want to hover on this slide
just for a minute, though, so that everybody
knows. This is not a cheap thing to do.
Either way you look at it you're talking
about over a half a million dollars, and this
combined with a user fee that's creeping up
to around $900,000 is a tremendous amount of
money for a small company. And so that was
one of the points somebody asked me to make
and so if you consider some of the
incentives, you may also just want to
consider the overall cost. And this doesn't
involve any of the costs around doing any
data integration or data standardization.
NEAL R. GROSS
So just to conclude and leave a
few minutes for questions, the major
impediments and the challenges kind of across
the board have to do with time from different
vantage points. Certainly there's cost
there. There is more expertise needed, but I
think there's also more expertise being
gained. Training is always an issue.
Getting the same folks out to meetings and
out to training can be difficult if they're
wearing multiple hats within a company.
Resources and I think process change is
probably underestimated. But electronic
submissions has delivered on its promise in
terms of major benefits, ease of use, reviews
both internal and external, the speed and I
think the one I'd like to highlight, the
accuracy which people doing electronic
submissions felt their submissions were
better. And I think that's certainly maybe
the takeaway from this is that you know we
have accomplished better submissions and so
that was certainly the point of all of this.

So I have a few minutes left for questions.
I do hope the panel found the information
useful.
much. Questions from the panel? Randy
Levin.
DR. LEVIN: Thank you for that
presentation. It was very helpful. And
thank you for all your help in electronic
submissions over the years in all the
different venues. You've been doing a lot of
work to help people move to electronic
submission so I want to thank you for that.
MS. SMERKANICH: Well, I
appreciate that. The template of the slides
now is called Levin-esque. You can find it.
DR. LEVIN: There's a little bit
of color in this slide.
MS. SMERKANICH: Well, there is.
More than you would have liked I would
imagine.
NEAL R. GROSS
DR. LEVIN: The question about the
cost and going back, and what about
implementing over time and reducing your
cost? How does that work or can you expand
on that a little bit?
MS. SMERKANICH: Well, since a lot
of the cost is based on implementing the
software, a lot of times that's going to be a
one-time cost so this is kind of taking that
out of - this is truly just costs around
production, doing the reviews, doing the
post-production changes. So it would be the
same consistently. In terms of time and cost
to implement the software and the people and
all of that, there's a lot of variables that
go into that and it changes from whether
you're a small company with a blank canvas to
a big company that has a lot of systems to
integrate, certainly.
DR. WOODCOCK: Randy Lutter.
DR. LUTTER: Could you go back to
the time and cost Scenarios Number 1 and
Number 2? I'd just like a little bit more

clarity if you would about exactly what these
numbers are supposed to describe. These are
the costs of preparing an NDA?
MS. SMERKANICH: Yes. Two
different size NDAs from the point of where
the documents are ready for if you want to
call it the publishing step. And electronic
in this case was both - an average actually
was almost a wash of doing an electronic NDA
with PDF tables of contents versus an ECTD.
It was pretty much a wash with that.
DR. LUTTER: So excluded from this
is any improvement in interactions with FDA
as a result of having submitted an electronic
document as opposed to a paper document?
MS. SMERKANICH: Yes, actually we
did have a minor couple of lines in there
about the ability to answer questions from
the agency based on a paper submission and
the ability to do re-analysis. And clearly
those are what bumped up the paper costs
NEAL R. GROSS
because they tend to be faster and quicker
and more efficient interactions with the
agency if you do have both the SAS data sets
in the proper format as well as the
electronic submission with all the
navigation. So we did include that, yes.
DR. LUTTER: Thank you.
Armando?
DR. OLIVA: Is there anything
upstream that the company could do before
publishing or coming to you for publishing
that would help reduce these costs, either
paper or electronic?
MS. SMERKANICH: Yes. I think
that's an excellent question and I think a
number of the speakers, especially the
gentleman from CDISC have addressed that. I
mean, the more standardization that occurs
upstream in terms of the data and the
documents, the more efficient the downstream
process. And so you see companies that have
good standards, employed standards, standards

that everybody has trained on. They can
actually create submissions a lot faster than
companies that have to go and re-map data or
start with documents that have, you know, no
consistent field. But having said that, all
of those upstream costs aren't even in these
scenarios and they can be considerable as
well.
DR. WOODCOCK: Ken?
DR. BUETOW: So from your
outstanding boots on the ground experience
here, what would be some critical things we
could do to facilitate this? So in other
words, obviously you're an evangelist, so
what are, you know, what is keeping you from
proselytizing at this point?
MS. SMERKANICH: Well, nothing
keeps me from proselytizing. A lot keeps the
message - well, I mean a lot of it is this is
perceived to be a nice-to-have I think by a
number of people. So when you see tier one
NEAL R. GROSS
companies saying they don't have upper
management support, that speaks to me that
this is perceived as a nice-to-have. I think
again to speak on behalf of the people who I
claim to be here for, there are so many small
companies that their focus is - it's a day to
day thing. It's get this study done. It's
get my database clean. It's get my database
locked. It's get my tables and listings, you
know, done. And for them to see the big
picture and to see the end product, sometimes
they don't even consider it until they're
sitting at the pre-NDA meeting and someone
says to them, well, is your data going to be
in CDISC format or is it just going to be an
electronic submission. So a lot of times
it's a matter of priorities for them. They
have to spend their money wisely. So much
drug development is being done at such a
lower level now than when I started it's
amazing to me, and those people have a
totally different focus. They're focused on
the drug discovery and the development piece

and they're not necessarily thinking about
the marketing application.
DR. WOODCOCK: Randy Lutter.
DR. LUTTER: One follow-up. These
data give such a compelling argument for
industry to submit electronically.
MS. SMERKANICH: As long as you
have that top piece.
DR. LUTTER: Rather than via
paper. What additional action do you
recommend for FDA in particular? Are there
mandates or incentives that you think we
should provide?
MS. SMERKANICH: I think that, you
know, again I'll kind of echo what Dr.
Hardison said, that incentives I think for
small companies probably hold more weight,
but mandates for bigger companies hold more
weight. I think that also in terms of just
communication issues, knowing that the FDA
was 100 percent all review divisions,
NEAL R. GROSS
reviewers behind this would also help a lot
because there still is this fear out there
that, well, my reviewer doesn't like this, my
reviewer likes paper, my reviewer isn't going
to use the data sets. So I think if we can,
you know, that's certainly - that's an easy
win as far as I'm concerned, but then they're
not my thousand people, so.
DR. BUETOW: Just real quickly.
So other than communications, are there
barriers? I mean because again, as someone
said there appears to be from a cost model,
you know.
MS. SMERKANICH: You're going to
pay one way or another.
DR. BUETOW: It's a no-brainer.
So what other barriers would you perceive
actually as a person who's literally doing
this that are actually slowing this down
other than communications?
MS. SMERKANICH: All right, I
think it's knowledge. I think if more people
saw what they look like at the end, and

especially with ECTD, which is my new thing
I'm evangelizing. If they saw the long-term
benefits of starting their INDs in ECTD and
moving forward with them I think that they
would, you know. And the fact that
redundancy goes away. I think that's one of
the things that has driven me absolutely
insane over the years is the amount of
redundance in the information that we submit.
Thank you very much for mounting that survey
and presenting.
MS. SMERKANICH: You're welcome.
DR. WOODCOCK: All right. Before
I introduce our final speaker of this morning
I would like to reiterate and ask people
please do not wander off into the office
suites here, okay? We don't have a facility
that's secured, you know, between the public
access and our employees. So please stick to
the halls and so forth, okay? Our final
NEAL R. GROSS
speaker this morning is Mark Scheineson who
is with Alston & Bird, LLP. Mark?
MR. SCHEINESON: Good morning.
Again, thank you for the opportunity to
present to this august panel. I'm
representing some of the little guys in
proprietary and generic drug development.
It's an honor to be back here with my former
FDA colleagues. I'm going to try to go
quickly both because Nancy's presentation
that you just heard was quantifying some of
the points I'm going to be making here and
because I'm standing between you and lunch.
The goal obviously that all of us
are discussing today is extremely laudable,
especially if it expedites reviews and allows
cross-application comparisons of safety data.
I would like to just get to the bottom line
and answer Randy's question or Dr. Lutter's
question which is, you know, mandates versus
incentives. From our perspective as
regulatory counselors to companies, we
believe what's holding back a flood of

electronic information that's ready to go is
a little bit of uncertainty on how to prepare
those applications and uncertainty on how
reviewers and uncertainty on how
investigators or inspectors would handle that
information and database. So from that
perspective, you know, any uniform standards
or guidance that the agency could issue that
helps specify some of those open questions as
well as the prospect of quicker reviews and
approval is enough to unleash a flood of
electronic data.
From the smaller companies'
perspectives there must be recognition that
the tasks involved in a transition remain
daunting, especially for those smaller
companies. Some of those tasks have been
noted before and are listed below in training
and installation of a validated document
management system that would be acceptable in
GMP inspections as well as in regulatory
NEAL R. GROSS
reviews. It's also difficult to find
qualified vendors. I think the little guys
are concerned that the experts in the field
would be captured a bit by the bigger guys.
And I think we've seen that whenever new
rules are imposed or new requirements are
imposed that finding the experts, whether
it's, you know, NLE labeling or you know
written pedigrees or RFID or other, you know,
there are a limited number of vendors that
are qualified and they quickly get
inaccessible to the smaller guys.
Other elements to consider are the
large capital investment that is required to
put these databases into place. Most of
these companies would probably utilize CROs
to a certain extent to try to do some of this
work of them, but not everyone can afford
Parexel or Quintiles or some of the bigger,
more sophisticated vendors. Additional
employees would have to be hired. There's
somewhat of a cultural shift that's necessary
as was highlighted by management sign-off,

but once they hear that the best way to get
an application reviewed promptly or quicker
is electronic database, it wouldn't be hard
to get management sign-off in our experience
anyway.
This is a cultural shift. It's
not unlike, you know, PDUFA for FDA and the
changes that that made to a culture and to a
methodology. Global harmonization is also
key. You know, if any standards that are
used can be global or harmonized that will be
an additional incentive. And the benefits
are largely a bit in the future and not
immediate. There's also a need for
universally acceptable standards as has been
highlighted.
It would be good that in GMP
inspections that FDA would rely solely on
electronic information. I know it's
important sometimes in the audit trail to see
changes and deletions and things, but you
NEAL R. GROSS
know as all of us know that use Blackberries
or email, I don't know if that's reduced the
paper flow or people just make copies of
their emails. And you know, that's a
lifeline that would have to be cut both at
FDA and in industry.
The deadline that was suggested,
or a January 2008 date might be a little
optimistic. I'm going to quickly review
three issues and the thoughts of those. If
it's mandatory especially there needs to be
adequate transitioning, especially for those
smaller companies that don't have the
resources, but for large pharma as well.
That just won't allow sufficient time for
small and middle-sized companies to perform.
As was noted, the cost for
achieving electronic submissions is probably
higher and more burdensome, at least in the
short run. I think a lot of the responders
to Nancy's survey were polite in their
listing of code words, like "expertise" or
"cost" or "time." That "expertise" and

"cost" and "time" is probably on both sides
of this transaction at FDA as well as in
companies. And the pricing of these
services, especially if they're limited
vendors that have that capability would
likely be what the market will bear. And
that isn't going to be insignificant. In
addition, you know for submissions that are
already in progress there needs to be some
transitioning and some transition rules to
allow the scanning or converting or just use
of the paper until new applications can be
presented. It's both time-consuming and
costly.
If the need to consolidate safety
data is driving this, and that's very
important to be able to look across
applications, FDA might consider, and maybe
it is already using this, some kind of
standardized internal electronic reviewer
software or checklist that could be utilized
NEAL R. GROSS
that would draw together some of the data
that the reviewers are compiling in their
application reviews. That could be done
internally without any need for rulemaking or
guidance.
Secondly, time is an issue. The
timeframe must be realistic and should be
based on outside research. And this kind of
Part 15 outreach is a first step in that
regard. But there needs to be ample time to
hire consultants, to select the appropriate
submission software, to ensure and validate
its capability to roll out that software
before the submission preparation can begin.
You know, further delays could occur as I
pointed out from just finding the right
vendors. And of course for companies that
don't control it internally and some that do,
they can't always control the timeframe and
the priority that their submissions are
given. FDA hopefully will meet with the
vendors regularly to evaluate and to motivate
the process if it opts the mandatory route.

But new rules should be flexible based on the
progress in the marketplace. So from the
perspective of these small companies, they
believe that it should be no sooner than
three years if there is going to be a mandate
on which the agency - you know, three years
from the time the agency issues a final rule
or final guidance on electronic submissions.
Last issue is seeking some sort of
staged implementation. The agency should
allow for an ample implementation time.
There should be a built-in phase-in period
that's pretty specifically prescribed and the
agency should consider accepting paper to
accommodate those submissions. However, FDA
should ensure that equal treatment is
accorded to the paper submissions during that
phase-in period. Finally, FDA should ensure
that the electronic submission does not
become a vehicle for requesting companies to
include additional information or data beyond
NEAL R. GROSS
what's already required. That was part of
the problem I think in the guidance that was
issued for the original Part 11 is that you
know, what records were covered by that
guidance became a concern.
So you've asked us a lot of
questions. Some questions for the agency to
go back and consider I guess would be some
kind of uniform specific software or systems
for submissions that could be recommended.
Uniformity obviously will ensure efficiency
by streamlining your processes internally as
well as the preparation of very complex
applications. On the legal and regulatory
side is what mechanism you'd use. I think it
would be a lawyer's preference to use the
rulemaking process, realizing how burdensome
that can be, but it would be great to go to
Part 11 and get all the information you need
as opposed to digging up, you know, numerous
guidance documents and other documents which
aren't really binding anyway on either side.
We also do believe that this is a substantive

rule in how it's likely to be structured as
far as having standards and otherwise
recommending uniformity. And in any
rulemaking there's always a small business
assessment, and it would be great if this
small business assessment could be based on
consulting externally. Small businesses and
kind of not speculating on the time or
resources that might be required. I remember
the HIPAA rule that we're all familiar with
that said it would cost, you know, two days
and $10,000 or whatever the cost would be of
that methodology. Also, the question is how
can this be harmonized internationally the
most quickly for companies that most of which
market overseas as well as in the United
States. Again, I thank you for the
opportunity to present these views. I'd be
happy to answer any questions that you have.
much. Any questions from the panel? Kevin.
NEAL R. GROSS
DR. FAIN: One quick question. I
was just wondering about the rulemaking
option. Mark, if you have any thoughts about
are there some significant legal issues we
would need to consider in writing such a
rule?
MR. SCHEINESON: I think you'll
avoid a lot of those issues by going the
rulemaking route and a lot of, you know,
potential litigation from companies that deem
the requirements to be too burdensome that
would delay the process. I think there are a
line of cases, Shalala cases in the late `80s
that kind of had a 5-prong test of what it
should include in the substantive area and
why this kind of rulemaking would be
substantive and not interpretative. Thank
you very much.
DR. WOODCOCK: Well, are there any
other questions? Trying to get away from us.
All right. Thank you all very much. This
concludes the morning session. Again, please
restrict yourself to the hallways and doors

and so forth. And we will reconvene and
begin promptly at 1 o'clock. Thank you.
went off the record at 11:56 a.m. and went
back on the record at 12:57 p.m.)
DR. WOODCOCK: All right. We'll
be having three presentations at this point
and then we'll be having a break around
quarter of 3:00. Our first presentation is
from a set of presenters. I'm going to read
all their names and they're going to do an
ensemble presentation. Debra Bremer, Vice
President, Development and Medical
Informatics at Pfizer. Diana McKenzie,
Executive Director, Information Systems,
Amgen. Sue Dubman, Vice President of IT and
Informatics, Theravance, Incorporated. Dr.
J. Robert Beck, Vice President and CIO, Fox
Chase Cancer Center. Diane Paul, Patient
Advocate. Bob Renner, CEO, Liaison
Technologies. And Mark Adams, Senior
NEAL R. GROSS
Associate, Booz Allen Hamilton. They will
have about 45 minutes and we look forward to
what they have to say.
MS. BREMER: So in one of the
previous presentations they talked about a
survey and summarizing the survey. We
brought all the survey respondents here.
Good afternoon. My name is Debra Bremer and
I'm the Vice President of Development and
Medical Informatics at Pfizer. And I do
thank you for the opportunity to speak this
afternoon on clinical information exchange
and electronic submissions, and especially in
the most coveted time slot of the day. I
guess we drew the short straw. We'll try to
keep this moving along. Maybe that's another
reason for the presenters, we'll keep it
flowing. And I'd ask respectfully that you'd
hold Q&A until all of us are done, our
presentations, and we do believe we've timed
it in such a way that we should allow time
for questions and answers.
So today I am presenting on behalf

of the CRIX community. CRIX, which stands
for the Clinical Research Information
Exchange is a secure shared technologies
standards-based service platform. We are
developing this exchange to deliver value-
added services to the biopharmaceutical
industry and for use in information exchange
between research organizations or business
partners and regulatory authorities although
CRIX can potentially deliver similar services
to support other regulated products.
We as a community formed CRIX
International as a not-for-profit entity just
this month with the vision of a public-
private partnership in mind. An entity such
as this is essential for the initial as well
as ongoing success of a clinical research
collaborative environment. The board of CRIX
International will be composed of
representatives from both the public and
private sectors and CRIX International will
NEAL R. GROSS
govern and oversee the operations of the
services. While we've taken the initial
steps to form an entity that embraces a
public-private partnership in its very
structure, if desirable it could readily be
merged into an existing strategically aligned
non-profit organization. And I would say
that for more specifics on the role of CRIX
International we have submitted to the public
record for this hearing a CRIX executive
summary document where you can get more
details.
We believe CRIX services will
deliver to users workflow efficiencies, lower
operational costs, reduced infrastructure
development expenses and faster drug
development times. CRIX can also contribute
to accelerated achievement of the goals of
the FDA Critical Path and NIH Roadmap
Initiatives. For example, a shared public-
private infrastructure provides a platform
for streamlining clinical trials and
potentially assisting with adverse event data

mining. In later testimony you will hear
examples of exactly how CRIX can streamlining
clinical trials.
The CRIX community encompasses a
broad range of stakeholders. Represented
within our community are hospital groups,
patient advocates, government agencies,
academic research centers, trade and
standard-setting organizations and industry
service providers. The CRIX community is
constantly growing and our goal is that it
will embrace all organizations involved in
supporting or overseeing clinical research.
CRIX is a collaborative shared
technology exchange. So it is fitting that
it grew out of two previous collaborations.
Both of the preceding collaborations were
looking to enhance and accelerate the process
of developing diagnostics and therapeutics.
In 2003 FDA and NCI initiated the Interagency
Oncology Task Force, the IOTF, to streamline
NEAL R. GROSS
the development of cancer drugs. In 2004 the
IOTF focus expanded to include a shared
technology information exchange. At around
the same time the PhRMA trade association was
engaged in an effort called SEBIX, exploring
how to save time and money through the
development of a shared infrastructure and
means of information exchange.
Last year, NCI launched CRIX as an
initiative to demonstrate the value of a
shared infrastructure, leveraging the
momentum of the IOTF and drawing on lessons
learned from the SEBIX experience.
Development then began on the first service
module for CRIX, the Federal Investigator
Registry for Biomedical Informatics Registry
Data, which is why we call it FIREBIRD. To
date, FIREBIRD has been developed under NCI's
Center for Bioinformatics with knowledge,
technical expertise and resources contributed
by the broader CRIX community. Going
forward, we propose FIREBIRD as well as
future service offerings be developed and

overseen by the formal public-private
partnership represented by CRIX
International.
The CRIX community is active and
thriving with broad stakeholder group
representation. The community holds the
shared goals of lowering drug development
costs and speeding new therapies to patients.
We believe that an information exchange such
as CRIX is essential to achieving those goals
and we believe that CRIX has the financial
backing, organizational sponsorship and the
right governance and operation structure to
make it succeed. We have demonstrated that
an exchange can work through the FIREBIRD
pilot and we propose to release the
production version of FIREBIRD in 2007. With
the governance structure that we're putting
in place through CRIX International, we are
confident that CRIX will continue to grow its
services within clinical research and could
NEAL R. GROSS
be extended to support other regulated
products. CRIX will continue to grow its
community and deliver value to community
members and patients.
Next, members from the stakeholder
community will provide their perspectives on
the challenges of drug research and
development in the current environment and
the value of a third party information
exchange in addressing those challenges.
They will also provide information which
supports CRIX as an ideal model for
regulatory information exchange and
collaboration. Six speakers will give their
insights on CRIX, each bringing a different
perspective on the need for, benefits of and
governance structure of CRIX. We'll hear the
viewpoints of representatives from large and
small biopharmaceutical companies, academia,
the patient advocacy community, a provider of
exchanges and an external industry sector as
well as a service vendor. Thank you and I
will now hand over to Diana McKenzie from

Amgen who will provide the large
biopharmaceutical company perspective on
CRIX.
MS. MCKENZIE: Good afternoon.
I'd like to thank you for the time to speak
this afternoon also on the value that we
think CRIX offers to the broader research
community and to Amgen in particular. My
name is Diana McKenzie and I'm here to
represent Amgen. Amgen is a leading human
therapeutics company that serves the needs of
millions of patients worldwide. We strive to
be an entrepreneurial science-driven
enterprise dedicated to helping people fight
serious illness and we believe that by
aligning with the CRIX Initiative we
demonstrate our commitment to this
aspiration.
I believe we are all familiar with
the industry challenges we face in addressing
both the costs and complexity of research and
NEAL R. GROSS
development. In addition to these, Amgen has
experienced significant and positive growth
over the course of the past several years.
In light of both these industry challenges
and our own growth, we are required to find
ways to improve our business processes and
our supporting IT infrastructure so that our
increased scale does not slow progress with
developing important therapies. These
improvements will help us to reduce the cost
of clinical research in an effort to maximize
both our investment in identifying new
therapies and making those accessible to
patients.
FIREBIRD serves as an excellent
example of the type of improvement that not
only addresses challenges for the broader
clinical research community, but also has
value for Amgen internally. We expect that
with the successful deployment of FIREBIRD as
displayed in the slide here we could reduce
the cycle time associated with capturing the
1572 NCV during investigator site initiation,

which is a common industry process, from
eight weeks to mere hours. In addition, we
believe that these reductions in cycle time
also lead to higher quality information and
improved ease of review for our regulatory
colleagues. Lastly, we recognize that for
our investigator community we demonstrate our
commitment to easing the administrative
burden for them with participating in
clinical trials, thus allowing them to focus
on the patient.
Our internal business case for
FIREBIRD, while conservative, demonstrates
the proposed reduction in clinical trial
costs at Amgen for both the current phase,
which is limited to the 1572 and investigator
CV, and future phases where we expect to see
greater return when expanded to encompass the
entire site initiation packet. We have not
incorporated additional CRIX capabilities
such as the patient recruitment matching into
NEAL R. GROSS
our business case, however we believe that
these capabilities will only increase the
value of CRIX to the research community.
In summary, we believe that CRIX
is an effort, one of many being pursued in
the area of health information technology
deployments which benefits all stakeholders.
I've had the privilege of working in this
industry for 20 years and I have learned a
great deal about what it takes to
successfully deliver IT-enabled solutions
within research and development. I have been
actively engaged in discussions to advance
the exchange concept since 2003. It is
because of the diverse CRIX community
represented here today, the quantifiable
business case we have developed and our
collective desire to ensure all stakeholders
benefit in some way that we believe the CRIX
Initiative can successfully help us to
address the rising challenges of speeding new
therapies to patients and reducing the cost
of research to maximize their access to those

therapies. Thank you. I would now like to
turn the presentation over to Sue Dubman from
Theravance who will provide the small
biopharmaceutical company perspective on
CRIX.
MS. DUBMAN: So I'll lower this a
little. I used to be taller. I would like
to thank the FDA and the NIH for the
opportunity to speak here today. My name is
Sue Dubman and I know quite a few people in
the room. I was actually one of the
originators of CRIX when I worked at the
National Cancer Institute and have been a
supporter of the need for CRIX from its early
beginnings in 2004 and actually suggested
that we do a conga line around the room
because this is an important initiative.
Currently I'm Vice President of Information
Technology and Informatics at Theravance, one
of approximately 900 mostly small to medium-
sized biopharmaceutical companies in the San
NEAL R. GROSS
Francisco Bay Area. Where it happens to be
38 degrees this morning.
So just some facts. Small to
medium-sized biopharmaceutical companies like
Theravance play a critical role in new drug
development. And according to statistics
compiled by PhRMA and bio trade associations
as much as 70 percent of new drugs in major
disease categories are being developed by
biotechs, and many of these, if not most of
these are being developed by small to medium-
sized companies. As such, any new
capabilities that allow a small
biopharmaceutical company and its many
partners, CROs, large biopharmaceutical
companies, academic research institutes and
others, to improve their ability to bring
safe, effective drugs to market sooner and
for less expense are a win-win-win for
everyone, but especially patients.
So talking with many other small
biopharma and having just submitted our first
NDA electronically to the FDA, I believe that

CRIX has the potential - and it really is
potential - to not only streamline electronic
transfer of data to the FDA, but also to
improve the processes for collaboration with
industry partners in the future. I think
CRIX can provide cost savings and scale
efficiency if CRIX supports use and adoption
of common standards for regulatory submission
by all stakeholders. That's industry,
government and academia. Today we find a lot
of inconsistencies and overlap among the
various standards and the interpretation and
use of these standards by multiple parties.
So CRIX with its vision of a shared
infrastructure and tools used by all has the
potential to address these issues, thereby
increasing the efficiency of exchange and
making interactions and decisions easier,
faster and cheaper.
Also, CRIX because it is
standards-based can also potentially make it
NEAL R. GROSS
easier and more cost-effective to collaborate
with our many partners. Standards by their
nature facilitate the exchange of
information, that's one of the reasons you
use standards, and so - but today there's a
very steep learning curve and cost to adopt
these standards.
If the shared CRIX infrastructure
can provide capabilities to create and use
data in standard formats and make validation
of compliance with standards easier, then
CRIX will have a favorable impact on the cost
and time it takes to bring new treatments to
market. Finally, the CRIX community has the
potential to ensure that the standards
address the needs of all stakeholders, not
just big pharma. And so I think it gives a
voice for small biopharma, and I think that's
going to become important in the future.
One side benefit of CRIX that I
don't think we even understood when we
started this effort is that there are other
services - once you have an electronic

infrastructure for regulatory submission and
for collecting information in standards
structured format, you have the potential to
leverage those systems in the future. So
CRIX also provides potential gains in
interoperability of clinical systems in the
future. And so for example FIREBIRD, the
CRIX clinical investigator registry
potentially provides and authoritative source
of investigator, site and protocol
information for connecting patients to
trials. And this is critically important as
recruitment of patients to clinical trials is
a huge problem. According to market
research, upwards of 40 percent of clinical
trials costs out of nearly $6 billion spent
annually on clinical trials is tied directly
to patient recruitment, and greater than 80
percent of clinical trials have major
recruitment delays. And these costs aren't
going away. I have more detail here in my
NEAL R. GROSS
testimony, but I won't read it. But we need
to find a new way. I think that's the bottom
line.
We need to find a better, more
cost-effective ways to do drug development.
CRIX addresses one piece of the overall
problem. With the CRIX Initiative we have
the potential to stay focused on the science,
the medicine and most importantly the patient
rather than the information technology
infrastructure. And this is critical to all
of us because some day all of us, our family,
our friends and our colleagues will be
patients. When that happens, we will need
and want the best possible, most cost-
effective and safe treatment. And as my
mentor at the NCI Dr. Ken Buetow always says,
the question is not why now, but why not
sooner. With that I will hand over to Robert
Beck who will provide the research center
perspective on CRIX. Thank you again for
allowing me the opportunity to speak.
DR. BECK: Thank you, Sue. I

would also like to thank the FDA and the NIH
for the opportunity to speak today. My name
is Bob Beck. I serve as the Vice President
for Information Services and the Deputy
Director of the Population Sciences Division
at Fox Chase Cancer Center in Philadelphia.
I'm also an outside director of IDM, a
development stage public biopharmaceutical
company. I'm familiar with the CRIX
Initiative through activities in the
strategic planning workspace of the Cancer
Biomedical Informatics Grid, or CABIG. I'm
speaking as an individual although attempting
to represent the academic health research
community. We believe that the Clinical
Research Information Exchange offers an
unparalleled opportunity to strengthen the
academic clinical research community.
Managing clinical research at an
academic cancer center has logistical,
regulatory and management challenges. At Fox
NEAL R. GROSS
Chase we handle over 150 active protocols
from individual investigator-initiated
projects to industry-sponsored trials, large
NIH cooperative group trials. Each protocol
of course has its own forms, procedures,
addresses, rules, regulations. Investigator-
initiated protocols are particularly
difficult. Although the sine qua non of
clinical research at an NCI-designated cancer
center, these are quite challenging to export
beyond the single institution where they're
developed due to lack of standardization
across institutions or protocols. Sponsored
trials, while offering some limited
standardization within a corporation's
offerings, generate their own problems. As
one colleague comments, every time I work
with a new sponsor it costs me money. That's
because new sponsors mean new forms, new
organization of trial data and other
inefficiencies that standardization of course
could address.
We recently celebrated the 20th

anniversary of Fox Chase Partners, a
consortium of over 20 institutions in
Pennsylvania, New Jersey and Delaware. Our
partners participate in clinical research,
add many patients to clinical trials and form
the substance of a community clinical
oncology program, or CCOP. However, the
problems of clinical trial management are
multiplied twenty-fold when extending them to
this community. This group could function as
a single collective entity with improved
automation and standardization of protocol
management and investigator data.
CRIX, in particular the FIREBIRD
initiative, offers several positive features
for busy academic clinical research
organizations. Cost savings can be achieved
by reducing paper shipment, reconciliation
and storage costs, by creating a single
electronic master repository for investigator
regulatory documentation for the entire
NEAL R. GROSS
institution and its partners, and by
minimizing the number of documents the
coordinator or investigator must manage.
Increased standardization can arise through
the establishment of a single industry portal
available for use between all sponsors, all
sites. This would particularly help our
partners, our CCOP. Standardization across
sponsors of technical management of
regulatory documentation would be a plus as
would the standardization of
idiosyncratically designed forms.
Establishing safe credentials that can be
used for other systems would represent a
further important step. Also, improved cycle
time is of import not only to industry but to
busy academicians. Reuse of frequently used
information leading to reduction in the
number of errors is part of FIREBIRD's
appeal. Also, accelerated regulatory
document package completion will shorten the
time required to initiate a new study,
whether initiated by an individual

investigator, a cooperative group, or an
industrial sponsor. Most attractive to us is
the reduction of redundant sponsor requests
and data entry for routine contact
information. We of course want to enter and
maintain our static data once for all
sponsors. When changes then occur, updates
through a central portal would provide
accurate information to all our sponsors
simultaneously.
Automation of the clinical
research enterprise at the federal level
allows more protocols and sponsors at the
academic site without increasing staff
resources. FIREBIRD and CRIX promise to
minimize paper and paper-handling costs and
thus to simplify the entire compliance
environment. This in turn could enhance
patient safety even as it facilitates
investigator compliance with appropriate but
burdensome regulation. Most exciting is the
NEAL R. GROSS
potential of these systems to leverage other
information management and informatics
investments, such as the electronic health
record. Fox Chase Cancer Center as part of
its participation in CABIG is developing a
clinical data warehouse that can take results
of translational and clinical research
directly into consensus records populated by
the electronic health record. This project
can be rapidly accelerated by the sort of
standardization offered and promised through
CRIX. This project can be rapidly
accelerated with the sort of standardization
offered through the Clinical Research
Information Exchange which will leverage
institutional electronic record systems and
clinical research technologies for the
benefit of all participants. Now I'll hand
it over to Diane Paul, a patient advocate who
will provide her perspective on CRIX. Again,
thank you for the opportunity to speak today.
MS. PAUL: I'd like to thank the
FDA and the NIH for the opportunity to speak

to you today. My name is Diane Paul and I'm
a 13-year survivor of advanced ovarian
cancer. While I belong to a number of
different advocacy organizations, I am
speaking to you today as an individual. I am
a member of the CRIX steering committee and
have been for the last year and a half. Over
the last 13 years I have witnessed firsthand
the roadblocks that exist in bringing
treatments to patients. Time after time I've
seen patients willing and able to enroll in
clinical trials who have to go back on
standard treatments because of the delay in
opening of trials. I've seen patients
waiting for new drug approvals die before
those drugs get to the marketplace. When
treatments do receive approval, they are
often no longer of interest to patients
because newer, less toxic methodologies are
emerging.
Prior to January of 2000 I was
NEAL R. GROSS
employed in the Information Technology
Department at the Fashion Institute of
Technology in New York City as a systems
analyst. When I first came to FIT in 1979,
there were many problems with admitting and
enrolling students in a timely fashion. The
different silos of information which the
admissions, financial aid and registrar's
offices maintained caused similar problems to
what I see as issues in the drug development
and approval process. The first step in
solving these problems was to get
representatives from each area to work
together towards a common purpose.
Communication between the FDA, the research
community and the pharmaceutical industry is
the first step in breaking down barriers
which hamper them all. The CRIX Initiative
provides the framework for communication so
areas of agreement can be documented and real
problems can be defined and solutions
explored. With this type of framework all
three entities can work together for the

public good.
The CRIX Initiative will enable
the standardization and electronic transfer
of data. This will both speed the
development and the approval process. It
will service patients in a number of ways.
Treatment choices often must be made - sorry
about that - in a relative short period of
time in order to best serve the patient.
Patients are often unable to enroll in trials
when they and their doctors deem it
appropriate because of the delay in the
opening of the trials. Timely openings will
improve accrual rates. This will have a
ripple effect on the time it takes to
complete the trial and get information about
efficacy and toxicities to regulatory
agencies and hopefully to increase the speed
with which treatments are available for the
public. Cost savings resulting from
standardization and electronic transfer of
NEAL R. GROSS
data should allow for more investment in new
therapies as well as savings in treatment
costs for the public. Better standardized
information from trials should provide more
meaningful treatment information for doctors
treating patients in the clinics following
the completion of trials and treatment
approvals.
Currently cancer patients and
their doctors need to search various
databases in order to find trials which are
open and available. One way the CRIX
Initiative should help is to establish ways
for patients to find appropriate trials
across both the private and the public
arenas. Matching patients to trials across
the health continuum will be a big step
towards improving accrual times for trials
and the ability to search the public and
private sector at the same time will allow
better information for patients and their
doctor to determine what trial choices exist
and what is the best choice for the patient.

Patient advocates work on many
levels of research and treatment development.
The CRIX Initiative includes patient
advocates in all areas of its governance
structure to assure that patients' concerns
will be heard and met by the initiative. I
volunteer my time to the CRIX steering
committee because I believe it is of utmost
importance that we change the current system
to better service patients. I feel the CRIX
Initiative will help to do that. I will now
hand over to Dan Ruggles of Liaison
Technologies who will provide a perspective
on how other industries have dealt with the
establishment of exchanges. Thank you.
MR. RUGGLES: My name is Dan
Ruggles from Liaison Technologies speaking on
behalf of Bob Renner, our CEO, who could not
make the meeting today. I'd like to thank
the FDA for the opportunity to speak today
and to share our experiences in setting up
NEAL R. GROSS
and operating a consortium exchange. Liaison
Technologies, formerly known as
ForestExpress, was founded in June of 200 and
is headquartered in the U.S. with operations
in Europe. We are a private for-profit
company. We act as a shared infrastructure
provider for an exchange will I'll define
later in this discussion. Liaison has been
funded by Global 1000 companies with a clear
intent to foster and promote standards with
our implementation of an exchange. Today we
operate a set of systems that process about
3.1 million messages per month with over
3,100 connected companies.
There are two types of
architectures in an exchange. They're either
point-to-point or a hub-and-spoke. The
point-to-point model involves a series of
direct connections between two unique
endpoints. Most industries have deployed the
hub-and-spoke model for their exchange. As
can be seen from the table, these exchanges
have gained significant momentum and business

model validation over time, and involve the
active participation of competitors to work
together to help drive down their own
respective costs and improve their cycle
times. Some of the common themes of these
exchanges are it does require industry leader
participation actively leading with visible
high-profile companies, and it does require a
clear separation between the standards
bodies, generally not-for-profit, and the
operating the exchange, which generally
follows a profit model. Significant
investment is required if you want to start
one from scratch. Nearly all successful
exchanges report investments of between $100
to $250 million, mostly in startup costs,
before reaching substantial benefit for the
members.
Now I'd kind of like to talk about
the functions. There are usually three
functioning parts in an exchange. The
NEAL R. GROSS
initial driving force is the top one, the
industry association, which sets and promotes
standards, governance and coordinates
funding. They frequently act as the voice of
the industry, marketing champion. It's a low
financial investment initially, but requires
a significant amount of time with senior
industry leadership resources. On the lower
left to yours is the shared infrastructure
operator, Liaison. We actually operate and
execute the shared platform which we helped
build and design. And last there's the
certified solution providers. This is
nothing more than a typical leverage model
using economics to speed the adoption of the
standards and in some cases the adaptation of
the standards over time based on real world
experience. Our experience has shown that
validation and enforcement activities for
industry standards can only be maintained
through a shared services model. Without the
exchange, certified solution providers and
individual companies will deviate from the

standard to accommodate their individual
needs. The exchange acts as a control point
to localize those exceptions. Companies used
to the exchange can provide advantage in
terms of cost economies, quicker to market
and improved cycle times.
I'd like to share with you some of
our key learnings. From our experience and
observation of other exchanges as well as our
own, getting started required strong industry
leadership and commitment. This model
requires a long-term view. The economic
benefits may not likely be reached for five
to seven years, although a breakeven
operation may come within two to three years
if managed carefully. The consortium
exchange is a proven model as exemplified by
my previous table. The three key
stakeholders all have to participate, which
is the association, the shared infrastructure
provider or the exchange operator and the
NEAL R. GROSS
certified solution provider. And lastly,
it's a long-term commitment by all
stakeholders for success. Now I'd like to
hand over to our last speaker, Mark Adams,
from Booz Allen Hamilton, to provide a
service provider's perspective on the CRIX
Initiative. Again, thank you for allowing me
the time to speak.
MR. ADAMS: I want to thank
everybody again for the opportunity to speak
to the panel from the service provider
perspective. As said, my name is Mark Adams.
I work for Booz Allen Hamilton which is a
management consulting company located here in
the Washington, D.C. area. I'll be
addressing these questions in my remarks
today.
The CRIX service delivery
providers will be responsible for the
construction, deployment and operation of
tools and components of the CRIX service.
The providers will also ensure that the
development and service provisions meet the

compliance requirements that are outlined for
them. The provision of these services is
provided to the community of users via a
contract which will provide the platform for
the service level agreements and other
activities. CRIX International would select
one or more service delivery providers to
deliver these services to the CRIX community
and would contract with these service
delivery providers providing structure,
scope, governance for their activities.
These service delivery providers must be
independent from CRIX International and can
either be for-profit or not-for-profit
organizations.
One of the key recommendations
from industry is to select a single prime
contractor to act as a service delivery
provider. This service delivery provider
would then subcontract with the entities
providing the specific subcomponents to
NEAL R. GROSS
comprise the CRIX services. The service
delivery provider subcontracts with specialty
vendors, then provide the key project
activities. Thus each subcomponent can be
supplied by best-of-breed groups without
burdening the CRIX community with the need to
identify and contract individually with all
of them. This includes software development,
deployment, hosting, support services and
others.
Industry regards a prime
contractor as the most efficient way to
provide the CRIX collection of service
modules. This also ensures that the
interactions between those providing the
specific services to the community are
consistent and flexible, and also allows new
services and/or service providers to be added
to the process without disrupting or adding
complexity to the users of the CRIX services.
Additionally, consistency and synergy can be
realized by providing shared help desk,
software support and other shared services

for all of the tools and capabilities that
are provided to the CRIX community. In this
simplified contracting and funding approach,
all the users of the CRIX services would
execute a single contract with the prime
contractor for each of the desired modules
which additionally provides for a single
channel for the collection of service fees.
A single point of contact also provides a
mechanism for single, well-defined issue
escalation and dispute resolution mechanisms
and provides for flexibility in resolving
such issues quickly and effectively.
The funding model for CRIX
International would depend on a set of
initial contributions from industry founders.
These contributions would provide the
foundation on which the initial organization
would be formed and then service fees for
usage would provide an ongoing revenue stream
to support module operation support and
NEAL R. GROSS
maintenance activities. These service fees
would go to CRIX International and would be
based on the levels of usage and an
organization's ability to pay. A CRIX user's
ability to pay would be measured by the
firm's global annual sales where firms with
higher sales are expected to pay more for
services. The annual fees for founders and
non-founders would be set in relation to the
projected market share in order to minimize
the capital investment requirements and
achieve a self-sustaining cash flow by the
third full year of operations. This will
ensure that the services provided by CRIX
International are available to all
organizations regardless of their size and
stature. As the CRIX organization reaches a
self-sustaining level, the fees can be
reduced or excess funds may be directed
towards the development of new services. The
process of selecting commercial suppliers for
the prime contractor and for the
subcontractors would be done via standard

competitive award process which will ensure a
fair, open and objective review and selection
of vendors in all the roles. This ensures
maximum fairness in the process and the best
value to the users of CRIX services.
Through the proposed process the
community involved in all the process and
phases of the new module development, both
that carried out internally and externally to
the CRIX International. From a governance
perspective, once the prime contractor is in
place, industry recommends establishing an
advisory board to ensure that the explicit
implementation meets the needs of the
customers as well as CRIX International.
This would provide management oversight and
leadership connecting the community with CRIX
activities. To facilitate this, it is
proposed that the CRIX International will
oversee the selection of modules, the service
modules, in an inclusive manner that ensures
NEAL R. GROSS
that appropriate business analysis,
incubation and development take place. This
provides a mechanism for successful, cost-
effective development and deployment that
meets the customer's needs and adheres to
regulatory requirements.
So in summary, the proposed
structure yields several advantages. Third
party oversight ensures vendor neutrality and
cost efficiency. Secondly, single service
delivery provider reduces the contractual
complexities and it mitigates the risks
associated with coordinating multiple
vendors. Thirdly, the approach promotes an
open dialogue and involvement that is
inclusive of the CRIX community stakeholders
in ensuring that all interests are met.
Finally, the investment review, incubation
and development oversight ensures service
module selection is technically and fiscally
sound.
Thank you for allowing me the
opportunity to speak to you today. My

colleagues and I would be pleased to field
any questions the panel may have. And I'll
ask Deb up. Thank you.
MS. BREMER: I thank you all and I
will open it to the panel for questions. Dr.
Mittleman?
DR. MITTLEMAN: I'd like you to
speak a little bit to how you see the role of
government in this. I mean, I think there
are a lot of enormous strengths that can be
brought by having a lot of aspects of the
private sector be involved. And how the
research community and the FDA interface
really will work here and how they can both
provide scientific input, expressions of
needs and opportunities with the private
sector folks is a little less clear to me
given the governance structure you're
describing.
MS. BREMER: Sure. Well first of
all, I think the first thing we want to say
NEAL R. GROSS
is we think it's absolutely necessary that
the government agencies - thank you, Diana.
I appreciate that. My tall friend. But that
government agencies, particularly the FDA,
are a part of this and so I would say there
are a couple of different ways. One is
formal participation in the governance
structure. The bylaws do call for a board of
directors that has I believe up to four seats
available for government agencies. And
again, we would foresee that certainly one of
those could be the FDA, perhaps NCI as well
and others. Secondly, we would also be
looking for active participation in the
exchange service itself. And I think most
people feel, you know, in terms of lessons
learned and things like that, if that is not
a part of the model going forward we would be
less than successful. And I think that
active participation starts very soon
building on the foundation of the memorandum
of understanding between FDA and NCI with
FIREBIRD in particular. And we hope that we

could kind of exploit that, expand that and
make FIREBIRD available for commercial usage
as well.
MS. PAUL: I also think it's
critical to the public that we have
governmental involvement in this project, I
think particularly since it's going to
probably involve sensitive data. Having the
government watchdogs on the various
committees is I think important. It will
make the project much more palatable to the
public.
DR. MITTLEMAN: To extend that
just a little bit, have you also thought
about other governmental agencies, EMEA, et
cetera, and how you want to interface with
them?
MS. BREMER: Yes, we have. So as
it's called CRIX International, the emphasis
is to be able to ultimately operate in a
global environment. And I think it was Mark
NEAL R. GROSS
that talked to the concept of advisory
councils and advisory boards. And our
thoughts are that essentially we would have
regional - over time we'd have regional
advisory councils so they're, you know, the
first would be what we have here in the U.S.
and then as that broadens we would foresee,
for example, the European arena having an
advisory council. Those advisory councils
also have board representation, one from each
council.
DR. WOODCOCK: Dr. Lutter.
DR. LUTTER: Earlier speakers used
the words "mandate" and "incentives" and I
haven't heard either word used in response to
the question what role for government. So
maybe I could ask you are you opposed to
mandates or incentives, or is there some role
for such actions by FDA or other federal
entities?
MS. BREMER: I think mandates will
be on a case-by-case basis depending on the
service that we're talking about. There will

be some services that in order for them to be
successful I think it'll you know, it'll
become intuitively obvious that it would have
to be a mandate in order for it to achieve
its outcomes. Others will you know not have
those same characteristics. In terms of
incentives, I think the incentives are
essentially the benefits that we've talked
about that are to be gained from something
like this. For example, lowering the
development cost for you for independent
entities, or lowering the development - the
review costs from a regulatory agency kind of
perspective. So besides the incentives in
terms of speeding drugs to patients which is
something that has you know in our hearts
also feels good, there are the financial
incentives to the folks that are involved.
DR. WOODCOCK: Other questions.
Ken Buetow.
DR. BUETOW: I assume it's safe to
NEAL R. GROSS
assume you guys are interested in third party
solutions? You wouldn't be opposed to that?
MS. BREMER: I think you could
assume that.
DR. BUETOW: I'm intrigued what
you all see as the relationship of an entity
such as this to the existing standards
development organizations and how this group
could, would play in the further generation
and propagation of those standards.
MS. BREMER: Bob, if you're
comfortable.
DR. BECK: I see us as an umbrella
organization that serves as a very large tent
under which companies, academic institutions,
the public and government can essentially
coalesce. And so standards bodies, the
governance structure will be reviewing their
work and basically we will provide a stimulus
I think to standards development. I can't
see any other role for an organization like
this to be other than a stimulant, not an -
and maybe, you know, reciprocal participant

on some bodies, but not actually doing the
standards development themselves.
MS. DUBMAN: Just to echo that, I
think you know we're going to try to leverage
whatever is out there and not reinvent the
wheel. And so we are using for example CABIG
guidelines in developing solutions so that
they're interoperable. And you know, I think
we're working you know closely with CDISC and
ICH and other organizations to make sure that
we're not, you know, again reinventing the
wheel.
MR. RUGGLES: One additional quick
comment and that's, you know, when we're
talking about some of these additional seats,
I think it's envisioned that standards
organizations like CDISC and others would
have opportunities to actually be on the
governance board of the organization. So to
make concrete some of these recommendations
they'd participate.
NEAL R. GROSS
DR. WOODCOCK: Ken, do you have
more? No. Okay, Armando?
DR. OLIVA: I recognize that CRIX
International just came into being this
month, so maybe this is an unfair question.
But there are many stakeholders that
obviously would like to see CRIX succeed and
to borrow a phrase from someone else on the
panel, why not sooner. And my question is
this. My real question is this. If a
stakeholder sees the need and maybe even has
the resources to develop a service, a shared
service, what's your advice on how that
stakeholder should proceed at this point?
MS. BREMER: So first of all we
welcome stakeholders that see a need and are
willing to get something started, to incubate
it so to speak, and FIREBIRD is an excellent
example of essentially that kind of a
concept. So we don't discourage it.
Certainly though it would be very helpful the
more stakeholders at large have an interest
in something, the better the model is going

to be in terms of usage. So if there was,
you know, a single stakeholder that wanted
something, but there wasn't broad interest in
it, it may not be a viable long-term model.
So there's a variety of different things that
we'd use to determine the order of services
to be brought into the CRIX environment. And
certainly the broad stakeholder community
represented in a large way through the
advisory councils, in a very focused way
through the advisory councils and through
other input will help advise the board of
directors in terms of what are the
prioritizations so to speak of the next
services to be provided. But we certainly -
we don't discourage that. We welcome that.
DR. OLIVA: So is there a process
or is there a mechanism right now where new
services can be proposed or discussed and
entertained for prioritization?
MS. BREMER: Back to your point of
NEAL R. GROSS
just being formed formally as of about 11
days ago. I would be remiss if I said that
there was an active process. To date what we
have are the startings of the board of
directors with the founding members and our
next task is to round out the board of
directors, get some officers in place and
then start working on service providers. But
in parallel to that we will obviously be
working on processes. As soon as we get a
few more people onboard that is certainly
part of our plans.
MS. DUBMAN: At the last CRIX
meeting we actually did discuss an additional
new service to bring forward for CRIX and
actually the discussion actually ended up
talking about what is the process. I think
we will get those processes in place quickly
now.
DR. WOODCOCK: Dr. Buetow.
DR. BUETOW: So to complement the
question on services, there's obviously the
interest in the prioritization of what

services come onboard, but what about - I
think from previous speakers there was
discussion about also, concern about access
to services and what would be in place to
guarantee that - especially it sounds like as
this is proposed there might be - "monopoly"
is the wrong word, but perhaps singular
source of services. How would you guarantee
that services could be provided to all of the
stakeholders in a timely fashion and that
it's not a few 800-pound gorillas that
consume all of the resources and then smaller
players are actually left on the sidelines?
MS. BREMER: So first of all I
would say again, substantial emphasis will be
placed on what the advisory council and the
CRIX community at large recommends. I can't
repeat enough, and I do represent
individually one of those big 800-pound
gorillas, but I can't emphasize enough that
we do not see ourselves being successful in
NEAL R. GROSS
this not-for-profit kind of entity, in this
overall shared infrastructure approach,
unless there's active participation by all
stakeholders. So it is not in the big
gorilla's best interest if we do things that
preclude folks, you know, that aren't those
big gorillas from participating.
MS. MCKENZIE: And I'd also like
to add that as we think about the overarching
governance body, our expectation is that we
have government representatives on that body
I think first and foremost. And I also think
that in forming CRIX International our goal
is to make sure that there's an oversight
body that's looking out for the interests of
the community. And one of the expectations
that's either outlined in one of the slides
or in the executive summary is that we expect
to put pricing in place that is actually
tiered based on the size of the different
organizations that would want to take
advantage of the services.
MS. PAUL: And the governance

structure includes patient advocates at every
level including on the advisory board. And I
and most of the people I know who are patient
advocates take that very seriously and I
think Ken you know we will be there watching,
so.
DR. WOODCOCK: Other questions
from the panel? Dr. Lutter.
DR. LUTTER: The services that you
offer would presumably be for sale at cost
because you're a not-for-profit. Would they
be available to any interested party, or only
people who are in some sense members or
subscribers to CRIX?
MS. BREMER: We see the services
as being offered to, as the bylaws point out,
all members of the CRIX community. And there
are different kinds of members of the CRIX
community, from founding members as we talked
about, you know, are onboard today, to
associate members. And the associate members
NEAL R. GROSS
are essentially the users of the service. So
there is a formality to saying, not a
bureaucratic formality, but there is an
active choice of being a part of the service
and paying fees, again, commensurate with
ability to pay so to speak. So there is, you
know, an active choice. I'm not sure if I've
exactly answered your question, Randy.
DR. LUTTER: For the non-associate
members, they would have no access without
becoming associate members?
MS. BREMER: That's correct.
DR. WOODCOCK: Dr. Buetow.
DR. BUETOW: In terms of risk
mitigation, I'm curious. This is, again, a
relatively large centralized activity. What
are the strategies that actually mitigate
risk with respect to the availability of
these services if you all decide at some
point in time that you don't like this?
What's the role of open source? What's the
role of open standards and other components
in what you all would develop? Would these

services, infrastructure and other components
be able to be run by other people besides
this so-designated entity?
MS. BREMER: Certainly. If you
think about the prospect of we are looking
for initially a core service provider, again,
we would be giving that service provider
oversight in terms of the order of priorities
of services and things along those lines, but
they would be essentially running the day-to-
day operations, providing the user support,
et cetera. So if the, you know, although we
don't anticipate - we anticipate this being,
that we're all in for the long haul, but if
that should happen, the core service
provider, if the company chose to do so,
could potentially keep things going.
DR. BUETOW: So would the services
definition be in the public domain? Would
there be a plan to share the services
definition as part of this public domain?
NEAL R. GROSS
MS. BREMER: Absolutely. Yes.
DR. WOODCOCK: I have a question.
I found the presentation from Liaison
Technologies very interesting. My question
is this set of services involves to me
something beyond what you presented as types
of services that were mainly for industrial
partners. It involves health, it involves
patients, it involves healthcare providers
and so forth. It seems to add some extra
dimensions. Have you seen any models where
there are many stakeholders with major stake
beyond the industrial partners in the
exchange? Maybe the financial networks, for
example, where they're all moving their money
around?
MS. BREMER: Dan, do you want to
come to the microphone?
MR. RUGGLES: Yes, the financial
industry, VISA is an excellent example on the
security side of it, and that's probably one
of the better examples out there. We do buy
and sell and keep track of things in that

regard, and in terms of confidentiality, in
some cases that information has to be
encrypted. So there is some similarity, but
nowhere near some of the things that you've
talked about or discussed here today.
DR. WOODCOCK: Complexity. It
strikes me those financial networks for
example, there are users such as me. I mean,
I'm just, I'm not really a user because I'm
always using some industrial partner's
whatever, setup, but is that how it would be
envisioned here, that a lot of healthcare
types and the patients and so forth wouldn't
have to be members of the CRIX community,
they would simply be users of this network on
behalf of the entities that set it up?
MR. ADAMS: So we create, as an
example, we create services as a solution.
And one of those services could be very much
the way you do your banking, as an example.
So I'm trying to use it as an illustration as
NEAL R. GROSS
opposed to I don't want to design or
architect anything. But yes, it could be a
web-based sort of service that's going back
to the pharmaceuticals also feed information
to security. Kind of keeps things separated
as to what you can look at and what you can
do.
DR. WOODCOCK: Thank you. Are
there any other questions? Armando.
DR. OLIVA: Yes, I'd like to probe
a little bit this whole idea about
competition. Using the VISA example. As a
user of VISA, I can shop around for the best
rate or the lowest annual fee or the perks.
And how would that - how do you see that
level of competition? How would that work in
using the CRIX model?
MR. RUGGLES: I think the key way
in which that would work, and one of the
reasons that the organization was sort of
designed with this multiple tier aspect to it
is such that the stakeholder is participating
in it, and that includes the government

representatives. The representatives broadly
of the individual stakeholders then initiate
a competitive and an ongoing competitive
process at two layers. One for the general
overall service provider who's going to act
as sort of the prime contractor, but also
through that to the folks providing services
up through that process. So there's two
layers essentially of traditional competition
going on to ensure that both the quality of
the service meets the requirements of those
stakeholders. Maybe unlike many companies,
the stakeholders themselves are those to whom
the organization is beholden, not you know
shareholders out in the world. But those
stakeholders then initiate that competition
that can be ongoing. So essentially you're
getting competition for the services that are
then provided more or less through the
central mechanism. So it's not quite the
same where you could go to any of several
NEAL R. GROSS
shareholders any time, but there's
competition for those roles and that's
ongoing and controlled by those who are using
the services.
DR. BECK: Two other thoughts
along that line. First, FIREBIRD itself is
an open source solution. So there's nothing
that would preclude somebody out there in the
community, in the wider community, in the
non-CRIX community from creating or modifying
or improving FIREBIRD and providing that as
an alternative to this entire enterprise.
Secondly, there's nothing in this enterprise
that would preclude the agreement to host
more than one service provider for a
particular service just like VISA versus
MasterCard at the level below - at the
operating level. So there's no reason why,
especially if we focus on some of the things
we've learned through CABIG and other
initiatives, a bias toward developing
prototypes in open source and then taking
them to the point where they're ready to be

supported or taken out. There's no reason
why there couldn't be more than one solution
at any level.
DR. WOODCOCK: We'll have to - go
ahead, and then we'll have to wrap this up.
MS. MCKENZIE: Just one more
point. I think one of the things that's made
the financial services model so successful is
the adherence to standards. And I think in
this case competition becomes better the more
they adhere to standards. It makes it easier
for them to come and fit into the service so
actually it creates just a nice momentum that
we're looking for in our industry for
adoption.
DR. WOODCOCK: All right. We
thank the entire CRIX community as
represented here for your presentations. Our
next two speakers are Ed Tripp, Program
Director for eSubmissions at Abbott
Laboratories and Bill Rosen, Executive
NEAL R. GROSS
Director, e-Health Policy and Standards,
Pfizer Global R&D.
MR. TRIPP: I feel a little bit
like following a production of A Chorus Line.
Dr. Woodcock, distinguished panelists, ladies
and gentlemen, my colleague Bill Rosen and I
would like to thank you for the opportunity
to present on behalf of the Pharmaceutical
Research and Manufacturers of America on the
subject of electronic submissions and public-
private partnerships. First I will address
electronic submissions. Then I'll turn the
podium over to my colleague Bill and he will
speak and address public-private
partnerships. The member companies of PhRMA
spent nearly $40 billion on pharmaceutical
research last year and with such a massive
investment we have been long been advocates
of data and information automation that can
lead to improvements in healthcare through
better access to information, and that in
itself could lead to better decision-making.
We wanted to make our position

clear. We fully endorse a plan to move to an
all-electronic submissions environment and to
that end we've partnered with FDA to
establish electronic capability for
individual case safety reporting, structured
product labeling, annotated ECGs and other
standard data and exchange formats. Our
reach goes to working with ICH, CDISC and
HL7. We've got broad leadership involvement
with the ECTD working group, the SPL working
group and today with the regulated product
submission, both the leadership and testing
teams. And our organization, the members are
implementers and we want to both identify our
concerns and add our voice as proponents of
the public-private partnership concept.
So as we look at some of our top
goals on electronic submissions, the first
goal deals with developing end-to-end
electronic environments. And when we talk
about end-to-end, we're talking about dealing
NEAL R. GROSS
with process and information flow upstream
and downstream of submissions. I want to
emphasize that a common environment
facilitates information sharing and transfer
of information all the way from discovery to
product retirement. So its scope needs to
encompass all transactions throughout the
product lifecycle. As Dr. Edwards pointed
out, we need to be looking at advertising
information as well as submission data. It
needs to be the entire scope of the process.
Goal number two goes to harmonized
processes and data that increase data access
and dissemination, and that leads to better
analysis. Critical to the success, we
believe having a long-term plan with
sufficient detail in the first 18 - 24 months
and establish long-range goals running out
over a rolling five years will help
facilitate the right activities and speed
adoption. The plan or roadmap needs to lay
out the key milestones for each initiative
that will in the end lead to an end-to-end

electronic system.
The most effective standards are
broadly harmonized. We were just talking
about electronic banking. Imagine what
electronic banking would be like if national
boundaries and regional jurisdictions
dictated that they use different standards.
We would never move money out of this country
and yet that's exactly what we're doing in
the healthcare industry.
So, how would we benefit by an
electronic submissions environment? That
question depends on how it's implemented and
it assumes the assumption of effective
standards. And by "effective standards" we
mean that they're harmonized and well
established or stable. And the focus then
can be directed at the science and at
information access and retrieval, and that
information access and retrieval will exceed
our best expectations. We only have to look
NEAL R. GROSS
to the Worldwide Web to see how
standardization of data and information can
increase our access and our ability to react.
Harmonized standards promote
consistent processes and procedures, reduce
redundancy. Standardization of clinical and
safety information and its transmission in a
globally harmonized format lays the
foundation for a future vision of a global
electronic health record. And from my
perspective that needs to be the target in
that arena as well, not a U.S. electronic
health record, but a global electronic health
record. I do not want to go to Germany and
find that my health record doesn't do me any
good there.
Now the use of electronic
submissions requires standards, but for
standards to be truly effective they must be
harmonized to achieve interoperability. If
we drive to globally harmonize standards, we
enable the use of the information not only
for marketing reviews, but for many purposes

across the healthcare domain. And working to
develop standards and processes for an end-
to-end electronic environment should be
viewed broader than an FDA objective. All
stakeholders need to be engaged to maximize
the use of the data and the information for
improvement of patient healthcare.
To the question of wouldn't all-
electronic submission environment change your
ability to initiate in a timely manner the
studies supporting your regulatory
submission. Well, from our perspective the
question needs to be re-worded slightly. Can
a sponsor improve the conduct and analysis of
a clinical trial with an all-electronic
submissions environment? And the answer is
undoubtedly yes, but. It needs to be done
correctly. Imposing standards on legacy
processes designed to utilize paper will not
speed up study startup. We need to evaluate
and redesign the process as well as develop
NEAL R. GROSS
the standard to achieve a more efficient
overall process. We need to extensively
pilot the standard and the processes and
allow sufficient time for tools to be
developed and to mature. However, the impact
of an improved harmonized electronic standard
that supports more efficient business
processes for data collection, monitoring,
review and analysis will likely offer more
dramatic gains and speed and cost reductions
for both the industry and the agency. If you
think about it, you wouldn't look at an old
process that required triplicate NCR paper -
and we all remember that I think. Most
everybody in the room remembers that - and
automate it by having your printer
automatically spit out three copies. We re-
engineered the process. We shouldn't combine
our efforts to the development of data
standards and automating the paper processes.
We need to leverage the automation by
redefining those processes.
Not having a clearly stated

compelling business case for electronic
submissions or any of the components of
electronic submissions hinders adoption. And
this - we've gone through the discussion of
mandate, no mandate, incentives. The bottom
line is there needs for each component of an
electronic submission system there needs to
be a clearly stated business case. That may
be the gun's at your head, this is a mandate.
It may be that there's an incentive. It may
be here's a rationale and a way to look at
it, and you're going to save hundreds of
thousands of dollars if you adopt this. But
it needs to be clear and stated up front or
we get this fuzzy participation. There's
some companies that want to be on the leading
edge. There's some companies that want to be
on the trailing edge.
The benefits can be realized with
the comprehensive long-range plan where every
one of those initiatives has a compelling
NEAL R. GROSS
business case and folks can see where the end
game is and plan where along the path that
they want to join in the process. Standards
churn, or the movement from version to
version, increases cost of implementation and
results in continual reinvestment. And we
have many players who sit on the sideline
waiting for the dust to settle before they
jump on. You only have to look at the
history on SPL to know that's the case. We
still have some folks sitting on the sideline
waiting through their first submission on SPL
because that standard has churned and turned
over a number of times.
A major impediment is a mixed
environment with too many options. Andrew
Tanenbaum, who's a Professor of Computer
Science in Amsterdam and the father of
structured programming, said the nice thing
about standards is that there's so many to
choose from. Well, we don't think that's a
very nice thing when it comes to trying to
move to a new environment and we believe that

the notice to withdraw the ENDA guidance is a
positive step.
Are there enough entities to
support the effort? It really goes back to a
question of churn. Lack of a long-term
roadmap leads to confusion and that combined
with instability of standards delays vendors
from entering the market and causes industry
to take a wait-and-see attitude. So if
there's a lack of testing before introducing
a standard that can erode its value. And
although the time to develop a well-tested
stable standard is longer, the overall cost
to implement ends up being much less. Faster
is not always better and through piloting
standards before implementation we can obtain
stable services and tools and actually get
faster adoption and lower cost.
Additional costs associated with
implementing a particular format of standard.
Best guess is the move from ENDA to ECTD is
NEAL R. GROSS
or will be more costly than the move from
ECTD to RPS, but that in part depends on how
good a job we do testing and developing the
RPS standard and the processes around it.
There are also hidden costs dealing with
behavioral change and that happens both at
the agency and the sponsor. We still get
requests from reviewers to provide paper.
That sends kind of a funny message when we're
trying to move our organizations along to an
e-environment. And our scientists are
reluctant to move from a paper environment
because they have to learn to evaluate not
just the content, but the bookmarking of
documents, the hyperlinking of documents.
The analogy is everybody in this room can
pull out their PC and type up a several-page
report and print it out. How many of you can
turn it into a webpage? And that's what we
ask folks to do when we create an electronic
submission. So there are costs associated
with finding the right resources. There is a
shift of skill set. There are costs in

modifying document management systems to
accommodate changes in structure and meta
data every time we shift a standard. And
then there's costs of new publishing tools
because not all - we can't count on the tools
that we have for one standard being developed
and made available in the right timeframe for
a new standard.
Are there parts more costly to
convert? And the answer is yes. I think we
heard this also from some other speakers
earlier today. Any change in electronic
format that ripples downstream into our
internal data collection processes actually
has an enormous cost associated with that
change. There are, at least for mid-sized to
large companies, there's a substantial
investment in data collection systems, and
when the format changes, there's a large
reinvestment to change those systems.
Sometimes we get lucky and it happens to hit
NEAL R. GROSS
at a time where we're looking to upgrade
those systems anyway and other times it's
totally unexpected. Once again, a well-laid
out 5-year plan will help us better stage
that kind of investment, but it is possible
for the fully implemented change to be more
efficient than the previous environment. And
that requires all parties to be involved in
the redesign.
How much time? The answer is not
a number. You've got to have the right
processes in place regardless of the time it
takes. We learned a number of lessons
implementing SPL and one of them was that the
lack of scenario-based development and
testing led to frequent changes and
additional complexity. You need to crawl
before you walk and walk before you run and
that means engaging all the stakeholders in
developing the new process, then develop the
standard, do scenario-based testing, pilot
it, be comfortable with the results of the
pilot and finally document the implementation

requirements well, then take it into
production. And if we do that, the uptake
will be faster and a lot less problematic.
Problems arise when we skip these steps to
compress our timeframes or when legacy
methods for exchange aren't withdrawn. And I
want to thank the panel again for the
opportunity to address them, and now I'll
turn the podium over to Bill Rosen.
MR. ROSEN: I too would like to
thank the panel. I'm vertically challenged
here. I too would like to thank the panel
for giving us the opportunity to speak today.
I'd also like to thank the multitude of you
who bothered to get up this morning and get
down and get into line as early as 7 o'clock,
and I hope that the turnout will show the FDA
and the NIH the interest that the topics that
we're talking about today have for the user
community. I'd also like to encourage that
community to please get your comments in by
NEAL R. GROSS
the deadline and judging by the number of you
that are out there this morning, I hope that
there will be plenty of them.
Now, last night on the way down
here, I don't have a slide for this one
folks, but on the way down here I happened to
start thinking about the history of e-
submissions and some of the beginnings of the
public-private partnership. Now, the good
news is that I can remember it at all. The
bad news is that I do remember it. Going
back to 1984 Research Data Corporation and
Abbott collaborated to submit the first
CANDA, or Computer-Assisted New Drug
Application to the FDA on a VAC server, if
any of you can remember what a VAC server is,
that was not located on FDA property but
rather was located at RDC. Unknowingly, they
delivered the first electronic submission via
a trusted third party entity. During
the late `80s NDAs were delivered
electronically at the FDA via something
called ONDAs, or optical NDAs that consisted

of scanned documents that were not easily
searchable as you can imagine, but they were
in electronic format. In the early to mid-
`90s CANDAs multiplied proliferously but
without standardization within our cross-
sponsors, the scope, features, hardware,
software, they all varied wildly. Sponsors
paid for one-off development. Agency
reviewers had to learn a new system each time
a new application was submitted. The entire
process was not efficient. There was a sense
of frustration on the part of both sponsors
and the agency due to the lack of standards.
However, there was agreement that electronic
submissions could increase access to
information, speed access to information.
These early CANDAs could not be leveraged,
however, on an international basis.
In 1992, the initial FDA CANDA
guidance manual was published. During 1994 -
1995 the Submission Management and Review
NEAL R. GROSS
Tracking or SMART project was established by
the FDA. The goal was to provide standards
and a system for industry to submit text,
data and image files in electronic format.
In the mid-`90s Document had emerged as the
de facto document management standard while
Adobe PDF emerged as the de facto output
standard and SAS transport format became the
de facto format for submitting data
electronically. In 1997 the Aris pilot for
e-safety reporting was initiated and the FDA
also drafted a guidance for e-submissions.
In 1998 ICH, the International Conference on
Harmonization, began to focus on a means to
create and transport an e-submission that met
the definitions of the common technical
document.
In 1999 the e-submissions guidance
for CDER was approved in January and for CBER
in November. And by 2000 a brokerage site
was presented at the DIA annual meeting and
PhRMA began discussions to consider something
referred to as InfoBroker which was intended

to establish a shared environment via a
trusted third party. In 2001 FDA speakers
mentioned the innovative use of third party
repository at the DIA annual meeting and
PhRMA developed a white paper titled Issues
in Developing and Implementing an InfoBroker
for the Biopharmaceutical Industry and
Regulatory Authorities. Also in 2001 of note
over 70 percent of the NDAs included at least
some electronic component.
In the spring of 2001 the ECTD or
electronic common technical document
specification was published, establishing the
format for a globally harmonized e-
submission, quote unquote, or the uncommon
technical document. By 2002 PhRMA began an
effort to build the concept then known as
SEBIX, the Secure Electronic
Biopharmaceutical Information Exchange that
was mentioned earlier by my colleague Deb
Bremer. In 2004 all knowledge harvested as a
NEAL R. GROSS
result of SEBIX was transferred to the
National Cancer Institute for further
development of the concept and we thank you.
You will notice during this brief history I
have used the words "brokered site" "Info
Broker" "shared infrastructure" "shared
environment" and "SEBIX"
anonymouslysynonymously for public-private
partnership. Now so that I don't step on my
tongue throughout the remainder of this
presentation I'm going to refer to "public-
private partnership" simply as P3. Thank
you.
In today's standards environment,
it is important the terminology is
standardized as well as data content and data
exchange standards. We recognize that there
are many definitions for the meaning of
"public-private partnership" or P3.
Minimally, PhRMA believes public-private
partnership is defined as a secure
sustainable shared infrastructure that will
provide services across the biopharmaceutical

and healthcare communities. For any public-
private partnership or P3, security and
protection of intellectual property is of
vital importance. As a result, efforts must
be taken to secure the infrastructure and
ongoing testing is necessary to assure the
effectiveness of that security. Independent
audits should be considered for this purpose.
Regarding general viewpoints on a
third party entity or entities providing
services, PhRMA supports the creation of a P3
to provide a shared infrastructure for
information exchange. PhRMA also recognizes
that the FDA has jurisdiction over other
products such as devices, foods and animal
health products. We believe this common
shared infrastructure should also be scalable
to provide services to these other industries
as well, achieving economies of scale. At a
high level the P3 must implement a common
standards-based electronic infrastructure not
NEAL R. GROSS
only for regulatory data and document
submissions but also for information review
and analysis.
To minimize any preferential
treatment we prefer that oversight of the P3
infrastructure be a not-for-profit
organization. The not-for-profit
organization should be empowered to establish
contracts with for-profit companies to
provide appropriate infrastructure and
related services. The body that oversees the
P3 would then have the authority to establish
multiple applications running on top of the
infrastructure in a profit or not-for-profit
manner.
PhRMA advocates for a pragmatic
approach where P3 has grown organically,
adding new services at a speed that P3 can
manage. Logistically, backup and recovery
are essential as is maintenance for both the
software and hardware. P3 must assure 24x7
service.
Some random thoughts about

benefits. P3 has potential to speed safe
effective medicines to patients. The focus
of pharmaceutical research should be on
health sciences, not on managing and
maintaining technology infrastructure that
supports delivery of electronic information.
P3 in this respect is an enabler. A shared
electronic information management platform
therefore targets redundancies across our
industry by providing a single
infrastructure.
We believe the business case for
each new application differs across industry,
health authorities and other potential users.
This has to be noted as new applications and
new ideas come up underneath the common
infrastructure. We recognize, however, that
for the P3 to be effective it must be
affordable for all, allowing for broad
participation. PhRMA envisions the P3 can
level the playing field for small to medium-
NEAL R. GROSS
sized companies, giving them access to state-
of-the-art technology at reasonable costs.
This will eliminate barriers to process
innovation and speed adoption of standards.
The spectrum of participation
includes not only large and small pharma
companies, but also CROs, healthcare
providers, as well as others in other
regulated industries. P3 will also
facilitate two-way communication between
sponsors, their business partners and health
authorities. It also facilitates a truly
paperless submission.
It's all about information and the
shared environment can serve as a single
point of reference for participants,
permitting sponsors and health authorities to
see the same information the same way using
common tools. I must re-emphasize the need
for security, security of both the
intellectual property that might reside on a
shared infrastructure and security that
guarantees that patient-related information

is managed appropriately.
Personal health records,
electronic health records and electronic data
captured for clinical research purposes can
be integrated at a high level, providing the
right information at the right time in the
right format in the most efficient way
possible to benefit all parties involved,
providing the standards are there to be used.
Proliferating multiple P3s will not allow for
efficient cross-referencing across
submissions, nor will it allow for efficient
information lifecycle management. The
implementation of standards must be focused
on the information itself rather than an
electronic information exchange platform.
However, interoperability of standards that
enable information exchange once again I must
state is imperative.
Some potential barriers. For P3
to be successful there has to be broad
NEAL R. GROSS
participation. I mentioned this once before.
Several other people have mentioned it as
well. Further, we recognize the potential of
a P3 across not only the pharmaceutical
industry but across the broader healthcare
community. If P3 is overly conservative as
it develops, the scope may be restrictive,
resulting in an effort that will have a
limited lifespan. Conversely, if the P3 is
overly aggressive it may spin out of control
with minimal success, also with a limited
lifespan.
We have a need for a long-term
electronic archive. Some products are on the
market a long, long time. I believe this was
mentioned by Dr. Edwards earlier this
morning. If we take for instance the case of
Dilantin which was first approved in 1938 and
it is still being marketed today, imagine if
that was approved today and was still being
marketed 70 - 80 years from now. Are we
going to be able to dig up the electronic
records for that and possibly is this a

service that the P3 can play for us, as a
long-term electronic archive?
Other potential barriers include a
lack of clarity as to the FDA's willingness
or ability to participate in a P3 and this
may delay or discourage full participation.
This has also been mentioned earlier. Others
may simply choose to follow rather than lead,
staying on the sidelines waiting for FDA and
others to engage.
Some conclusions. Efficiencies
cannot come at the expense of increased
complexity and costs on the side of the
pharmaceutical industry. We should all be
looking for win-win scenarios where
information can be presented and preserved in
the most effective and most efficient and
cost-effective manner possible. The agency
shares the responsibility for processes they
impose on the industry they regulate and how
those processes contribute to the overall
NEAL R. GROSS
cost of medicine. We should all endeavor for
an optimized process that creates, develops
and implements globally harmonized standards
that increase the likelihood of success for
shared information exchange. For the
pharmaceutical and healthcare industries, we
must establish a long-range plan for managing
data, thereby creating a true information
lifecycle. Long-range, the P3 should be
inclusive of information generated from
discovery through product retirement with
integration of submissions across the drug
development continuum. PhRMA recognizes the
value proposition of P3 and that it varies
from one region to another, but extra efforts
should be made to establish globally
harmonized standards. Standards in the use
of public-private partnerships can speed safe
and effective new therapies to patients
everywhere.
Once again I would like to express
my gratitude to the NIH and the FDA for
allowing us to have time today to express our

opinions. I would also look forward to
working with the FDA and the NIH and the
research community on electronic submission
standards and any P3 effort. Thanks.
much. We'll have questions now from the
panel. Dr. Mittleman.
DR. MITTLEMAN: I was wondering
what PhRMA's view on the presentation that
was made by CRIX and particularly the CRIX
International kind of model, how does PhRMA
react to that?
MR. ROSEN: I may not be the right
one to answer the question. I've been
involved in this concept since 2001. I was a
member of the group that worked on SEBIX and
I was one of the people who went over to the
NIH to do a data dump back in 2004. PhRMA
supports the concept of CRIX although I would
also have to be negligent if I didn't say
that that is not universal across the 40-some
NEAL R. GROSS
odd companies that are members at PhRMA.
Anything to add, Ed?
MR. TRIPP: No.
DR. WOODCOCK: Dr. Levin?
DR. LEVIN: Ed, so it sounds like
there is a business case for electronic
submissions.
MR. TRIPP: I believe there is. I
believe there is. I don't believe we've
clearly stated it or sold it well.
DR. LEVIN: But inside your
company there's a - you have a business case
for?
MR. TRIPP: Inside our company
we're reacting to the reactions of the FDA
and say if we don't do something, we're going
to have a problem. So it's more of a gun to
the head than a clearly stated business case.
DR. LEVIN: So in your company
that you're not developing your own business
case as far as this is going to be a benefit
for us internally to our company and waiting
for?
MR. TRIPP: No. In most cases it
is reactionary because the dollars need to
compete with other initiatives across the
company. And there isn't a clearly stated
and obvious business case that you can pull
forward. And so it tends to be sold -
investment tends to be sold as a matter of is
it a regulatory compliance issue. And if you
look at - we ran some numbers across the top
20 pharmas. If you look across the top 20
pharmas, over about the last eight years
they've invested a half a billion dollars in
building submissions systems and document
management systems that support that and
continue to invest. It's been a moving
target and it's getting harder and harder to
go make the ask.
DR. LEVIN: So that, so different
companies will be at different stages as far
as their business cases because some have
made a business case already that they want
NEAL R. GROSS
to go into the electronic without the FDA
mandating something.
MR. ROSEN: I think that's
correct. Indeed, if we take a look at how
processes are changed backward of the
submission process there are clearly cases
where document management for quite a long
time has been employed and the whole concept
of submission-ready now being carried forward
to the concept of archive-ready and the
business case that enables the management of
electronic information and also for knowledge
management as well.
DR. LEVIN: So it's - and this
goes back to questions that were asked
previously about what can the FDA do to move
forward. And it sounds like for some it's
the mandate will be the piece, but why don't
you address that?
MR. TRIPP: I think if we work
together on a collective long-range plan, if
there's a target out there of when we
collectively believe we should make the move

on any of the initiatives, then I think we
ought to also have candid discussions in part
around what is the business case and what's
the business case for Pfizer is not
necessarily the business case for Abbott
Laboratory. Everybody runs their business a
little differently. But you can put together
templates that state this is how it's being
used. Plug your own numbers in or plug your
own processes in and I think we've failed to
do that effectively in the past. And I think
in order to make things move faster it will
be helpful to do that.
DR. LEVIN: And when you say "we"
who?
MR. TRIPP: We collectively.
There's a group of us standing up here that
have been involved in this for awhile. Help
drive standards, help, you know, drive
electronic submissions and so forth and are
out in the forefront beating the drum, but we
NEAL R. GROSS
need a little banner to go along with the
drum, and that's the business case.
DR. LEVIN: And I just, and I
question about when you're talking about
staging the investment. So let's say you
decide there's a business case for whatever
reason. What's that process like and how
long does that take to move your investment
forward?
MR. TRIPP: Well, and again it's
probably different for every company, but for
us if we make a change to - if we decide to
make a change to our document management
system for example, not changing it but
modifying it, assuming it's not a real small
patch or fixed to a bug, but it's reacting to
a change in the standard, we start planning
for the investment in May of the year before
we start making it. So the May before the
January we want to start making the advance
we start planning, building up the numbers
we'd start in January, and generally by the
time you implement it and validate it, you're

talking September or later.
MR. ROSEN: Just one comment.
Randy, I think the timeframe differs
drastically with the size of organizations.
Smaller companies are apt to be able to move
more quickly, again possibly because they
don't have established systems and standards
internally whereas larger entities are apt to
have you know a greater problem moving and
being flexible to the change. I'd also have
to say that I think there's a difference
between making modifications to existing
systems and existing processes versus doing
business process change, or the re-
engineering of a process and absolutely
turning over, let's say for instance a
complete document management system. And
that would go for any size entity.
DR. WOODCOCK: Dr. Buetow?
DR. BUETOW: A slightly more
technical question. You spoke relatively
NEAL R. GROSS
forcefully against the concept of multiple
3Ps. I'm just curious in today's technical
universe of services-oriented architecture
and under the presumption of codified global
standards, can you further articulate why the
disadvantage of multiple partnerships as
opposed to a single centralized entity?
MR. ROSEN: Please don't
misinterpret what I said. I'm not against
multiple P3s. I think that there is reason
for multiple P3s. However, if you stop and
look at the examples that I've presented
about being able to link across a submission
lifecycle, there's obviously a problem if
that is located on different physical
entities. So I was merely pointing out some
of the drawbacks that may occur if this is
not planned and rolled out in an efficient
manner.
MR. TRIPP: One other thing to add
to that. As you look across a P3 concept and
where some of the savings are, some of the
savings and attractiveness if you have a

single disaster recovery plan, you have a
single backup and so that maintenance cost
gets multiplied depending on how many P3s.
DR. BUETOW: That may be in part
answer to sort of a follow-up question.
You've mentioned this and I just want
clarification of how tightly coupled the
service provider should be, or the end
service provider should be to the - from your
all's perspective to the 3P. Is the 3P -
where do you see the relationship in the
service providers? Are they loosely coupled?
Are they tightly coupled? Does the 3P drive
and determine who all the service providers
are?
MR. ROSEN: I would think that it
is necessitated by the board of directors of
say, for instance, CRIX in this instance, and
the community of stakeholders at CRIX to
ascertain what applications would be running
underneath the platform. I would also say
NEAL R. GROSS
that the quality of service, the timeliness
of service would be mandated by any contract
that would be formed between that entity to
put applications up underneath that
infrastructure. So I would hope for a
tighter relationship rather than a looser
one.
DR. WOODCOCK: One other question.
Go ahead.
DR. BUETOW: If I could just - and
we absolutely appreciate this concept of
concern of churn of standards. So how do you
all - do you all have a proposed path that we
actually standardize the standards? I mean,
this comes back to our incentivized versus --
MR. TRIPP: Well, I think we're
collectively learning together over time. I
think the HL7 draft standard for test use
model that's been adopted by RCRIM is a good
model. We need to have the patients patience
to let it play through and we also need to
make sure that get the stakeholders engaged.
And in some cases the stakeholders have other

things that are preventing them from getting
immediately engaged. I know we talked with
our colleagues in Europe about engaging in
some testing on RPS late third quarter, early
fourth quarter this year, and we got the
answer that they can't participate till first
quarter because of other things going on. We
can't throw up our hands and say, well, we
can't wait for you. We have to wait and let
the process mature if we're going to get a
valid standard out.
MR. ROSEN: One other comment.
I'd state that the CDISC ODM model and the
BRIDG model are helping I think to bring
about harmonization of standards across the
various communities.
MR. TRIPP: Yes, absolutely.
DR. WOODCOCK: Dr. Levin.
DR. LEVIN: Yes, just two things.
You just brought up a point about the
testing, but there's a business case for
NEAL R. GROSS
having resources for that too and so some
people it's going to be the same issues?
MR. TRIPP: Yes. I think there
needs to be a well-established test plan for
those so that we can go to those people who
are stakeholders and say, hey, we need a
stakeholder who is willing to test storyboard
one, two and three. Because they're related,
it's a short, manageable piece of work.
Who's going to step up and do that? And then
four, eight and 15 are related, we need
somebody else to pick up that. And I think
we're just collectively learning that those
are the kinds of details we need in this.
This is a relatively new process as you're
aware of.
DR. LEVIN: But the same issues as
far as justifying - going back to your
management and justifying that resource is
going to be the same issues.
MR. TRIPP: But I can measure what
that ask is as opposed to I just need an open
head count to work on this for ad infinitum.

DR. LEVIN: My question was back
to the CRIX or public-private partnership.
MR. ROSEN: "P3" works, Randy.
DR. LEVIN: Okay. Does it have to
- why is it - why did you say not-for-profit?
What about a for-profit organization running
that?
MR. ROSEN: A for-profit
organization running the infrastructure, I
think that a number of companies would be
reluctant for fear that monopolistic
practices could evolve if there was a total
lack of competition in a for-profit way.
Then certainly the cost of services could be
driven up instead of driven down.
Additionally I'd mention that the whole idea
of the not-for-profit entity itself is indeed
to channel any profit that's generated back
into the process and thereby reducing the
cost not only for all of the founding
members, but also for the associate members.
NEAL R. GROSS
DR. LEVIN: Now, the banking
industry with the VISA, that's a for-profit
model?
MR. ROSEN: No.
DR. LEVIN: Their - VISA is a not-
for-profit organization?
MR. TRIPP: Yes, it is.
DR. LEVIN: How about if you say a
public-private partnership can be other than
CRIX?
MR. ROSEN: I guess it could be.
It would not be my desire given all the
effort of all the many stakeholders who've
put numerous hours into that effort over the
years.
DR. WOODCOCK: All right. We're
going to have to conclude this presentation.
Thank you very much for your comments.
MR. TRIPP: Thank you.
DR. WOODCOCK: All right. Our
final presenters in this segment of the
proceedings are John Rapoza, CEO of JRRapoza
Associates, Inc., and Ari Kaliannan, Chief

Executive Member of Newtech Global Solutions,
LLC.
DR. RAPOZA: Audience, I don't
have any specific presentation. I'm just
going to make a couple of comments. I'm in
the business of basically dealing with
generic drug companies and in the process of
getting them approved applications. And in
the clients that I deal with there has not
been a lot of interest in electronic
applications. Primarily the cost, although
there are some. The middle-sized companies
have expressed interest and they do have
applications they're putting together. The
primary reason I think is the motivation to
do this given the fact that the generic drug
office is so far behind in these application
reviews that they have difficulty seeing the
advantages of making a change to a different
application format. However, I think the
OGD, the Office of Generic Drugs, is kind of
NEAL R. GROSS
making progress in that direction. I think
with the SPL, implementing and mandating the
SPL as part of the application was a very
good idea and that's how I got involved with
Ari here and he's going to talk a little bit
about his document management system.
And the other thing I think
they're doing which is most helpful is the
electronic review which essentially takes the
key elements of the application based on the
CTD format and allows a very quick review
application of the application for what the
agency would consider to be adequate
information. So I think they're going in the
right direction. I think it's going to be a
long time before you'll see, you know,
applications being submitted by small generic
companies because many of the drugs that
they're chasing just don't have the financial
margin to do that. Having said that, Ari is
going to give a little presentation about his
system which I became aware with the SPL
which is really a lickety-split system for

dealing with transfer of documents. And he's
going to talk about his systems as well.
Thank you.
MR. KALIANNAN: I hope I have
learned to use the computers. I take this
opportunity to thank the agency for giving
this opportunity for the presentation. My
experience in the license industry is two
years. I have been tasked to automate the
small generic industry with $16,000. It has
not come yet. U.S. dollars. It's not Indian
rupees. But I ask the question like what
software do you have. We have only one
commercial graphics systems, nothing else.
It's a good place to start with. So we ask
the questions to the users what you are
familiar with. Only two that are familiar
with this Office, Word and Excel. We started
from there. What it takes to make use of
these systems more useful to the end users.
Since we do not have much expertise or
NEAL R. GROSS
collaboration with other agencies, we had to
talk to three vendors. We planned on being
Microsoft Office, talking to them, how you
can make this Excel as user-friendly for the
regulatory environment, how we can make the
Office product more useful. They said you
wait for us, Office 2007 will be the answer.
We are not going to wait for Office 2007. We
need something to be done for this year. So
we started exploring that way with standards
and that's why we took the SPL as a reference
point to start our work.
Having said that, I think that the
last two presentations speaks very well about
the industry's needs and the standards and of
the non-profit organizations. I just want to
go through this here. Basically the business
challenges are editing among the business
partners and agencies, especially when your
really small companies who do not have
dedicated resources and not only one
discipline, who has to wear multiple hats.
That's a challenge. Dedicating, locating

files across databases and applications.
It's good that my company did not have one
database system to work with. I had to start
with file servers. Difficulties enforcing
common vent, check-in/check-out processes
have to happen because there's none. We
ought to establish one. Lack of integration
and familiar cost-effective document
management systems. Any vendor you're going
to ask for it. They are not going to sell it
for $16,000. We already cleared one. That
and - that's a challenge.
IT. Even though I was given the
title of IT Director, I preferred work in the
field first two years first. Without that I
said I will not take over the responsibility.
Having spent two years on the field, I take
over the responsibility to move forward.
Seeing a couple of applications awhile in the
market. What we can do with what we have.
That sort of approach we took. We used the
NEAL R. GROSS
Microsoft Office Word package to create the
SPL example data and we talked to our friends
in Microsoft even though they are not the
good one for security and standards, but
still you have to go with their blessings for
some work. So they had a solution called
BTSTAC sort of which basically gives a tool
to integrate the XML files. So we tried to
leverage the application, tried to integrate
whatever the possible automation we could do
within our capacity. Also we talked about
the enterprise systems, links, commercial
systems. Even though the mission was there,
there was not enough funding. So we started
out able to - XML for the form and work for
the management system using InfoPath tools
which is also available with Office, which is
very affordable. The question is who is
going to build the path, who is going to
validate it, how is it going to happen. So
these are the questions we are to answer for
the users first.
We took the initial step of doing

one thing first. That's what we did with the
SPL. And we are currently working on the
CDT. We are seeing wherever there is an XML
there is a place to go and look for. And
that is how we start the next one. Also we
are working with Microsoft and Alliance
program called BioAT trying to put the
puzzles together with the various vendors and
see how we can play a role in this
environment. That should be our next
initiative.
Anyway, nothing is more important
than compliance security, especially
environment like this. Having come to this
country without knowing what is 911, I spent
the last eight years dispatching, creating
software for at least 200 agencies in this
country. With no standards you know how
difficult it is to make the systems work
together. Finally comes to SPL and escape
the global XML example. Spent two years in
NEAL R. GROSS
understanding it. I saw that if justice can
be done with XML, why we cannot do the
pharmaceutical solutions and that's where we
are trying to look at it.
I just went through the review and
put up a couple of action items for our own
site, how we are going to deliver this.
Again, since this is a regulatory
environment, all business processes are to be
initiated and entered in the validated state
and systems. We already initiated with a
change control process. We leverage XML
data, use the built-in workflow mechanism
within the Windows 2003 environment and the
upcoming Windows Vista. I don't want to sell
Microsoft products, but somehow the features
are available which allow the vendors and
give the opportunity to derive some solutions
with the tool sets. And that is how we are
going forward developing this ECTD framework.
What we do, we do validation by integrating
various data sources, including any business
applications, including ERP systems, links,

common data systems, document management
systems, laboratories. We clear the portal.
We take the repositories in an XML format,
integrate them into portal environment so
there it's easy for remote access.
This is the philosophy behind our
two products today, SPL and ECDT, which makes
use of the Microsoft Office so it's less
training for the end users and it's easy to
use. And we take advantage of these
facilities portal. We'll do with Windows
Vista work for management features which
allows us to create a cost-effective
solution. We are engineers. We applied an
end-to-end system and product management
services from design, development, testing
and validating. This would be my brief
introduction. With this I leave the podium
for questions.
NEAL R. GROSS
Any questions? All right. Thank you. Okay,
what we're going to do is we're going to take
realistically a 15-minute break. Please come
back here. We will reconvene and begin
promptly at 10 minutes after 3:00. Thank you
very much.
went off the record at 2:52 p.m. and went
back on the record at 3:10 p.m.)
DR. WOODCOCK: We are going to
reconvene and get started. Our next speaker
is Mark Rutkiewicz, Director, Quality
Assurance, AGA Medical.
MR. RUTKIEWICZ: Good afternoon.
I feel a little like a PETA member at an NRA
meeting, or actually a cop at a FBI sting.
I'm with the medical device industry, one of
the companies in the Twin Cities. How many
other people here are associated with the
medical device industry? Oh, I've got a
couple of others. Yay. I'm with AGA
Medical. We make devices for making
occlusions in the heart for congenital heart

defects, but I'm going to talk today about
best practice of sharing electronic documents
in the medical device industry. I've been in
the industry about 20 years doing pacemakers,
defibrillators, external defibrillators,
implantable hearing devices at a few
different companies and one of my jobs has
been to engineer information so we can make
things faster and make change better.
I'm going to overview quickly the
rules and the history. I really want to talk
about a proposed process we've been using in
the medical device industry for managing
product information which is just as complex
as what we deal with on a submission. And
actually we are using a model that was
actually developed by the high-tech industry.
The laptops you see today cost - we can buy
them for $500 from Dell. Ten years ago they
were $5,000. So the high-tech industry has
taken a lot of techniques and configuration
NEAL R. GROSS
management, information control and applied
it to reduce costs. One thing they've also
done is instead of building it themselves
they outsource it so I'm going to talk about
a business-to-business model that's been used
by the high-tech industry. It also can be
used by the medical device industry and used
for submissions to the FDA, but worldwide.
Quickly on the device side, I
don't know if everyone knows. The types of
submissions that we do are we do PMAs, IDEs,
HDEs, 510k's and additional supplements
associated with that. For Europe, medical
device companies do either design dossiers or
technical files and there's a lot of overlap
between those submissions. In addition,
Canada has licenses. Their submission
requirements are lower. We don't have to
submit as many documents for those types of
areas. And then Japan and Australia, totally
different animals. We're not going to get
into that.
But the rules for creating an

electronic submission for the medical device
industry has evolved over time with different
authorities and different needs. When Europe
consolidated with MDD and the AIMD
requirements, they standardized this
technical file and design dossier which then
simplified, at least for medical device
manufacturers, not having to submit to every
country in Europe. It's always been done as
books or volumes and PDFs or electronic.
We've done some versions of the PDF that the
pharma's been doing, and each company also
has tracking methods on databases, but they
were just typically tracking specific parts
of a submission, you know, dates, what kind
of submission it was, who it was to, but not
the entire submission that's tracked
electronically by most companies. A couple
of years ago I ended up doing a pure
electronic submission to the FDA. I had a
paper, but it was really done electronically
NEAL R. GROSS
with the ENT group for an implantable hearing
device.
But historically why the
submissions are set up the way they are is
because every department owns their
information. There's a DMR file, there's
clinical files. We've seen a lot of
technical clinical information. The clinical
side isn't as large for a medical device as
the pharma side, but there's DHR files,
complaint files, engineering report files.
They're all managed independently. And what
happens when we create a submission of
medical device, you go into each of the areas
and you put it together in a binder and you
get the submission out. Well, these things
in these other areas can change and they'd be
- it becomes very complex when you get into
medical device because of the different
process interactions and what's
interchangeable and not interchangeable, what
needs to be submitted and not submitted.
So the history has been we create

each submission book and for each country too
independently of each other, even though
there's a lot of overlap. And then databases
or manual paper methods were used to track
these. And one thing that happens with this,
if we submit - a medical device manufacturer
submits something to the FDA, you can't
cross-link documents that have been approved
already. In a medical device company like
ours, we have about five or six different
products, either IDE or PMA, but basically
the same material, but each time we submit
the biocompatibility report and each time the
biocompatibility report gets reviewed by
somebody over and over again. There's no
reuse which makes the effort by the FDA a lot
more difficult.
So what I'm going to talk about is
some strategies that have been used by the
industry, the high-tech industry and the
medical device industry. Best practices.
NEAL R. GROSS
They've talked about - some of the other
speakers have talked about configuration
management. There is a process called
Configuration Management 2. It's an
institute that's a national one that actually
trains people all over the world. There's
tens of thousands of people that have been
trained in this. Aircraft industry is really
huge in this and military too. It also talks
about flexible end configuration management
too. You talk about flexible information
structures, products and processes. And they
haven't gotten to submissions, but it's
designed to be flexible.
I talked about information
mapping. I think that's been part of the
process to create the common technical
document. You know, defining who your
audience is and organizing the information
with XML. I'm going to go over a quick
little information map where manufacturers
control their information and talk about open
standards. And I think everyone has talked

about this. XML I think is the way to go
with this. I started using XML about eight
years ago for some information management
processes and it's still going great today.
This is a quick map-out of the CM2
process. They're an institute based out of
Phoenix. They're associated with ASU and the
University of Tennessee and they do training
on this. And some of the basic concepts here
is that you start with a master document
which you're going to submit and then it goes
through a change request, a change order and
a change implementation process. That can
happen internally or it can happen at a body
that's going to be accepting the information.
And it's called engineering change request
because they're talking product structure,
but the same structure can apply to the
submission because a submission is made up of
chunks of information. A change admin 1 is a
roll-in process that talks about do you want
NEAL R. GROSS
to even accept this thing to begin with. And
so sort of the first step in FDA, is a
submission submittable. Second step, the
change order is done. Does each section get
approved? And third is a change in admin
where you actually implement and that's a
feedback process. This process works
internally at a manufacturer but can also
work at a regulatory body.
So it's a 3-step process. Part of
the thing with the parts of the submission,
each chunk of the submission gets unique
identifiers so you can then allow for
reusability. Part of CM2 also is status and
revision identifiers for all objects of the
submission. This allows you to then reuse
linking existing objects. You know what rev
this part number's at or this document's
number at. Understanding interchangeability
so when you make updates to these
submissions. We currently have - one of our
PMA supplements out there has 440-some
supplements because we also use it for

emergency compassionate use concepts. So
we're doing these variety of kinds of
submissions with our submission, but it's all
tied to a base PMA so to be able to re-link
that information together becomes complex
when you're talking that many supplements.
One thing too in the medical
device industry versus pharma, and we're
talking pharma having you know drugs out
there for 40, 50 years. In a medical device
company a lot of - in the pacemaker and
implantable defibrillator industry the
products were being revised every 18 months
and had to get re-approved. So the speed and
the way of making that change, these
techniques we started using at some of the
bigger companies. Then I started
implementing these concepts to smaller
companies and they seem to work really
effectively. As going out from CM2 there's a
basic concept on information mapping. I'm
NEAL R. GROSS
not going to go into details here, but it's
basically organizing and understanding your
information types, which is a big key concept
in XML, understanding the type of information
that is in the document.
Regarding here the next slide,
what's in a manufacturer? In a medical
device manufacturer there's information
inside the company that we have control of
and there's information that's outside the
company. And part of the submission is
typically combining these things together
into one object that combines device master
records, what it takes to build a product,
design history files, the reports, some
regulatory documents that lead in and talk
about the submission, clinical documents that
are part of the clinical reporting. You
might tie in articles and papers to support
the submission. So we want to be able to tie
that all together into a common format.
So an example here is a BOM
structure. You build material. It's

creating a structure that's different than a
standard folder. In a standard I saw
earlier, some of the documents, you create a
folder and create folders within folders. In
this one I could link the same object across
multiple sections and not have to recreate
it. In a submission there's top level object
and then there's lower for each section and
you can tie different kinds of documents to
the different sections. And then you could
route, as we'll get into a little bit later,
you can route each subsection independently
for approval either internally at the
manufacturer or externally.
This basic process is what's been
done in the high-tech industry. There's
about 20 key circuit board manufacturers in
the world that make most of the circuit
boards and most of the electronics that you
have around here, your phones and stuff. But
there are hundreds and hundreds of design
NEAL R. GROSS
people because they end up outsourcing the
manufacturing. Those key 20 manufacturers
use one software application for managing the
information that they have to create the
circuit boards, and all the design houses
then follow that standard. And then the
design houses can move from manufacturer to
manufacturer and they can get their
information consistent to any manufacturing
house. So it's a manufacturer-to-
manufacturer-services standard.
So the standard we're going to
talk about is an XML standard for managing
the structure of this information. One thing
we talk about XML is paper to web. It's also
machine readable and also it's been done a
lot for the documents. That makes a lot of
sense to take the document XML, but combine
it with the structure and change management
associated with it.
So the needs I'm going to be
talking about in this process, what the high-
tech industry has done and what medical

device companies are doing is using a PLM
system, a product lifecycle management. We
talk about lifecycle of a submission. It's
not just a document control tool. There is -
in a document control tool there's no way of
creating a building material. A BOM allows
you to link objects together that have no
relationship to each other at all. You'd
have to create separate meta data fields for
all the objects to keep creating these
different links. The BOM allows you to be
independently created. The submission then
has a - would have a lifecycle by itself too.
You know, you could create whatever you
needed to. These tools that are out there
are very flexible. I can call it whatever I
want, but pending on a review hold. Design
the information architecture of a submission
with all the document types in a submission
record. So there's clinical reports and
records and then there's design reports. You
NEAL R. GROSS
might have - we submit things on
manufacturing process flow information.
We'll submit articles and papers. And if you
define a standard architecture for all that
information that goes into any kind of
submission to the FDA, then we can build
submissions that the FDA then can read. With
these, these tools allow you to attach any
file type to any of these objects. You can
attach PDF, XML, Microsoft. You can attach
files that nobody can read because a lot of
these tools have neutral file viewers that
can read 500 file types. We typically put in
in the high-tech industry there's the ability
to read a schematic and that's in a unique
file type that typically nobody can read.
But in a tool like this, a neutral file
viewer allows you to read it without having
to buy a $500 tool or a $1,000 tool to view
it.
Design. The design process has
multiple submission types. Like I said, for
medical device industry there's half a dozen

different kinds of submissions but then with
biologics and drug there's different other
styles. But if they can all be built with
the same basic architecture. And the
electronic submission needs to be designed
for approval, electronic approval and
routing.
So part of this too, like I said,
some of the information that goes into a
submission, initial regulatory summaries to
highlight the overview of what's being
contained, supplements, the DMR for the
medical device master record, all the specs,
drawings, labels, IFUs that are part of the
submission, the DHF, all the reports,
engineering reports. In the medical device
companies there's a lot of product that's
software. There's software in the products
or there's software - the medical device is
software. Software instantly doubles or
triples the size of any report because of the
NEAL R. GROSS
amount of documentation associated with
validating software that's used in a device.
Along with the clinical documents, annual
reports, postmarket information.
So the standard that the high-tech
industry is using today is called PDX,
Product Data Exchange standard. It's been
standardized with the IPC. It's the
International - IPC is the standards
organization for printed circuit boards
because they were using it. Multi-part
standard in how to create structures. It
focuses on the content between the
manufacturer and the electronics manufacturer
service providers. So tying those together
and that's the standard that a lot of work
with Intel, Dell, a lot of these companies
have used this standard for their
information. XML provides a standards way of
providing simple structure. And there's a
standard user group that - I looked this up
online. I'm not associated with this group
directly, but Barb Goldstein at NIST is

running this group.
Next thing I want to show you is
just an example of in my organization what
one would look like. We're in the process of
implementing these tools that are used by the
high-tech industry. What you see here is
this is a submission record. This could be -
I gave this a generic number. It could be
the number the FDA assigns when the PMA gets
it. That's what I've done in the past is
generically create this and then once we get
a number we change the number. Subsection
and then attach structure to it. These are
the documents, the reference documents for
that subsection. Here's another section with
different documents. And this just shows the
BOM explode. One thing by creating it once,
what happens is we ended up taking it, copy,
save as, edit it for Europe. So then we give
a copy of this then to our notified body in
Europe and they'd review our submission the
NEAL R. GROSS
same way with slight changes. You get a
different number, but basically the same
content. With this kind of structure too you
can click on any of these records and look
back and see where it's used. You can see
how many different submissions one document's
been used for.
What happens then with this is the
BOM is created internally and released
through a change order process. Then it can
be exported out once it's approved. I have
create a PDX file. And down here is what a -
this is just a file object that's sitting on
my desktop when I created this XML. You send
the PDX to the approve body. You can send it
FTP, through the web, send out a CD,
whichever method. But basically it's an
object called PDX. There's three PDX viewers
out there in the industry, it's like Adobe
viewers, that allows you to - this one shows
the exact same structure with a neutral file
viewer. Same structures, and the files are
all attached so you can click and see them.

So this is neutral. It's not even in any
system now. It's available for everybody to
use, anybody to use.
Then the process would - we didn't
end up doing this with the ENT group. We
ended up sending them the CD with the PDX
viewer and they were able to use the paper
copy for routing, but then for reviewing they
were able to take the CD with them and look
at it whenever they needed to and just carry
it on a CD versus the whole structure. But
the process that normal manufacturers in the
high-tech service industry would use is they
would take the submission from the design
house, route, import it into their system
with the structure, break up - that's you
know approving this initially is a change
implementation as I talked about as far as
CM2. Next, break each section up into
different sections for routing for approval.
So different groups would get different parts
NEAL R. GROSS
of the submission to approve. Then it would
have to come back together internally and be
put on another change order to release the
entire submission to an approved status. And
then it allows you to link these record types
together across multiple submissions. So if
you received a submission with a report
that's been approved previously you'd see
that when it gets imported in. And this is
what all the contract board manufacturers in
the world are doing this structure today.
So we were using a PLM tool in the
industry, product lifecycle management, not
just document control tool. The majority of
medical device manufacturers use PLM tools
because we have to have control of our
documents. All these tools, there's the
three that the industry uses is Agile, Matrix
1 and PTC from Windchill. Windchill from
PTC. A lot of people do this kind of
structure. They're all 21 CFR 11 compliant
for audit trails and electronic signatures.
Agile has a focus, the one we're using. A

lot of medical device manufacturers use the
Agile tool. There's other ones out there
too. It becomes a fairly simple tool to
track and implement. A lot of these
manufacturers can implement these things
fairly quickly. There's a case study that
Dell ended up implementing their Agile tool
in about six months. And they were 100,000
parts in 30 sites, so. A lot of these tools
are becoming easier and easier to use.
They've been around for about 10 years.
So recommendation here as from the
medical device side. Design a flexible and
proven submission process that can take into
account all the variations in all kinds of
submissions. Use industry standards and best
practices, I'm talking CM2, using some PDX
standards or XML standards. Use off-the-
shelf tools. Don't try and create something
custom. A lot of the stuff's out there now
that's available that's a lot easier to use
NEAL R. GROSS
than even 10 years ago, so. This is my
background. So any questions?
much. Questions from the panel? And I would
like to mention, Randy Lutter had to leave
and Clark Nardinelli who's Director of the
Econ staff at FDA is now sitting on the
panel. Any questions from anyone?
DR. LEVIN: Can you use your
system to submit something in the regulated
products submission standard?
MR. RUTKIEWICZ: What's the
specifics? You mean is there a specific
format?
DR. LEVIN: Well, there is a -
we're working on a standard in the Health
Level 7. It's an XML standard called the
regulated products submission standard.
Maybe if you stick around for maybe a couple
of speakers will discuss that.
MR. RUTKIEWICZ: Sure. Yes, with
these kind of tools, they're designed - as
you create different classes of objects you

can create as many different - they're
designed to be very flexible. You can create
as many different data fields that can map to
existing fields, and then the question is how
do you create the standard, the output, the
XML and tag it appropriately so it matches
the standard.
DR. LEVIN: Right, because here
we're working on a - using your
recommendations, it's just not the same
standard.
MR. RUTKIEWICZ: But if it's using
XML, then you can easily translate one XML
standard to another because as long as you
define the data types and the PDX in this and
they map, you can easily create translations
that can map to the common.
DR. LEVIN: Okay, I think we'd
like to speak to you after the meeting.
All right, thank you very much. I have
NEAL R. GROSS
another announcement. There was a ring left
in the ladies room. If it belongs to anyone
please just come up to me and describe it and
you can get it back. Our next speaker is
Jason Rock who is CIO of Global Submit, Inc.
MR. ROCK: I can tell you with a
high degree of certainty that ring does not
belong to me. I'd like to thank the panel
for allowing us to speak and my name is Jason
Rock. As Dr. Woodcock said, I am the CIO of
Global Submit. I am also the project lead
for the HL7 regulated product submission
message. And being the project lead for the
HL7 regulated product submission messages
allowed me access to a lot of folks in the
industry as well within the agency. I have
worked with all the centers so far. We have
evaluated a lot of their systems compared to
the RPS message. We started the testing
process. We have also done this with several
folks in the industry as well.
And today we heard a lot of good
presentations on different topics. And what

we're going to try to focus today on in my
presentation is cost. And we heard a lot of
things with cost so I'm going to try to not
highlight the things that we've already
heard, try to go over some of the things that
we haven't heard yet. And I'm going to
change my presentation slightly around to
focus more on market interactions and market
forces, particularly around competition. So
if my presentation seems slightly disjointed,
I changed things up slightly because a lot of
the content that we've talked about has
already been presented. I want to give,
based on the feedback that I've gotten from
several review folks that's worked on
regulated product submissions, several
industry people that I talked to, I want to
provide new feedback to you that you haven't
heard today.
So one of the big issues that we
see is on implementing a pure electronic
NEAL R. GROSS
standard. As you heard today, there's a lack
of resource out there. That means there's
not enough people to do the job that we
currently have. What I'm going to try to
provide is some solutions to that as well.
Then we're going to talk about, again on the
same flavor, on how to save costs within the
industry. We're going to talk about the
multiple guidances that are out there from
the FDA and around the world. We're going to
touch a little bit on the disparate
infrastructure, people having the same
systems, doing the same jobs. I'm sure the
FDA knows that they have the same systems
doing the same jobs. Abbott has the same
systems that does the same jobs. You know,
everyone else has the same systems to do the
same jobs. We're going to talk about how we
can consolidate those and we're going to talk
about getting input from the industry early
on in the process, from the broader industry,
not just people that, you know, work at ICH,
not just the people that work at the FDA and

how that's really going to help us evaluate
our systems earlier in the process. And the
whole focus of today's - of my presentation
is really going to be talking about costs.
And then I'll probably pepper in a couple of
things. I have an idea for a business case
of how we can sell this to management and so
there will be some other stuff in there as
well.
The first roadblock that we need
to overcome is the lack of resources that we
have in the industry. And you know as most
of you know there's probably not enough hands
to do the current job that we have. And the
reason why we don't have enough hands is
because we have an industry full of
specialists, and that is a lot to do because
of the FDA process. People that submit to
devices group that creates PMAs, they're not
necessarily trained to do drug submissions so
we have specialists. Anytime you have a
NEAL R. GROSS
specialist you have a high barrier to entry.
The high barrier to entry increases costs.
So we need to figure out a way that we can
share resources, lower the barrier of entry.
That will allow us to have more resources
coming in because we'll have a larger market
thus decreasing our cost and sharing
resources.
The next issue that we have, and
this is going along the same lines of sharing
resources and specialization. A lot of that
is multiple guidances. Multiple guidance for
the same business processes is very costly,
costly to the industry. When you have to
submit to CBER a PDF label and to CDER an SPL
label, that's not only cost infrastructure
and time when people are diverted from
different things, it's also very distracting.
So wherever possible we need to consolidate
the guidances. We also need to start working
on one submission standard throughout the
agency, one standard for all the things we're
working on, and we also need to reach out to

a global community. You know, although for
all the same reasons that I've talked about
so far about consolidating your resources
where most of these companies are global
organizations. Although you're going to get
good cost savings and good sharing of
resources working internally, we also need to
work externally as well.
And the first thing that you know
give a little story of when we started the
regulated product submissions our goal was to
create one model that could be used for all
regulated products worldwide. And the first
thing that people said was you know it
couldn't be done. And a lot of it was
because of different business drivers. You
have in medical devices, they regulate from
sunglasses to pacemakers. You have vet
medicines that have hooves, fins, beaks and
tails. You have drugs that has 100,000
submissions a year, some of them over a
NEAL R. GROSS
million pages. You have foods that - there's
an active submission out there that's been
active for over 30 years and they're still
getting information through it. And so
people thought it was very difficult to come
together, though it was impossible.
So what we did initially was we
started off with two groups. It was the
medical device group on one hand. On the
other side we had the drug and biologic
group. The drug and biologic group had a
head start. They have already worked on
electronic submissions in the past. Based on
their experience they created up a submission
model, a draft submission model that we
started to work on. The devices group we're
doing the same. They had the benefit of
working from the drug group because they had
a head start. They said that the model
doesn't meet their needs. They then took
that model, made some tweaks to the model.
Then we presented it to the drug and
biologics group and at that point the drug

and biologics group unanimously voted that
this was a better model that served their
needs.
So what we did at that point is we
then started the two groups. We thought this
was great, we were making good progress. We
then asked both groups to come up with a list
of priorities that we wanted to do for the
project. What we found out is both groups
independently came up with the exact same
list. The orders were different, as you can
imagine. In a device submission you know
they have a lot of products, a lot of models
you know in a device submission. In a drug
submission there were issues more because
they have a standard already. We had issues
more of well we need to be able to
communicate back and forth to the sponsor.
But we found out the list was exactly the
same. One and two was flip-flopped, three,
four and five were exactly the same. So we
NEAL R. GROSS
combined both groups and I've been sitting
around talking to everybody that's in these
groups and everyone's happier that we
combined them. We're learning that the drug
folks and biologic folks have a lot to learn
from the device folks. The device folks have
a lot to learn from the drug and biologic
folks. And I look forward to start bringing
the vet med folks as well as the food people
as well. So the one thing we learned from
RPS is that we can work together.
Now, here's the action item for
the FDA. If we're working together you know
as an industry, we need you to start working
together too. We need you to start looking
at all your processes and procedures. We
need you to consolidate your efforts because
when we - and for devices, when we have to
send the submission and then get an
application number, and for drugs the same
exact RPS test submission, we have to ask for
the application number first, it becomes very
difficult. We're going to still have

specialization and specialization increases
cost.
The next thing, it goes along the
same lines, is that the fact that I have the
pleasure of working with all the centers in
the FDA. You know I'll let you in on a dirty
little secret. I hope noone from the FDA
pulls me aside afterwards. They have
multiple systems that do the exact same thing
and that comes from their tracking system.
And you would expect that from any large
organization. I know I'm talking to industry
folks. They have multiple systems doing the
exact same thing. I've talked with members
of Abbott, letting their dirty little secret
out, and they have systems that are only for
device folks. They have systems only for the
drug and biologic folks, and they're doing
the exact same thing. And the reason why is
because the people they have to deal with on
the other end are doing things differently.
NEAL R. GROSS
So again, if we consolidate our efforts, work
globally, we can get a larger market for the
consultants and the vendors. And it's the
competition of that larger market is where
we're really going to drive down prices.
Because what's very expensive right now is
you have a very small market, all these
different styles are very small market and
you have very few vendors willing to get into
those markets. And those very few vendors
when they do get in the market, to recoup
their costs they've got to charge a lot of
money. Okay, this stops. That's why
everyone that goes into these markets
basically almost all the companies focus on
the big guys because that's only where they
can recoup their costs. They can't justify
charging big pharma millions of dollars and
for the exact same system charging a food
company five dollars. They just can't
justify that. But if we consolidate and we
can consolidate our markets, you know, we're
going to allow more companies to come in

because they're going to see a bigger market.
The processes are going to be consolidated so
therefore it's easier to get in. And when
it's easier to get in we're going to have
more companies. More companies always drive
down costs.
And this is a couple of things
where I'm trying not to throw some criticism
any one way, but I want to make comments
about other presentations. I think for the
CRIX model it's important to realize that if
you have one company that's a not-for-profit
or one for-profit company and that's your
only vendor that you have to choose for,
they're going to cost you more money than 10
companies that are doing the same process.
And the reason why is because there's no
competition. There's no incentive to be
cost-effective. There's no incentive to be
innovative. So you're going to get worse
products for more money. So although I
NEAL R. GROSS
believe in the CRIX model, I think the CRIX
model is the way to go, I would urge the
board of the CRIX board to allow for-profit
vendors, multiple for-profit vendors to plug
into the network so that the industry has
choice.
My last point is from lack of
industry, lack of input from the industry.
We have good experience both on the regulated
side and on the industry side. We need to
leverage those experiences and we need to
work with organizations that have the policy
and the procedures to allow us to do this in
an open environment, in a fair environment.
And that's related to standards developed
organizations. And we need to focus on
international standards development
organizations. We need to focus on
organizations like ISO where we can put our
business requirements. We need to focus on
organizations like CDISC where we can work on
clinical data. We need to focus on HL7 who
does the messaging, but we need to do it

together. We all need to do that. And the
reason why we need to do it together is
because the cost is too high if we don't.
We've all seen what happened with the ECTD.
We've all seen that it came out and there
were no vendors that were willing to come out
of the gate and say here's a product, start
using it. We saw regulatory authorities slow
on the uptake because without the input from
industry, without the input from the vendors,
without the consultants noone can support us
out of the block. So what we need is we need
to all go in the same development
organizations where we can all go and then
where we can all participate. When we do
that we can evaluate our systems internally,
we can see if the standard meets our systems.
If they don't we can make recommendations to
make them meet our systems. We can probably
do that in a low cost earlier on in the
process until we implement and we'll get your
NEAL R. GROSS
support, your consultants, your vendors, your
software, your software companies out of the
block being able to support you. Now it's
not going to happen immediately. That's not
going to say, oh, we have a draft standard or
a final standard and we're ready to go, but
what that means is everyone's been a part of
the process and soon after that you know once
things are finalized, soon after that people
have already been developing. I've talked to
all the vendors that do the ECTDs at least in
the United States, most of them in Europe as
well, and they're all preparing for regulated
product submissions. If this development was
happening behind closed doors they wouldn't
know what was going on, they wouldn't know
the standard or where it was going and they
wouldn't be able to participate early in the
process. They wouldn't be able to support
you when the standard is ready.
Obviously we need strong
leadership. And we're going to need to reach
out to people that we've never traditionally

worked with in the past. You're going to
have to see all of the agencies, we need the
leadership you know from people like Dr.
Woodcock and Dr. Levin and Dr. Oliva to tear
down the walls of the centers and start
working together. We're going to need the
food product associations, pharmaPhRMA,
bioBIO and AdvaMed to start working together,
start interacting with each other, start
making joint recommendations to the FDA. And
we also need to start working more
effectively with our global partners. And
when our global partners are unwilling and
unable to work with us, we need to continue
to extend our hand. We need to realize that
to get our true cost savings it's really
going to come with working with our global
partners. But we also have to realize right
now they might not be able or willing to join
us. So we have to constantly communicate
with them. We have to tell them we're
NEAL R. GROSS
working in international organizations.
We're working in open process. This process
doesn't favor the FDA, or the NIH, or any
U.S. companies. And we need to bring them
into the fold.
We also need managerial leadership
to realize that it's cheaper to work together
than to work in silos. We also need these
leaderships to come together and fund new
positions. To work in standards development
organizations is much cheaper than not to,
but it is not cheap. We need reviewers at
the standard development organization table.
We need the publishers there. We need the
managers there. We need IT at the table
working together. We need people inside
companies whose big part of their job is to
work with SDOs and we need people inside the
FDA whose a big part of their job is just to
work with SDOs and to be a liaison.
To sum up, I think I'm okay with
time. It's - to sum up real quick, we need
to first and foremost, although I'm a big

believer, we're all - the big cost savings is
going to be working with the global
community, we need to first look internally.
We need to do what we can and the first step
is really the FDA needs to take the
initiative and consolidate their guidances.
We see what's going on in Europe. They're
making a valiant effort. They want to work
together. It's been very difficult and my
opinion is the reason why they're not moving
as fast as everyone who I talked to would
like to is because they don't have one
person, one authority that can say you're
mandated, you must start working together.
Inside the FDA that's not true. You have the
commissioner that can mandate and says the
agencies can work together. So that is
something in my opinion, I know although
controversial, in my opinion is where we need
to start. We need to work within SDOs in
international SDOs. That will show our
NEAL R. GROSS
global partners that we are serious of
considering their needs. We also need to get
the feedback from both industry and agency
alike.
And what this is going to allow us
to do, this is going to allow us to share
resources. We're going to get improved and
cheaper infrastructure. This is going to
come from competition. As I said earlier, we
cannot allow one organization, not-for-profit
or otherwise, to monopolize the space. They
won't be innovative. They won't figure out
new ways to make things cheaper. We need to
have competition. And it's very important
that we get industry buy-in early on in the
process. And if you're participating at the
table and you're making your feedback into
the standards development process and then
the standard does not work for you, the only
person that you have to blame is yourselves.
So in short I believe we do all this, we're
going to increase adoption of electronic
submissions and hopefully we can do it sooner

than later. Thank you for your time and I
guess questions.
DR. WOODCOCK: Thank you.
Questions from the panel? Dr. Levin.
DR. LEVIN: I have a comment.
Just to take time now to talk about the - in
standards development organizations it's not
easy to develop these standards and get all
the people cooperating together. And I just
want to thank Jason for the work he's been
doing in the regulated product submissions
standard. It's very difficult to get all
those groups together and you've been doing
an excellent job in doing that. So just to
thank you for that.
MR. ROCK: Thank you sir, and I
don't think I could have done that without
the support of the FDA, yourself and several
industry individuals out there such as Ed
Tripp from Abbott, Karen Sailor from
Medtronics, and so many of you.
NEAL R. GROSS
DR. WOODCOCK: Any other
questions? Thank you very much.
MR. ROCK: Thank you.
DR. WOODCOCK: Our next speaker is
Laurie Rose, Director, Product Management,
SAS.
MS. ROSE: Thank you, Janet. And
I am from SAS. I am the Director of our
Product Management and Strategy organization
for Health and Life Sciences. We looked at
the docket a few weeks ago and decided that
based on some of the experience that we've
had in the industry for over 30 years and
working with the FDA that we might have some
hopefully good contributions to make
regarding those issues of discussion. And we
do appreciate the opportunity not just to
share our thoughts, but also to hear the
perspectives of everybody else who has
presented today.
Two of the areas that we feel
probably most qualified to address are - were
in the sections around the electronic

submissions about the implementation itself.
As not being a sponsor and not having those
costs and time issues but supporting those,
we wanted to focus more on the
implementation. And then also on the section
about third party entities. We've provided a
number of standard tools for the industry for
years for analysis and data management, but
in more recent years we've also been
providing some evolution of our technology to
support things like the CDISC standards and
the electronic submissions. So we wanted to
focus on that from our experience not just
providing those solutions, but also from
hosting those for the industry.
Before you think too much about
that picture, I probably should have backed
up, but I think what most people think about
a submission they think about an event.
There's a timeline. You start a project.
You develop protocols, trial design models.
NEAL R. GROSS
You get your statistical analysis plan. You
start running those trials, collecting data
and analyzing those. And at some point you
come to a belief or theory that what you have
is in a desired state and it gets packaged up
and it becomes a submission. And hopefully
there's been some communication with the FDA
along the way with pre-submission meetings,
but still viewed more as an event. And for
this discussion today from our perspective
we'd really like to take a look at that as
more of an ongoing process. Not just an
event, but the electronic submission as a
process itself. And part of what we want to
concentrate on is that theory about that
electronic platform. What can that be and
what can it hold to better enable this
process. We believe that from the sponsor
side you know this is something that would
basically have global accessibility. It
would be internet-based so people would
basically just need thin clients and access
the system anywhere in the world and do that

collaboration in real-time if necessary.
That might be somewhere a little bit more in
the future. I think one of the questions in
the docket was should there be a phased
approach, and certainly developing the tools
and the infrastructure to support this does
need to be phased. Exactly what would happen
in those different phases needs to be
examined I think in more detail to make
strong recommendations along that. But
basically with that platform holding things
like common tools and even a common shared
workspace between sponsors and the FDA to do
that review is a recommendation that we would
like to make to the FDA and see how that
might be able to be accomplished.
Also just from the phased
approach, if we look long-term, and I've got
a slide at the end that will sort of give a
little bit more of that futuristic view, but
initially I think just being able to handle
NEAL R. GROSS
those components and that ECTD lifecycle and
the amendments to that, but even further down
the road using the same concept and possibly
even the same type of platform for post-
marketing and reviewing that with sponsors as
well.
If we look at what might be inside
that box, and I'll have a little bit more
detail later, but basically there are a lot
of technology pieces that have to be put in
place and infrastructure from whoever is
maintaining those systems to create those
firewalls and those partitions to allow for
the privacy that's needed so that the
sponsors using the system can have their own
private workspace for development and
testing. And when those elements are at a
point where they want some preliminary review
to promote those to more of a shared
workspace. We're also looking at the same
type of platform for supporting something
like JANUS where the FDA has their own space
where they're collecting all of that

information. You've got probably the best
treasurer of clinical information in the
world and being able to mine and analyze and
understand that data in a much better way.
One of the things that we look at
from our own perspective around cost, and
we've been providing a hosted system as an
offer probably just in the last four or five
years now. And we are seeing more and more
of the industry asking us to provide that and
give them that service because there are so
many benefits. If you look at what a box of
software would cost if you're getting a set
of CDs and you're going to take that software
and manage it and install it and validate it
yourself, there's a certain cost to that and
it's certainly going to look lower initially
than if you look at the one that's going to
be hosted where somebody else is providing
hardware and services with that. But we have
seen not just from our own analysis, but more
NEAL R. GROSS
from the return on investment analysis of the
industry where we're having both large and
small pharmas asking to have that hosted
because of that reduction in all the
resources, not just financial costs, but the
burden for IT primarily. I believe it was
ThinSpring earlier who mentioned the
exponential cost when you start looking at
complexity and just that one validation
component. That is something that is an
exceptional part of the puzzle when you look
at having to install and maintain and
validate a regulatory compliant environment
for FDA submissions.
I would have to say that we are a
very strong supporter of CDISC. We're proud
to have our own staff, Dr. Ed Helton, as one
of the board of directors. We also have
member on the Industry Advisory Board and a
number of our research development and
consulting staff on the technical committees.
So we would certainly want to continue that
promotion with the FDA to support those

standards. I think our comment around that
though is that we see both as a vendor as
well as working our sponsors trying to
implement those standards that they do need
to be simplified. There are many and I do, I
believe we lost our CDISC people, but very
well aware of the great efforts that have
been made to get those standards to a point
where they are today where there has been
agreement with the industry to come up with a
number of those models that they can
implement. But we see such a vast array of
difference between the CROs and the sponsors
who are implementing those standards.
Everything from the ones who are just trying
to see how they can cram everything into SDTM
at the very end so that they can do a
submission using the guidance for SDTM maybe
for the first time to those who are really
embracing what that business case might be
and using those standards throughout the
NEAL R. GROSS
entire process, and trying to really gain
some of those benefits of using that standard
data, especially the global organizations
where they've got multiple sites, multiple
entities who are contributing a specific
study.
I think the bottom line in what
we're trying to comment here is that there do
need to be more tools. And I'll even take a
moment to maybe kick SAS a little bit and the
other vendors and the other providers. I
know Octagon provides services to do CDISC
transformations. A lot of the sponsors don't
know what's available. They don't know what
tools are there. They're aware of many of
the review tools that have been developed
with the FDA through CRADAs, but they don't
all apply necessarily to the submission
process. And so they just don't really know
what's available. So for those of us who are
providing those tools, we need to probably do
a better job of educating the market about
those. And then the FDA, we also want to

support having those standard methodologies,
whether it's through guidances or the support
of those standards for developing an ECTD
using CDISC standards throughout the process.
Just to liven up and get away from
all that text, here's a nice colorful slide
with a lot of text on it. Don't want you to
get too bogged down in what's in all of the
boxes, but this is basically to sort of take
a step into that platform. There would be,
you know, the sponsor's environment, they've
got a lot of inputs coming in, they've got
clinical data management systems, they've got
EDC systems. The one, there's a small box
right in the middle there, that meta data
integration, and that's pretty critical
because that's where all of that information,
whether it's TOX data, PK lab, the clinical
information, whatever it is, is going through
a process and hopefully as CDISC standards
become adopted more, that'll more and more
NEAL R. GROSS
become CDISC standard data that's going in
through that process and eventually being
merged with the documents and the other
information into an ECTD.
This shows kind of a handoff
between the sponsor world and then this
shared world where there is this compliant
repository, where there are capabilities like
audit trails and electronic signatures, a
meta data registry that allows that access
into the data and the analysis and the
results, and then it's all being hosted by a
third party system. Now this picture I have
to say I kind of cheated a little bit. This
is not a SAS picture, it's really just a
visual, but SAS has experience with something
that's very similar to this. I hope most of
you are aware of the FDA pilot that was done
with CDISC recently where there was a mock
submission. There were a number of industry
players who provided a lot of work and a lot
of input, and they actually used SAS's
electronic platform to host all of that

information. So these different parties from
different areas across the country were able
to load and put the information in there.
And then the one thing that was different is
that it was not a shared repository. There
still was that package and handoff at the
end. But that mock submission was also done
in a matter of months. It wasn't done over
years. And it was done very quickly, and
hats off to everybody who was involved in
that project because it was really amazing
how much they accomplished I think between
December and September of this year when they
presented at the CDISC interchange. But we
certainly would promote that this type of
shared environment we do believe would have
great benefits in reducing that overall cycle
time in running a trial and getting to a
submission standpoint.
Let me switch gears a little bit
to the third party entities. And I'm really
NEAL R. GROSS
- we're really approaching this from how we
have been used as a service provider in
hosting a system for the sponsors and for the
industry. The first bullet point there,
neutrality, what I really mean by that is
that the person hosting that system does have
to be neutral. In other words, there need to
be some definitions around what can be
provided by the person doing the hosting. So
you're hosting, you're providing
infrastructure, you're not accessing our
touching the data in any way. So that's
really what we're meaning by "neutrality."
The experienced third party is
just critical because there are risks, very
high risks, that are assumed by that third
party when they take on that responsibility.
But one of those is to take the risk off of
the people using the system. So for example
if you're concerned about audits and whether
you're going to pass an audit, you need to
have all the standard operating procedures in
place. You've got to have the ability to do

that validation and do it correctly to
alleviate that risk from the people using the
system.
On the business process modeling,
we believe that that third party can
certainly be a facilitator, but should not be
dictating what processes are in place
especially if we look at a shared environment
between the FDA and the sponsors. Those
processes need to be agreed upon by both
parties from a business modeling standpoint.
And just on the collaboration side, excuse
me, I'm hitting a mouse here accidentally.
The framework for collaboration - I'll just
skip down to that part just a little bit.
The security mechanisms have to be put in
place. I've heard a lot of people say
security, security, security as well as
communication. But that is important, not
just from the physical security of what's in
that data center, but also privacy, user
NEAL R. GROSS
access, things like that. And the user
profiles themselves. Whatever is in that
electronic platform, there are going to be a
number of different tools and applications.
It's not going to be just one system. So
what types of users will be accessing it and
what level of access do they need. Do they
need programming access, do they just need to
be able to do simple reporting, but
understanding those profiles of the people
accessing the system. Managing things like
change, the change management procedures and
the standard operating procedures in place
for that, the whens and the wheres and the
hows. And then also having some agreement on
the governance.
On the services side, and Jason
really - he hit it hard. We really believe
that whatever services are provided, there
should be some very standard services for
providing the infrastructure, providing the
implementation of that and providing some
training around the implementation of that

platform. But additional value-add services,
consulting on how to do things more
efficiently using the platform, that's fine
for that third party entity as long as it is
an openly competitive environment. So Jason
hit that one hard, and thank you Jason. I
agree wholeheartedly with you on that one.
The technical services, I believe we also
heard some notes about this earlier, but
certainly things like disaster recovery and
backup and all the planning that are
associated with those types of technology
services that need to be provided as well.
These are some of the attributes
that we see that are necessary. The first
one just being 21 CFR Part 11 compliant, and
that sort of goes without saying except that
there's some complexity in managing a system
that has that. So that third party needs to
be able to provide that type of service and
that validation. Security, once again,
NEAL R. GROSS
whether it's at a user level or a data level
within the physical entity itself. And then
the scalability and performance. And
certainly as we have gone through some pilots
with customers who want to expand users very
quickly. We know that you have to be able to
have the ability to grow. The third party
entity needs to be able to expand hardware.
They need to have data center capability.
They also need to be able to meet the
expectations on performance, response time,
hitting a button, how long does it take for
something to happen. If this electronic
platform is going to enable people to move
more towards electronic submissions it can't
become a bottleneck so performance is very
key.
I think we talked about the
registry and repositories, basically having a
meta data-driven system that allows for data
loading and exploration and analysis,
reporting. Some of that goes back to those
user profiles. What are their needs going to

be? What capabilities need to be in there?
A couple of additional attributes,
and these have been requests coming
specifically from the industry. If there is
an electronic platform, they do want to have
that global access, it needs to be web-based.
It does need to integrate with different
types of tools. It needs to have some
extensibility and openness. And I know
within our company we're looking at things
like service-oriented architecture. We have
capabilities within our own system right now.
We need to be able to expand those into other
areas and other integration with other open
tools as well as other industry tools. So
all of those capabilities need to be part of
the system as it grows. A lot of people are
also requesting more work flow. A lot of the
smaller organizations, the biotechs, are not
- for many of them it's the first time
they've done electronic submission and
NEAL R. GROSS
anything that you can build into a platform
to make that process easier so they have to
maybe pay less to those consulting resources
that help them is very valuable. We also see
a lot of the larger pharma who would like to
have some automation just to speed things up.
We also see that it's very important to
support that end-to-end process of
information flow from HL7 to CDISC and
through the ECTD process. And then the
sponsors themselves also want that common
area supporting the standard data that they
are trying to adopt themselves in their own
organizations to do cross-study review.
A little bit about that future
look. What's in green is a little bit out.
Probably not very many people are using that
kind of methodology right now. It's more
what's happening in the yellow. It's just a
little bit of a different view of what we've
looked at before, but what we believe is that
with the electronic platform, as CDISC
continues and the organizations really help

start building the standards for the e-
protocol and trial model design, that
electronic platform is going to be fed with
really automatic feeds that are set up with
data and meta data coming into the system so
that there is a meta analysis. There are
automated procedures for doing that analysis.
And the safety and efficacy results of that
analysis can then be used to feed back into
those systems things like adaptive trials to
really help make the whole process more
efficient, to take some of that guesswork out
each time, to use that information and
improve those processes, improve those design
models and those protocols as we move
forward.
Any questions?
much. Does the panel have questions? No?
All right. Thank you very much again. Our
final scheduled speaker today is Terence
NEAL R. GROSS
Zagar who is Chief Technologist at Northrop
Grumman Information Technology Health
Solutions/Life Science.
MR. FISHER: Thank you, Dr.
Woodcock. We have a slight change to our
speaker list. My name is Harry Fisher and I
am the General Manager for Life Sciences at
Northrop Grumman. And I'm accompanied by our
Chief Technical Officer Rim Cothran and Terry
Zagar, one of our program managers as well.
So with that being said generally we'll be
talking about the same things. But based on
how the day has gone, and I will say I think
I heard someone, I think it was Debra said
she had the worst speaking slot. But as I
look at the empty seats I say hmm, I wonder.
But I think those empty seats are going to
see that the best was left for last hopefully
and we can add some value here.
So one of the questions we get and
one of the areas we're going to address is
first off very quickly the question always
becomes why is Northrop Grumman here. Don't

you guys build the B-2 bomber? And yes, yes
we do do that. But we have about an $8
billion IT practice of which about a half a
billion of that is for health solutions. And
what I'd like to offer today is not some
theory around some things that might be
interesting. I think there's a lot of `Mom
and apple pie' here that absolutely makes
sense from an e-submissions standpoint and we
certainly support CRIX. What we'd like to
offer are some specific points regarding the
third party entities that we have direct
involvement with today and share some
experience that could be directly germane to
the activities that CRIX could be doing or
any potential third party entity.
So some of the questions that we
wish to address. I'm not going to read
through these, but are around models for
setting up a third party entity. I believe
it was Bill Rosen said P3, public-private
NEAL R. GROSS
partnership. What does a P3 look like?
What's the experience with it? What does it
cost to ramp up? What about this thing
called monopolies? And is cost-effective
mostly in a competitive environment, a
different environment, et cetera, et cetera.
So these are the questions that we wish to
address.
Why is this important for us to
talk about this? Well, I've highlighted five
programs that I think can show some various
ways to look at the analog of CRIX along with
any other type of submission agency that
would be involved here. The first one is
AHLTA which is something we do for the
Department of Defense and the second one is
the Federal Bidirectional Health Information
Exchange which we do for the VA. Those two
you can look at together, but what's
important about those two? What's important
is that Northrop Grumman runs the largest
electronic healthcare record system in the
world. It's live. It operates today. So I

know there's been a great deal of talk about
EHRs and how they work and how they wouldn't
work. We do this today and on a conservative
estimate we have 10 million beneficiaries
globally. On an extended beneficiary list we
have 20 million. The difference being is
that on the one hand for the VA we handle
things like the technical aspect of how this
works. We don't tell them the process. We
don't tell them the content. We allow for
bidirectional communication. And while in
certain cases in certain environments people
will say the technology is the last thing you
need to worry about. Well, what we find is
if the technology doesn't work nothing else
works. For the Department of Defense we do
something a little bit different. We deal
with the processes, the content, the
strategies and the technologies underneath of
that. So we deal with eight to ten separate
proprietary EHR systems on a global basis and
NEAL R. GROSS
try to figure out how to make them all work
and talk so that DoD gets what they need for
their beneficiary base.
The two other ones beneath this,
PHIN, NHIN. Rim will talk a little bit more
about NHIN, but we have over a thousand
people in Atlanta working on activities like
this, a public health information network,
and what that means and what the benefit
would be to the populace in the U.S., but
also because it's a globally-based activity,
how do you enforce that. How do you make
that happen.
And then finally what's of
interest here and it might be very germane
for this discussion as well is MedDRA. We
are the maintenance and service and support
organization for MedDRA. So for anyone out
there who deals with MedDRA, has a
subscription to MedDRA, you've dealt with
Northrop Grumman. Northrop Grumman maintains
that solely and in completeness. And we do
that under the organization which we'll talk

about a little bit later which is a not-for-
profit organization.
So when we think about what that
means to us and we think about some of these
questions, what do we see? Well, when we
think about the question of what are the
things that are important, multiple
stakeholders. It's almost essential that we
see a public and private entity
participation. And we see that very clearly
because in the cases where things actually
have to happen, if we don't have both support
in the public and private sector then it
stops. And for one hand on the public side
what we see is that's where the mandates
come, that's where the regs come from. But
if we look in the private sector, if there's
not an associated ROI it stops dead in its
tracks. If we go down to different
stakeholder needs, it's very, very different.
But what you have to always promote is a
NEAL R. GROSS
collaborative interoperable environment.
Now I'm going to use the case
example for what we do, and we do a little
bit more of this in MedDRA, but in MedDRA we
have stakeholders globally. We have a 25-
person management board. They're made up of
government individuals, NGOs, industry
associations, private partnerships, all over
the globe. And I can tell you there are some
groups in there who will say no one should
ever make money on anything. So how can we
actually charge people for MedDRA? Then
we'll have other people who say well you know
what, we should charge them as much as they
possibly can bear because then we can invest
those funds in other things as well. Now,
we're not in that discussion with the voting
member. We're in there to help facilitate
the discussion, but the fact of the matter is
we're the ones who implement what comes out
of that. So in terms of the different
stakeholder needs we see that.
We see workable models for a

public-private partnership in many different
ways and some of those we've already
discussed. But if you get down to the bottom
line, there are multiple workable models and
we're experiencing that right now. So I'm
going to skip this. Rim will come back to
this. But the model that you see here, and
I'm not going to go through all of this, is
related back to what we do with MedDRA right
now. Now what looks very complex there can
indeed be very, very complex. And rather
than go through that model, you can read
through that. How did that evolve? It took
years for that to evolve. So we're in Year 5
of this agreement. We've just mutually
extended the agreement through 2011. And
what that means is we've gotten really good
at doing this. And it took a long time to
figure it out. And it cost a great deal of
investment. And so there was a - and I don't
have the graph for this, but you know if you
NEAL R. GROSS
look at your investment and finally where you
get to the breakeven point, it took a great
deal of time to get there. But here's the
benefit. The benefit is that in this area,
that model that you look at, the costs for
the average subscriber have decreased by 20 -
50 percent over the last two years. Why is
that? Because we figured out how to do this
model. We figured out how to efficiently get
membership information in to change the
schemas as required, to go through the
process of medical evaluations and process
evaluations, all the different things
involved in this. So in any event the way I
look at this is there's something out there
that makes sense.
And then finally from my side,
these are the things that I think will help
in terms of separation of responsibilities.
One of the things that we don't do both in
our public work with MedDRA, what we don't do
with our work with DoD and with VA, we don't
have a vote. I mean, if the board decides to

go in a certain direction, if the DoD decides
to go in a certain direction, we have to
carry those orders out. And they have to
make sense, and obviously we have influence
over that because we have data and we have
observations. But the reality is as you look
at this there's a separation of
responsibilities throughout. But the key
that I would leave you with is that it is not
a vendor relationship. In this model if we
go forward the stakeholders are vastly
different. There's certain competitive
advantage. There's certain non-competitive
advantage. But if you think that you can
have a vendor perform tasks for the lowest
possible price you will get exactly what you
pay for. You must invite partnership in this
type of scenario because the folks who will
be closest to how things actually work will
be the ones who can help you make decisions
as to what's the best public interest both
NEAL R. GROSS
now and in the longer term. So with that
I'll turn it over to Rim Cothran to talk a
little bit about our experience with NHIN.
MR. COTHRAN: Thank you, Harry.
Before we get started on this slide I think
the first question you might end up asking
yourself is why are we talking about NHIN. I
mean, NHIN is - the Nationwide Health
Information Network is a mechanisms for
exchanging clinical information among
physicians for point of care. And yes that
is what the primary goal is, but
fundamentally what we're talking about here
is an information exchange that takes into
account the business processes, regulatory
environment, privacy needs and security needs
of the health industry. And I think that if
we talk about how NHIN is being developed and
the public-private partnerships that are
being put in place there to support its
development, it gives us at least one model
that we ought to look at for other parts of
the health industry as well. Also, if we

think about NHIN which is represented as the
central circle in this diagram here, it's
usually the solutions themselves that people
focus upon.
Well, what the Office of the
National Coordinator for Health Information
Technology within HHS put in place was really
a number of parallel efforts that all worked
in conjunction with each other in order to
develop an overall solution. So we'll start
at the top of the chart here and talk a
little bit about practices and policies.
They established an organization that their
charge was to look at the practices and
policies associated with privacy of patient
information. And it's important whether
you're looking at regulations that are
established at state level as it is for
clinical care or more broadly at the national
level as it is for the regulatory industry,
it's important to establish those policies
NEAL R. GROSS
and to continue to maintain those policies.
Electronic submission, electronic exchange of
health information allows for different
problems, different issues than we've had in
the past and it's important to continue to
monitor what policies need to be in place to
facilitate it.
As we move clockwise around the
circle next we reach certification. And what
they also put in place is a separate
independent organization charged with
certifying systems that end users would end
up using. The primary reason for that was to
help those end users understand the
functionality in a competitive environment.
We've already talked a little bit about why
it's important that that competitive
environment exists, and it is, but it's also
important for the end users to understand how
different products address the needs in that
competitive environment. And certification
and a body that supports certification is
important in order to realize that.

So we continue to move around to
standards. What they also put in place was a
public-private partnership to identify main
standards for information exchange and the
implementation guides that should be put in
place. One of the things you can say about
the health industry is there are plenty of
standards today. And in fact, the HITSP that
is the embodiment of the standards
organization identified well over a thousand
standards for health information exchange and
need to identify exactly which standard were
properly maintained, how they were
maintained, identify which ones met the needs
for information exchange and how they would
be interpreted to best implement health
information exchange in a uniform
environment. As I said, that's a very good
example of a public-private partnership in
that that organization is made up of 200
different independent organizations. It
NEAL R. GROSS
includes standards development organizations,
it includes private industry, that has a
stake in this industry and is interested in
providing services for it. And government
agencies that are both interested in
regulating, promoting as well as
participating in that exchange.
And then finally we reach back to
the Center for Architecture Exploration for
NHIN and that is really to bring about
solutions for the exchange of health
information. What's interesting about the
approach to development of the Nationwide
Health Information Network, the NHIN, is not
that government was interested in putting
that in place or that they were interested
necessarily in establishing a single vendor
that would own NHIN and that's who you would
approach. But instead, inspiring industry to
take on this as an endeavor by funding early
organizations to explore different solutions
and in a public format describe what those
possible solutions were. And in fact, the

culmination of this program is coming up soon
in January where the public is invited to see
demonstrations of the NHIN that have been
developed by a number of vendors working in
partnership with all of these organizations.
What's important is that these organizations
don't only operate as public-private
partnerships themselves, but the
interoperation between them is also
important. The certification body takes
standards that are developed by the standards
organization in order to produce the
guidelines for certification. And both of
those bodies have to understand the solutions
that are being developed as part of the NHIN.
If we move on to services, now we
talked a little bit already today about the
different services that might be developed or
might be offered by a third party provider.
There are really two different buckets that
you might put these services in. The first
NEAL R. GROSS
listed here is provider services that are
primary services. The electricians and
plumbers that are actually doing the software
development, doing the interface development,
establishing the infrastructure, et cetera,
and we've talked a little bit about that and
the need for that to be a competitive
environment. Also, the organizations that
maintain the standards as participants in
standards development. There's also a need
for secondary providers, certification
bodies, auditing bodies, et cetera, that
monitor the exchange of information.
What's also important though is
there is this second bucket. There is an
opportunity for providers of services that
actually establish and facilitate the
exchange of information. This is a
developing concept within NHIN and ONC today.
But the idea that there is a place for an
organization that acts in the same way that
your internet service provider does, that
establishes connections between organizations

and facilitates the movement of information.
Some of the services that are necessary to
facilitate that for NHIN are listed here and
I won't go through all of those because this
is specifically about the movement of
clinical information, but a few of them are
probably interesting to highlight. The first
is the translation. Really what we're
talking about there is an arena where the
standards are developing, standards are
changing and not yet established and there
are legacy electronic systems within the
organizations that may want to exchange
information. There is a place for an
organization that provides as a service a
mechanism to translate between different
standards, to transform information from one
standard to another to facilitate the easy
exchange of information. It takes the burden
off of the exchange in organizations and the
receiving organizations, but still promotes
NEAL R. GROSS
the use of those standards.
Many of the other services that
you can imagine here would support routing of
information to the right organizations that
provides querying of information, et cetera.
What you will see missing from this list
though is it's not necessarily that this
third party organization build a repository
to hold all of that, that it is possible to
just offer services that facilitate the
exchange of information without needing to
own that. You can think for instance, again
as I turn to your ISP. Your ISP does not own
your financial information, but facilitates
the transmission between your bank and your
own financial application at home without
actually storing any of that information
centrally. And there is a place within the
health industry also for an organization that
we've begun to call network service provider
that exchanges that information without
actually owning it. Terry?
MR. ZAGAR: So there actually is a

Terry Zagar, as everyone knows. What I'd
like to do is to start to wrap up here and
finish up the day. One of the questions I
think that was posed by the FDA was what
kinds of things can the FDA do to actually
support the establishment of a third party
entity and service. I think when we consider
the FDA, there's really two roles that we
see. One role is really as a user of a
system. FDA will be a point node if you will
in the contemplated electronic information
exchange and so there's specific requirements
and those requirements are going to actually
represent diverse points of view across the
FDA. We also though see that the FDA as a
regulator is very much involved in the review
of the system. They will be reviewed in the
validation activities associated with the
platform. They will be reviewing the
processes that are being used by all the
people who use the system. There's a
NEAL R. GROSS
question of that guidance from the FDA also
being consistent in terms of how it gets
provided. In both of those areas what we
would recommend and what we would like to see
is that the FDA speaks with one voice. And
this is important in terms of how this comes
across from a standards perspective, but also
in terms of what it communicates to the body
at large. I think industry does not want to
have to deal with different points of view if
you will and the extent to which FDA can help
to facilitate that by having coordinated
positions and bringing those forward I think
would be very powerful.
Another area I think where we
think FDA can contribute here is really in
recognition of some of the stakeholders.
We've talked about patients as being one of
the stakeholders ultimately when you talk
about the efficacy and safety of a product,
but one of the - if you look at where a lot
of the source data comes from, it's from a
medical practitioner who's performing and

supporting clinical trials, collecting
information and validating that information
is reasonable and rational and that they
actually performed or administered in
accordance with the protocol. By the same
token though, those same medical
practitioners are also involved in other
health-related activities over and above just
clinical trial support. They're dealing day
to day with patients. To the extent to which
you can provide a service that supports both
those medical practitioners in the clinical
trial realm as well as in their day-to-day
activity realm would be very powerful in
terms of getting them to buy in. If I looked
at stakeholders today and I said well, who
really represented the medical practitioner,
I didn't see that voice really here as part
of the comment. And we think there's a good
opportunity for some of that discussion as
Dr. Cothran talked about what's going on with
NEAL R. GROSS
the NHIN. There's opportunities to leverage
that existing investment that's going on at a
national level to say what additional
functions can be taken advantage of and
leveraged for some of the contemplated
electronic exchange here.
The final point I think we'd like
to address is this whole question, and I
think it's come up several times today, of
mandates versus incentives. If we go back
and look at MedDRA as a model, if you look at
the data in terms of adoption rate from when
we started on day one with the MedDRA MSSO in
distributing the terminology back in `98 and
where we are today, you know it's gone from
zero to 1,500 subscribers. But there's been
some very significant spikes if you will in
adoption, and those spikes corresponded
almost exactly with mandates from the
Japanese Ministry of Health, Labor and
Welfare that people would use the MedDRA
terminology as part of their submissions.
Also corresponded to the similar mandate from

the E.U. in terms of the European Medicines
Agencies requirements. So I think mandates
have a role here in pushing adoption. I
think I have to agree with some of the
comments that were made today though that in
mandating the use of electronic submissions,
the FDA should do that in the context of a
window, a timeframe in which that should
occur. It should also exist with a waiver
process for those organizations that for
whatever reason are not able to actually
accomplish the transition in that timeframe.
At least they can establish a plan by which
that will happen. We've used that very
effectively in some of the activities we've
done that I've been associated with with the
SAFE-BioPharma Association in trying to bring
in certain providers to support credentialing
for a lot of the pharmaceutical companies.
On the incentive side we also see
that incentives can play a very strong role
NEAL R. GROSS
here. To the extent that there's a business
case I think that helps a lot of the large
pharmaceutical and the smaller pharmaceutical
companies to participate, but we also see
that incentives play a very significant role
here in terms of can you reduce - you know,
if you submit electronically do you get some
benefit in terms of somehow your application
or submission goes to the head of the queue.
I think those types of activities would be
very important to facilitate adoption and to
promote it. I don't see that you know
necessarily that the time for review is going
to go down with an electronic submission
platform. I think more time, you know the
time box is the time box and that will be
used for quality purposes rather than for
sorting through paper purposes and so you
will get better results. And to me that's
the ultimate goal everybody wants.
To sort of wrap up, in terms of
the question of the concept and the
feasibility of a electronic platform

facilitating information exchange, we think
that's very feasible and it can be
demonstrated. We also think that this whole
question of - I think Bill coined the P3, the
public-private partnership term. We very
much think that that's required for success.
All the organizations do need to participate
and all the stakeholders do need to be
represented. And the structure I think, as
Harry talked about such as for MedDRA where
there's actually a structure whereby
different points of view can be accommodated,
but also no single point of view necessarily
is pushed over another.
And then finally you know is it a
single entity that should do this or multiple
entities. We're of the opinion that multiple
entities are required. You may start off
small with a single entity because there's no
basis if you will from a scalability and from
a business case to say we need more
NEAL R. GROSS
providers. But at some point if for instance
this is mandated, then I suspect you would go
very quickly to multiple entities to support
this. If it's not mandated and has to grow
slowly I think you'll start with a single
entity and that will lumber along for some
period of time. Again, I think there's
analogs that we've seen out in industry where
that's been the case. And with that if
there's any questions.
DR. WOODCOCK: Questions from the
panel? Go ahead.
DR. NARDINELLI: Yes. I'll work
backwards. It's not too surprising that when
you get a mandate you would get a spike, but
would you say that that's had much effect on
your overall trend? I mean are we talking
about just a general upward trend with an
occasional spike, or are the spikes, the
mandates, driving the upwards trend?
MR. FISHER: What we've seen,
generally speaking, is it would look like
this. If you looked at the trending it would

go.
DR. NARDINELLI: Right, that's
what I'm picturing.
MR. FISHER: So it wasn't -
there's no step function activity.
DR. NARDINELLI: It's not a step.
So it's really you know, it could well be
described that it's just speeding up what's
happening.
MR. FISHER: Right. And what you
also see in that regard too is that we do see
even today, we see folks drop out. But we do
know generally for a fact, I mean we can't
say for certainty, but any organization
involved in drug development has a MedDRA
subscription now. So I mean that's not
really up for debate at this point. And so,
but the growth period is pretty much over
unless we're talking, which we are, about
going into the other geographies.
DR. NARDINELLI: And then I have
NEAL R. GROSS
one other question. You have actual
experience with falling costs. What is the
source? Is it happening at the platform
itself, or is it at users becoming better at
using?
MR. FISHER: Well, it's
interesting. From the MedDRA perspective,
which again it's the one that's out there,
it's a live one, is that the users themselves
don't actually do a lot in terms of the
MedDRA process. They take it and they
consume it and what will typically happen is
that in a major or smaller or solo
practitioner, they simply have - they can
interact with the MSSO and then they get the
information they need. But what we've
managed to do is we've gotten the process
down to where the expectations and what we'll
do and how fast we'll do it and then also
when we will update the standard, depending
on what the basis is, we've managed to get
all the stakeholders on the same page so we
don't chase that 20 percent with 80 percent

of the cost anymore. That's really where
it's come out. And so we've learned how to
do this better and so the systems underneath,
I mean we've commoditized those. So the
technology underneath, the platform, the
hosting, all that, that's being commoditized.
DR. NARDINELLI: Okay, thank you.
DR. WOODCOCK: Additional
questions? Ken?
DR. BUETOW: Could you drill down
a little bit deeper for how you all worked in
public-private partnership and implemented in
the Federal Health Information Exchange with
the VA and the Public Health Information
Network? Because not to take anything away
from the NHIN, but that feels much more
conceptual yet to me, whereas I know you guys
actually have boots on the ground and there's
actually working systems in PHIN and Federal
Health Information. So what is the public-
private relationship, how is it deployed, how
NEAL R. GROSS
do you actually deal with the delivery of
services?
MR. COTHRAN: Well, both of those
two are very different programs. The
FHIE/DHIE is a longstanding program. It's
been up for about seven years now in
operation and as you say, where NHIN is still
very much conceptual in the pilot stage, FHIE
has been operational. That whole program
came about with the VA recognizing that they
needed an exchange of information. The real
need was movement of military personnel from
the DoD into the VA system and a need for the
VA to exchange among its different clinics
and hospitals. The VA established the
program itself, identified contractors to
develop that system and there is a single
system that's put in place at the VA, largely
operates itself now. We support that through
technical refreshes, et cetera, but that's
primarily a VA-owned system. Go ahead.
DR. BUETOW: So basically in that
model it is a single system. It is a single

contractor who's actually executing and
running that particular facility.
MR. COTHRAN: That is correct.
Now, as it moved to bidirectional, now we're
looking at two agencies interoperate with
each other, with the DoD and having a
completely separate architecture than the VA
system and needing to interoperate among
themselves. It turns out that we are now the
single contractor that supports both of
those, but that happened through an
acquisition. And the truth of the matter is
is it was two separate organizations, two
separate contractors that needed to work with
themselves and two separate federal agencies
that needed to work between each other in
order to bring that about. And that was
established before the acquisition, so it
wasn't the acquisition that facilitated that.
Now if we move to the PHIN, the
PHIN is also - it's hard to say when the PHIN
NEAL R. GROSS
started. Often you'll hear people talk about
two or three years ago is when the PHIN was
actually established, but the underlying
systems that make up the PHIN today, many of
them are longstanding. Our support of the
PHIN was again in a very tight partnership
with CDC. CDC owns that network. We did not
develop it for them and hand it to them, but
it was a give-and-take among both of us.
That doesn't mean that while CDC is managing
the project that they also architect it,
design it, et cetera. We had architects and
designers that were part of that. We would
help establish policy. We'd help establish
standards that were associated with that.
And we weren't the only contractor there. As
Harry mentioned, we are the largest and we
support everything that the PHIN does, but
there are other contractors that are part of
that as well to produce a single product, a
single integrated family of applications, but
working with a number of different
contractors that each interact with each

other and with the CDC to bring that about.
Now if you give me just a second
to talk about the NHIN, it is a very
different construct than this. And what HHS
is proposing there and what ONC has been
promoting is that there is an environment
there not for a single solution, but for
multiple solutions that need to interoperate
with each other. And they started that by
awarding four contracts to four different
vendors for only one of them and having them
independently establish solutions that would
meet the needs of specific organizations that
might want to connect. Because all of those
organizations may have different
philosophies. And as we move into next
year's activities within NHIN, you're going
to continue to see independent solutions
there that will probably interoperate with
each other. I think the model there that you
should be thinking about is how cell phones
NEAL R. GROSS
work, that there are multiple vendors, but
whether I am a Cingular or a Verizon
customer, I can still call you and it's
because those vendors taught the standards
necessary that allow them to interoperate
with each other. And that's how I believe
NHIN is going to grow up, so it will look
different than FHIE will.
DR. WOODCOCK: Any other
questions? All right, thank you all very
much. This concludes the scheduled speakers
for today's hearing. Are there any members
of the public who would like to make some
comments at this point? If you could take
the microphone there and identify yourself.
MR. BARD: Good afternoon. My
name is Bob Bard. I'm with Aastrom
Biosciences. We're a small biotech company
in Ann Arbor, Michigan. I know we try to get
some small industry conversation, but as the
panel reviews what they've heard today, and I
know it's a lot of information over the last
eight hours or so.

We stand at about 16 people.
We're in cell tissue and gene therapy. We're
a cell company. The ECTD which was talked
about in great detail is a very rigid kind of
document and for us to try to fit in there
has been very difficult. In fact, we started
and then pulled back because it just didn't
fit. When you listen to some of the people,
especially Edward from PATH, he's a good
proponent of this, but he misses part of the
things that I have had conversations with him
about. We're a small company. He's got
maybe five or six or seven or whatever number
of people. In my whole regulatory department
there are two of us. I have a clinical
department of three. When we start looking
at these kinds of things, it's very, very
difficult. We're very much in favor of
electronic documents. In fact, the mandatory
part of it is one of the ways I am going to
push it higher and higher. Think about this
NEAL R. GROSS
in great detail though. There are costs in
personnel training, cost of acquisition of
software, cost of acquisition of hardware.
There's a multitude of sins involved in this
that haven't been brought to light in this
meeting and it's not just ECTD. What I came
here for is I though this was a global
discussion of electronic submissions. I
remember being at the L.A. District Office
I'm going to guess about maybe eight or nine
years ago. We were talking about
registration being all-electronic. That
hasn't occurred yet.
I'm in great favor of making as
much of this electronic as possible, but for
small companies, implementation is very, very
difficult. What we have to learn as we go, I
have a very big distrust of the vendors that
supply this software. I don't know if
they're going to be around tomorrow and I
spend anywhere from $20,000 to $100,000, for
a small company that's a big expense, and
then I have to hope that they're going to be

with us. As you go through this, I'd like to
see some way of getting this down to a much
simpler style. XML is fine. Does it have to
go through a vendor? Something that could be
more generic, something that would allow us
to work. I remember this started off as an
Acrobat system where we had an Acrobat
backbone, an Acrobat document. We've gone to
- and it's much more difficult. Can we get
to something that works in XML that anybody
can pick up and work with it easily because
this is getting to a point where you're going
to want us to do this and I think as soon as
the FDA mandates it, EMEA is going to mandate
it, HAN's going to mandate it, everybody that
utilizes the ECTD format is going to fall in
line and we're going to have electronic
submissions as the requirement.
I applaud this group coming
together, at least giving us an opportunity
to listen, but I think there's a lot more to
NEAL R. GROSS
be had here. Taking it all the way and
getting rid of paper is a great idea, I just
hope we can do it at small companies because
when we start off small we have big hopes.
Thank you.
DR. WOODCOCK: Can you stay at the
microphone? Does anyone on the panel have
questions? No? All right, thank you very
much. Other members of the public? If you
can come up to the microphone and identify
yourself.
MR. BRENNAN: Michael Brennan from
Centocor, J&J. There's been a theme that's
come through a couple of the presentations
that first started with Al's talk early this
morning about do we have the right people, do
we have the right skills. I think most of
the responses, most of the discussion really
focused on the technical aspects of, you know
the staring at a computer and hitting the
right keys. But I think what we have to
recognize is the marriage between people that
have the content knowledge, the regulatory

knowledge merging with the information
processing, it's a very transformational
change that needs to happen within our
processes.
Randy, you challenged in terms of
what's the impact on our business processes
and if you're going to move to fully
electronic, we need to have that marriage
within our companies. We need to have people
to focus on the role of being an information
manager. Not an IT manager, but a true
information manager.
DR. WOODCOCK: The younger
generation provides us some hope. Couldn't
resist one editorial comment. Any questions
from the panel? Thank you. Are there any
further comments from the audience? Yes.
MR. KUBECK: Hi, I'm Bill Kubeck.
I'm from Steeple Laboratories. We are a
small pharma company, second tier. We just
migrated from third tier to second tier.
NEAL R. GROSS
Very proud of that. We are 150 years old and
we're really good at paper.
And a number of comments that were
made in terms of the business case on going
from paper to electronic. Paper, I get the
question all the time. We've got the papers.
We can just continue to do it. Migrating to
electronic submissions is - there is a
significant investment in terms of the IT
infrastructure. The total cost of ownership
can be quite great, three to ten times the
original amount in terms of acquiring the
actual technology to do so, in terms of just
corporate personnel, in terms of training.
Steeple Laboratories are great
supporters of standards. We're involved in
the creation of standards and we want to go
fully electronic. But we need to also
understand, we file, we have branded
pharmaceuticals, we have generics, we have
OTCs, we have cosmetics, and by the way, we
register our products in a hundred different
countries with all these different standards.

The issue for us is how can we effectively
manage our submission information to all
these different regulatory agencies and all
these different standards. Now if we just
even look within the United States there are
a multitude of standards depending upon which
division we're talking to. Question to the
panel is when can we expect to have an
integrated standard for submissions?
DR. WOODCOCK: Well, under a Part
15 hearing you're not permitted to ask the
panel questions. However, I will respond to
that question. The FDA has formed a cross-
agency bioinformatics board and we are
engaged in this very moment in developing the
plans for converging our standards across all
the different FDA areas. And that does not
mean, as we've just heard from you, that the
hundred regulatory agencies around the world
are going to fall in line with us, but if we
have a strong and we work in a public-private
NEAL R. GROSS
partnership I think towards consensus
standards we can go far that way. But your
point is well taken. We recognize that.
We've heard that actually from many other
speakers today.
MR. KUBECK: One last point in
terms of migration. We are not ECTD-
compliant. We're making the full transition.
Now comes the issue in terms of getting the
public awareness of RPS. And I would like
your thoughts in terms of that transition. I
can't go, you know again, for my senior
management to embrace going to electronic
submissions the question they're asking us,
ECTD right? And that's the question mark.
DR. WOODCOCK: I think as people -
please submit to the docket issues you have
around those particular timing and
communication issues that would be very
helpful. Did anyone on the panel have
further questions on these? Thank you very
much. Yes, from another volunteer.
MS. AZEVEDO: Good afternoon Dr.

Woodcock and the panel.
DR. WOODCOCK: Can you identify
yourself?
MS. AZEVEDO: My name is
Antoinette Azevedo. I'm President of e-
Submissions Solutions, a professional
services firm assisting companies in
selecting and implementing and validating
document management and publishing. I also
provide submission publishing services.
Point one. I tried to comment to
the notice that was sent out and the web link
didn't work.
DR. WOODCOCK: That's I think been
addressed. We're sorry about that.
MS. AZEVEDO: Okay. Secondly, I
work with companies that are sometimes prior
to the IND phase. And I've worked with them
for the last six years. I've been in
business about six years. One of the factors
that prevents companies from starting INDs in
NEAL R. GROSS
the ECTD phase is the fear that if they start
ECTD, always ECTD and that will prevent them
from licensing out products because the
business model is to be virtual license out
or sell out and they're afraid that that will
be an impediment to doing a deal with a
partner. So that's one interesting factor.
So if ECTD were mandatory throughout the
lifecycle, this would remove an impediment to
starting in the IND phase.
The other issue is a training
factor. I, because I provide submission
publishing services, I have a boutique
operation and oftentimes go into the customer
environment and work on the systems that the
customer has already purchased and validated.
And the customers can't find people to
operate their own systems so they bring in
consultants.
Secondly, in my practice I have a
need for consultants to come and help me
execute my projects and it's impossible to
find people of the right skills to perform

the functions. And the skills are not just
the regulatory skills which was - is really
critical, it's the computer skills. It's
also the knowledge of the applications, how
to use Microsoft Word to go to PDF, how to
troubleshoot problem PDFs. Recently we used
Global Submit to validate a submission I was
working on, and this is a vendor's system
that has been used to produce many ECTDs
submitted to the FDA including pilots. And
we found lots of issues reported by Global
Submit that were surprises, including issues
with the standards themselves. That is,
there are issues with the ECTD, DTD and with
various lists and so on that caused false
positive issues to be reported by the
validations. This whole set of things are
big impediments, but if there were mandatory
requirements with an appropriate timeframe
and outreach by the agency to especially the
small, the tier three and tier two companies,
NEAL R. GROSS
really critical to making this a success.
DR. WOODCOCK: Thank you. Any
questions? No. Any further comments? I
think we may have time for one last comment
here. Yes.
MS. PALLA: I'm Madeline Palla.
I'm with the Animal Health Institute. And I
just wanted to comment that the animal health
industry is often slightly CDER and CBER in
electronic requirements and we monitor these
issues closely and it's going to be very
complicated in the animal industry to know
and conceptually to give you an idea.
Listening to the speaker today talk about you
know having a notepad laptop to, you know. A
veterinarian out of the office is not going
to be able to utilize that.
And I just wanted to, I cannot
speak necessarily to the costs personally as
I am with a trade association and not a
particular sponsor company, but a lot of our
animal health companies are divisions of
larger human pharmaceutical companies and so

it may not be as great of an impact to those
companies that are already implementing these
requirements in other areas of the
corporation. But for some of our animal
health companies that are solely animal
health it's going to have a large impact.
And we do plan to answer the other questions
in the notice in our comments that we submit.
Thank you.
DR. WOODCOCK: Thank you. All
right. Is there one more person who wants to
contribute? All right. We have a few more
minutes here.
MS. VATLEIT: Thank you. Hi, I'm
Stephanie Vatleit. I'm with Animal Drug
Alliance and I would mirror in large part the
AHI position. We are cautiously optimistic
about electronic submissions, but we sense
some real hesitancy on the part of CBN to
work with us on those submissions and a
hesitancy on the part of our member companies
NEAL R. GROSS
to work through the business case. We've
heard a lot about that today and it is really
important to the bottom line as there are
lower margins on the animal health side as
you might expect.
And we would also like to
acknowledge that it's not only a burden to
get the software and hardware necessary for
making electronic submissions on the part of
the sponsors, but also on the part of the
centers. And I think that's a real concern.
DR. WOODCOCK: Thank you.
Questions? No. All right. I would like to
reiterate first that the comment period
remains open. We hope those of you who have
benefitted today from all the comments, it
should inform your comments to the docket.
We hope for a robust set of comments that
will allow us, NIH and the FDA to move
forward in our various venues. I want to
thank the panel very much. I'd like to thank
Paula McKeever for putting this meeting
together. She did a fantastic job. I think

we learned a tremendous amount. And I'd like
to thank all of you who stayed till the end
for your attention. Thank you very much and
this hearing is closed.
went off the record at 4:56 p.m.)
NEAL R. GROSS

Much more than documents.

US Food and Drug Administration: 06n-0464-tr00001

Uploaded by

Document Information

Description:

Copyright:

Available Formats

US Food and Drug Administration: 06n-0464-tr00001

Uploaded by

Description:

Copyright:

Available Formats

Related titles

GOVERNMENT

FOOD AND DRUG ADMINISTRATION

ELECTRONIC SUBMISSION OF REGULATORY

JANET WOODCOCK Deputy Commissioner

The transcript constitutes the

Presentations to the Panel

Ron Celeste, ThinSpring...................55

Tom Klaff, Surety.........................97

Mark Scheineson, Alston & Bird, LLP......138

Debra Bremer, Pfizer.....................150

Diana McKenzie, Amgen....................157

Sue Dubman, Theravance...................161

Bob Beck, Fox Chase Cancer Center........167

Diane Paul, CRIX . . . . . . . . . . . 173

Dan Ruggles, Liaison Technologies . . . 178

Mark Adams, Booz Allen Hamilton . . . . 183

Ed Tripp, Abbott Laboratories . . . . . 211

Bill Rosen, Pfizer . . . . . . . . . . 225

John Rapoza, JRRapoza Associates . . . 253

Mark Rutkiewicz, AGA Medical. . . . . . 263

Jason Rock, Global Submit . . . . . . . 287

Laurie Rose, SAS . . . . . . . . . . . 307

Harry Fisher, Northrop Grumman . . . . 328

Robert Cothran, Northrop Grumman . . . 335

Terence Zagar, Northrop Grumman . . . . 347

Bob Bard, Aastrom Biosciences . . . . . 363

Michael Brennan, Centocor . . . . . . . 367

Bill Kubeck, Steeple Laboratories . . . 368

DR. WOODCOCK: Good morning, I'm

Janet Woodcock. I'm Deputy Commissioner for

Operations at the Food and Drug

Administration and I'll be serving as the

presiding officer at this hearing. On behalf

of the Commissioner of Food and Drugs, Andrew

von Eschenbach, I'd like to welcome you to

this public hearing on electronic submission

of regulatory information and also on

creating an electronic platform for enhanced

With me today on the panel that

will be listening to the presentations are:

Dr. Ken Buetow who is Associate Director for

Bioinformatics and Information Technology at

the National Cancer Institute; Mr. Kevin Fain

who is Associate Chief Counsel, Food and Drug

Administration; Dr. Randy Levin, Director for

Health and Regulatory Data Standards at Food

and Drug Administration; Dr. Randy Lutter,

Associate Commissioner for Policy and

Director for Bioinformatics in the Office of

Critical Path Programs at FDA; and Dr. Lana

Skirboll, Director of Policy and Planning at

NIH. And it's no accident that NIH and FDA

are together on this panel. We have a great

deal of both agencies' interest in these

So first let me describe briefly

the issues we're going to be talking about

today and then I'll review the format for

this hearing. FDA and NIH is interested in

hearing about issues concerning the

feasibility and effect of an all-electronic

submission environment as well as issues

related to the electronic regulatory

information exchange platform. Over the last

decade FDA has been moving toward

transforming all regulatory submissions from

paper to electronic means. For example,

we've issued regulations related to voluntary