You are on page 1of 371



+ + + + +


+ + + + +


+ + + + +



+ + + + +

December 18, 2006

+ + + + +

9:00 a.m.


JANET WOODCOCK Deputy Commissioner


The transcript constitutes the

minutes from the public hearing held on

(202) 234-4433 WASHINGTON, D.C. 20005-3701
December 18, 2006.


Presentations to the Panel

S. Albert Edwards
TAP Pharmaceutical Products...............10

Thomas W. Littlejohn
Piedmont Medical Group....................34

Ron Celeste, ThinSpring...................55

Tom Klaff, Surety.........................97

Nancy Smerkanich
Octagon Research Solutions...............113

Mark Scheineson, Alston & Bird, LLP......138

Debra Bremer, Pfizer.....................150

Diana McKenzie, Amgen....................157

Sue Dubman, Theravance...................161

Bob Beck, Fox Chase Cancer Center........167

Diane Paul, CRIX . . . . . . . . . . . 173

Dan Ruggles, Liaison Technologies . . . 178

Mark Adams, Booz Allen Hamilton . . . . 183

Ed Tripp, Abbott Laboratories . . . . . 211

Bill Rosen, Pfizer . . . . . . . . . . 225

John Rapoza, JRRapoza Associates . . . 253

Ari Kaliannan
Newtech Global Solutions . . . . . . . 255

Mark Rutkiewicz, AGA Medical. . . . . . 263

Presentations to the Panel (cont.)

Jason Rock, Global Submit . . . . . . . 287

Laurie Rose, SAS . . . . . . . . . . . 307

Harry Fisher, Northrop Grumman . . . . 328

Robert Cothran, Northrop Grumman . . . 335

Terence Zagar, Northrop Grumman . . . . 347

Public Comments

Bob Bard, Aastrom Biosciences . . . . . 363

Michael Brennan, Centocor . . . . . . . 367

Bill Kubeck, Steeple Laboratories . . . 368

Antoinette Azevedo
e-Submissions Solutions . . . . . . . . 372

Madeline Palla
Animal Health Institute . . . . . . . . 375

Stephanie Vatleit
Animal Drug Alliance . . . . . . . . . 376

(202) 234-4433 WASHINGTON, D.C. 20005-3701

9:44 a.m.

DR. WOODCOCK: Good morning, I'm

Janet Woodcock. I'm Deputy Commissioner for

Operations at the Food and Drug

Administration and I'll be serving as the

presiding officer at this hearing. On behalf

of the Commissioner of Food and Drugs, Andrew

von Eschenbach, I'd like to welcome you to

this public hearing on electronic submission

of regulatory information and also on

creating an electronic platform for enhanced

information management.

With me today on the panel that

will be listening to the presentations are:

Dr. Ken Buetow who is Associate Director for

Bioinformatics and Information Technology at

the National Cancer Institute; Mr. Kevin Fain

who is Associate Chief Counsel, Food and Drug

Administration; Dr. Randy Levin, Director for

Health and Regulatory Data Standards at Food

and Drug Administration; Dr. Randy Lutter,

Associate Commissioner for Policy and

Planning at FDA; Dr. Armando Oliva, Deputy

Director for Bioinformatics in the Office of

Critical Path Programs at FDA; and Dr. Lana

Skirboll, Director of Policy and Planning at

NIH. And it's no accident that NIH and FDA

are together on this panel. We have a great

deal of both agencies' interest in these

various topics.

So first let me describe briefly

the issues we're going to be talking about

today and then I'll review the format for

this hearing. FDA and NIH is interested in

hearing about issues concerning the

feasibility and effect of an all-electronic

submission environment as well as issues

related to the electronic regulatory

information exchange platform. Over the last

decade FDA has been moving toward

transforming all regulatory submissions from

paper to electronic means. For example,

we've issued regulations related to voluntary

(202) 234-4433 WASHINGTON, D.C. 20005-3701
electronic submissions of regulatory

information which many of you are familiar

with. We've also issued numerous guidance

documents to assist in the submission of

these various regulatory documents in

electronic format. We've also collaborated

with manufacturers, with standard development

organizations, with healthcare information

suppliers and other government agencies to

develop data standards and build databases

for sharing certain clinical trial

information. NIH has also in the last five

years been extremely interested in sharing

clinical trial information on behalf of its

investigators and the research it pursues.

To help us answer our questions

related to these two issues, we published in

the Federal Register on November 21, 2006, an

invitation to interested organizations and

individuals to participate in making

presentations at today's meeting specifically

addressing the questions that were listed in

the notice. These questions were divided

into the two topics. The first set of

questions addressed the transition from paper

submissions to electronic submissions,

related costs, interested in time spent and

how to implement this transition. Now we

recognize there are a broad range of FDA-

regulated products and that they're on a

spectrum of experiencing electronic

submissions right now. Some product areas

such as pharmaceuticals, there's a great deal

of electronic submission. In other areas,

much less to no electronic submission and few

associated standards. And therefore, we

recognize that whatever we do we're going to

have to take a step-wise approach to

implementation depending on the particular

area on where we're starting and that will

probably influence how long it would take to

get there.

Now the second set of questions

relate to the concept and feasibility of an

(202) 234-4433 WASHINGTON, D.C. 20005-3701
electronic platform that would facilitate the

exchange of clinical research information and

regulatory product information and the role

of the public-private partnership in the

creation of such a platform. Our experience

to date has shown that, although we may have

the standards in certain areas for electronic

submission, we don't perhaps quite have the


Fourteen people signed up today to

help answer those questions and we'll hear

from those folks first. When that is done

and if time permits we'll open the floor to

anyone else who may wish to address those

questions. If you're a scheduled speaker, we

request you stay within the allotted time and

I will make sure you do.

Before we go on, let me stress

this is a listening exercise for FDA and the

NIH. We really want to hear what you have to

say on these issues. We are having this

meeting transcribed, we hope, and the members

of this panel and their staff will pay

careful attention to what is said today as we

decide our future course. And I expect that

you can look forward to prompt action from us

on these matters. The direction will depend

in part upon the input we receive today. The

docket will continue to stay open until

February 16, 2007 to receive additional


Now there are several housekeeping

items I'd like to address. Number one,

please restrict yourselves to this room, the

hallways, the stairs if you like and the

lavatories, and do not move otherwise about

the building because it's an FDA facility and

there are other people working. At lunch we

have only an hour. There is across Fisher's

Lane a cafeteria-style public restaurant that

you can go to and there are some other

restaurants down. You go out this door and

go to your left. So B but we have talked to

the folks across the way and told them they

(202) 234-4433 WASHINGTON, D.C. 20005-3701
may expect a larger group today, so hopefully

they will be ready and so we can all

reconvene promptly at the time for the

afternoon. So we have a very busy day and

I'd like to begin with our first speaker.

Okay, our first speaker is Dr. S.

Albert Edwards, who is the Director of

Regulatory Affairs at TAP Pharmaceutical

Products, Inc. He will speak for 30 minutes.

Dr. Edwards.

DR. EDWARDS: Okay, thank you very

much. I'm here today in my current role as

the Director of Regulatory Affairs and

Operations. If I were speaking to you 20-

some years ago, and I'm not going to divulge

the exact number of years, I could say I'm Al

Edwards and I'm here talking to you from the

FDA. So I've kind of sat on both sides of

things and I would say to you initially I

wish in my lifetime early on I had access to

electronic submissions to have done my job.

The greatest enemy I guess that I faced at

FDA in my time was the fact that I would get

to that very, very last volume in the NDA and

then I'd have to go into the documents room

and I'd have to try and find it. Documents

rooms then got closed and secured which was a

good idea at FDA at the time, but then it put

all the documents in the control of the

document clerk and if you wanted your volume

soon, you had to come up with an incentive

plan. And my incentive plan was to always

turn in my volume request with a suitable

incentive, okay? I'll talk more about more

formal incentives later, but those kind of

things were needed.

Back to present day. I want to

share with you eight observations that I have

about electronic submissions. Those span

about the 10 years of experience that I've

had and this next slide will show you what

TAP's experience has been completing about

700 ECTD submissions to date. We did a

complete conversion in September of `05.

(202) 234-4433 WASHINGTON, D.C. 20005-3701
That conversion included all original INDs,

supplements and amendments. The conversion

also included all original INDs and

amendments. And the last thing we did was

convert all of our existing applications. So

we are a 100 percent ECTD shop.

FDA is our customer. I can

reinforce that because I sat on the other

side of the fence many years ago. I'm not

going to take time to read this entire slide,

and Dr. Woodcock, when I get to my allotted

time I know you're going to cut me off. So

everybody has my handout. Overall it has

taken TAP some time to put the ECTD process

in place. I want to emphasize that that's

process and technology on our side of the

fence. I suspect it will be on FDA's as

well. We have some improvements along the

way and that is we intend to control content

and meta-data from authoring through

submissions to archival. We believe that the

electronic tool sets are currently available

for others to manage information and complete

ECTD submissions, as well.

Let me take you on my travels

through these general observations. A couple

of footnotes along the way. Number one, I

recently learned in an important meeting that

adult audiences really won't listen to you

unless you give them about five why's for

everything you tell them. So you will see

"why" pop up on the screen here very often.

That's to convince you or at least open your

mind to my points. Some of my points will

be, `Oh yes, Al, we already know that.

Please go on.' Some of them will be rather

controversial and so that's kind of the span

of where we're going today.

Number one. We, meaning FDA and

industry, should be a partnership, must work

closely together and listen attentively to

each other's concerns. Why? Computers are

fussy beasts and the information must not be

changed when it's moved from one electronic

(202) 234-4433 WASHINGTON, D.C. 20005-3701
environment to another. Two, second why,

again my former life as a reviewer.

Reviewers depend on this information. It's

their day-to-day feeding and care that

happens, to perform their job functions.

Also, the industry depends on this

information submitted to support and maintain

healthcare product approvals. Why again? If

electronic systems cannot talk to each other

successfully and share unchanged information,

the process can be compromised. I'm not

saying it's compromised every time, I'm

saying it can be.

Observation number two. We must

carefully select and closely supervise any

third party that may handle electronic

systems and any information stored, either by

FDA or by us in industry. This is contract

organizations that produce, transmit and

otherwise interface, touch e-submission

information. Why? The lifecycle of a

healthcare product, the development,

marketing of a prescription drug for

instance, and I know there are other products

at issue here, is a long-term process that

could last decades. In contrast to that why,

the lifecycle of electronic systems and

software is much shorter than a healthcare

product lifecycle. Only third parties with

extensive appreciation for and commitment to

managing the disconnect between this long

lifecycle that we have for our healthcare

products and the short cycle for systems and

software should be considered suitable

candidates for handling e-submission

information. For the good of patients that

we serve we need to be in this for the long

haul, folks. It's not just something you can

do on Sunday afternoon. Forget about it.

Again, still on observation two,

why? Disconnects resulting in information

migration and retrofits are costly endeavors.

Migration and data retrofits may be likely

outcomes when inexperienced, low-bid, third

(202) 234-4433 WASHINGTON, D.C. 20005-3701
party contractors are used. Why? This is

not only a dollar cost, it is a time cost.

Migration and retrofits take time. Systems

are down. Delays disadvantage patients or

consumers, whether the delays originate in

FDA or from us in the industry.

Finally, confidentiality issues

may arise due to incidental or inappropriate

release of information. I would let you know

that most of these releases are not due to

the system, they're actually due to humans.

And so therefore staff at third party

organizations have to be of suitable quality

and they must be invested in the security of

the information that we entrust to them.

Very, very important.

Observation number three. We must

jointly learn how to handle and access

repetitive e-submission information.,

Investigator Forms 1572s, and updates more

efficiently and find ways to reduce that

workload. Now I'm speaking about

prescription drugs. That's been my life for

the last, well, let's say 20-plus years.

Let's not go there. But it's been a long

time. And we must find ways to more

efficiently handle that information. I would

say current industry submission systems are

clogged with this information and it easily

accounts for 20-50 percent of an average

month's submission activities.

The current trained labor force,

and I think this was in the Federal Register

as well, from e-submissions available within

the U.S. is not sufficient today to handle

the upward spiral of this workload if we add

more to it. Why? I think we must change and

we must turn our attention to and use our

limited resources to handle more difficult

and challenging repetitive submission types.

My example here is advertising and promotion

submissions. This will require at a minimum

the integration of three-dimensional scanning

and viewing. We all watch TV every day. We

(202) 234-4433 WASHINGTON, D.C. 20005-3701
have camera phones. There are other forms of

video. I think that you know - one of the

folks in my office always asks me, `What

you're saying, can it pass the Washington

Post test?' And I guess I would challenge

all of us, whether we have FDA jobs or NIH

jobs, NCI jobs, industry jobs, if we said, we

do this number of submissions electronically,

but yet we almost have a whole group of

submissions over here, advertising and

promotion, other things, and we don't do

anything for those. We don't enable those in

any way. Would that fly? And I'll just

leave that kind of as an open question.

This brings me to a point I want

to put in capital letters obviously and that

is an e-submission paradigm should be an all-

inclusive one. eSubmissions should be the

Club Med. That's where we're going. We must

learn to use our systems, to use them

repetitively for all submission types, if we

are to maintain credibility with our external

audiences. Commissioners' offices, Congress,

consumers we serve, corporate entities that

we have to report into.

Why again? Repetitive use of the

same tool set for multiple submission types

will drive down the costs - cost is a key

factor in this meeting - and provide some

positive business cases and incentives for

all to participate.

Observation number four. We must

find a common ground to move forward with

making e-submissions a requirement and not an

option. We must provide incentives -

remember that Snickers bar, only now we're

going to go for something bigger - to move

from the current state of approximately 3-5

percent of all submissions sent to the FDA

electronically to a 95 percent level very

soon. 2009 is my example.

Here I'm going to depart from the

why's. I'm going to give you a few proposed

incentives. Incentive number one, reduce the

(202) 234-4433 WASHINGTON, D.C. 20005-3701
PDUFA fee for those submissions requiring

fees by some percentage or amount when a

submission comes in in eCTD format.

Incentive number two, reduce the PDUFA review

clock by some number of days for applicable

submissions in eCTD format. Proposed

incentive number three, grant a tax credit to

those in industry, industry entities that

have ongoing e-submissions programs. These

incentives are my opinions based on 10 years

of working experience with e-submissions.

They do not represent my prior experience at

FDA, nor do they represent my current

thinking of my employer, just so you know

that. But I think we need to consider these

seriously and we need to be creative about

what it is that will move us from - I think

there are 200 people in this room. I think

the next time we meet we should have about

2,000 to hear this.

We must also find ways to enhance

and support FDA offices germane to e-

submissions. FDA Office of Business Process

Support and the FDA staff that currently

handle the electronic gateway. This has got

to be a requirement on the way to the 95

percent goal. There are great people out

there. They do tremendous jobs. We at TAP

would not be able to be where we are if it

wasn't for these offices. And I want to

commend people that put these folks in place

and the work they do every day. They have

clarified many things for us so that we can

move ahead and convert to all-eCTD.

Observation number five. Common

ground must include a master plan in which we

can all invest. This plan should include

what? Things like what are we going to do

about E2B, SPL and the acronyms can go on,

folks. I could spend another half hour on

acronyms if I could remember, okay? If I had

enough RAM up here to remember them all.

When? We need to get serious

about the order of adoption of new standards

(202) 234-4433 WASHINGTON, D.C. 20005-3701
for e-submissions. The investment requires a

significant buy-in on the management chain on

both FDA's side and within industry.

(Sound system interruption)

DR. EDWARDS: Stay tuned for

further interstellar travel. We may be

leaving at any moment.

Why? Currently, electronic

systems deployed for the most part are highly

customized, configured to meet individual

needs and expectations. We can't continue to

afford this diversity. Taxpayers can't

afford it, we in industry can't afford it,

it's just too expensive. The electronic

silos that we've created by our current

approaches will soon be obsolete due to

evolving technology. There's no better time

to make a master plan than right now, today.

There is no better time.

Why? And you'll see, I sort of

got tired as I got to observation five

because there are fewer and fewer why's. The

time interval grew much shorter so I couldn't

get all my why's in here. Standards

development organizations are and will

continue to produce more and different types

of proposed standards for our consideration.

We need to select those standards which will

yield maximum benefit in an orderly manner.

Again, a master plan is not something that

can't change, it just needs to have an

orderly process we can all invest in.

Observation number six. The

master plan must include a schedule to

control lifecycle development process.

Before I put this next piece up, I am going

to say that SPL is a very good thing that

happened and I don't want what I've put up

here to be in any way negatively interpreted

about SPL itself. I'm only using that as an

example and as you can tell I now have some

nice background music to lead into this

controversial statement that I'm about to

make. Thank you for that intro.

(202) 234-4433 WASHINGTON, D.C. 20005-3701
Why? One merely needs to look at

the impact of structured product labeling

around October of 2005 for a justification

for this. At least in our shop there were

two additional releases of software

corrections and changes that were made in

November and December. For those with

ongoing eCTD programs, this triggered

multiple change requests and re-validations.

This took some other people out of the work

queue in our shop to deal with this. These

were unscheduled.

Why? The inclusion of SPL was not

compliant with ICH eCTD requirements. Thus

the eCTD software providers had to go outside

the confines of ICH to produce a fix. So we

had to violate one standard in order to make

another one work. I think we can do better.

The disconnect between the SPL and the ICH

eCTD requirements to me would have become

evident had there been adequate pilot testing

and we could have considered some appropriate

alternative measures to move that forward.

Observation number seven. We must

include a Plan B, not to be confused with the

other Plan B, okay? This is a computer

process now, not another process that also

was Plan B, in our master plan. Why? We

must have an alternative in this master plan,

a Plan B when the electricity goes out or a

disaster strikes. I have with me in this bag

a CD-ROM. I also have my little thumb drive.

If the presentation wasn't present here,

maybe I could go to Dr. Levin's computer, we

could rig this up and I could give my

presentation on a - you know, continue.

Because I have backup. And so for any plan

we have, we've got to have backup. The best

example I can give here are prescription 15-

day safety reports that we need to keep

turning in, whether the lights are on in my

shop or whether they're on in Washington or

not. I tell this story often, that during,

not an eCTD NDA, but an eNDA, about five or

(202) 234-4433 WASHINGTON, D.C. 20005-3701
six years ago a little squirrel gave up his

life and took us completely out of publishing

because he decided to jump across this

generator grid about two miles from our

facility in Lake Forest, Illinois, and that

took out the entire grid for about four

hours. So you need to have a Plan B. We all

need to have one.

Why? We must continue to perform

our jobs when electronic systems are not

available. When an outage occurs, we need to

go on, get things done. Why? It's just good

business to have a disaster recovery plan for

these critical electronic systems. Each

party, again, industry and FDA should know

when the other party is down. We shouldn't

be trying to send electronic systems in if

FDA's systems are down. If our systems are

down, we should be notifying FDA that they're

down and why they're down and how long

they'll be down.

Observation number eight. And now

you can tell I really got tired because there

are no why's to this one, okay? But we must

bring the correct focus and attention to the

required infrastructure surrounding e-

submissions. That's not only technology, but

it's process. We must find a way to get the

same level of attention and appreciation for

e-submissions as is currently accorded the

content of a new drug application. Again, my

former life as a reviewer, I know how

important NDAs are and there has to be a

match now between meta-data and how it gets

transmitted and the content. If you don't

get all the content because the meta-data is

screwed up, it doesn't matter and that

reviewer needs access to everything.

So far, Dr. Woodcock, you have not

cut me off. Okay. I'm going to quickly go

through the conclusions. We must quickly

work - I'm sorry, we must work together. I

must work quickly. Carefully select third

parties. We must learn how to efficiently

(202) 234-4433 WASHINGTON, D.C. 20005-3701
handle and access repetitive information. We

must find common ground to move forward in

making e-submissions the requirement. The

common ground must include a master plan.

That master plan must include a controlled

lifecycle development process. The master

plan must include Plan B, and meta-data and

infrastructure must equal the content of e-

submissions in terms of importance.

To whom much is entrusted, much is

expected. I don't know who the author was,

but it's a great thing. Or, we must all hang

together lest we all be hung separately.

Benjamin Franklin said that. As far as I'm

considered it's truly time to put away our

perceived differences, move e-submissions

from a 3-5 percent level of participation to

the 95 percent level in the next two years.

I want to thank FDA, thank you Dr. Woodcock

for allowing me to speak. Thank you audience

for your kind attention.

DR. WOODCOCK: Thank you, and do

the panelists have any questions for Dr.

Edwards? Randy.

DR. LEVIN: Yes, I have two

questions. One regards the incentives. Is

the candy incentive out of the question?


DR. EDWARDS: No, we just need to

talk to Mars, the candy company.

DR. LEVIN: My question is about

your comment about our labor force is not

sufficient. What kind of skills are not

sufficient? Can you expand on that a little


DR. EDWARDS: I think that - and

Randy, I'll just give you the experience in

my shop. People need to be able to pay

direct attention to a computer screen for 8-

plus hours. That's the first thing they need

to do. They need to be extraordinarily

vigilant in terms of what they see on that

screen to see if there are any errors or

omissions so that we can produce a viable,

(202) 234-4433 WASHINGTON, D.C. 20005-3701
quality product that you or your reviewers

want to look at. Those people are not

necessarily PhD's. Some of them are high

school graduates. For every, let's say, four

people that I bring into my shop, usually two

of them survive because it is a unique skill

set of someone who will pay that much

attention to a computer continuously. I'll

admit I can't do it personally. I can't

spend eight hours of my day staring at a

computer screen, but that's what it takes to

put an eCTD together. And so we just need to

be conscious of the fact that as we move

forward, we don't have good parameters to

pre-test people to understand who can do a

good job and who can't.


DR. BUETOW: Implicit in your

presentation, which I should thank you is

very useful, is the need for this

infrastructure to be comprehensive, but also

the need to actually have a master plan for

its deployment, if that's a fair

interpretation. But I was curious. Can you

give us any insight - this is obviously huge

- where would be the sweet spots to hit some

of these repetitive tasks that you've

mentioned? What would be high-priority


DR. EDWARDS: High-priority

activities, I can't remember which slide it

is, but I think we have to go to advertising

and promotion submissions. I know that's

very controversial, I know there's a lot of

resistance there. Before I came to the FDA I

worked at MD Anderson Hospital in Houston on

the wards. The way the hospital

administrator let me on the wards to do drug

monitoring as a pharmacist because of course

they believe a pharmacist couldn't do it. He

said I'll give you the most difficult ward

first. You prove to me you can do it there,

you can go anywhere in the hospital after

that. So I guess, and it reflects a little

(202) 234-4433 WASHINGTON, D.C. 20005-3701
bit of my personal opinion. I think we need

to do the difficult stuff first to prove to

people that we can move on.

DR. WOODCOCK: Any other questions

from the panel? Armando.

DR. OLIVA: How long did it take

your company to move to the all-eCTD

environment once you made that decision?

DR. EDWARDS: We carefully

monitored the eCTD both from the point of

view of guidance documents and available

software providers from July of 2000 through

July 2003. We made a commitment at that time

on paper to move. We produced our first test

disk for Ken Edmunds and all these other

wonderful people that work in business

process support. May of `04 we produced our

first NDA in eCTD format in `04. And my

boss, who's also a very big supporter of

this, came into my office and said so you've

done the NDA. So now show me you can do

everything. So that's what we had to do in

2005. So that's kind of our timeline. I

think there are probably other experiences in

this room that I'd refer you to.

DR. WOODCOCK: Other questions?

Thank you very much, Dr. Edwards. Our next

speakers are Dr. Thomas W. Littlejohn,

Medical Director of the Piedmont Medical

Group and Mr. Doug Pierce, President of

PeerMed, Inc.

DR. LITTLEJOHN: Thank you, Dr.

Woodcock. Good morning. Thank you for

allowing my colleague and I the time to

address this panel. My name is Tom

Littlejohn. I am the Medical Director of

Piedmont Medical Group which is an

organization of independent clinical research

sites in the two Carolinas and Tennessee. I

have been involved in clinical research for

20 years and have functioned as a principal

investigator on over 300 trials. My

colleague is Doug Pierce, President of

PeerMed, Inc., a healthcare-formed company

(202) 234-4433 WASHINGTON, D.C. 20005-3701
located in Virginia and North Carolina.

We are here to address the issue

of electronic data capture, storage and

transfer as it applies to clinical research

sites. We applaud the efforts of those who

are working to convert the traditional paper

model into an electronic model. This

transition is picking up momentum and has

been clearly shown to reduce costs, improve

quality and accelerate the process. All of

the activity to date, however, assumes that

with the exception of large academic centers

with EMR, that the original data, the

wellspring, if you will, at clinical research

sites will be captured on paper and then

transferred to a case report form, whether it

be paper or electronic. These source

documents are sacred to the integrity of the

trial and must be accurate, accessible and

protected for many years. They must remain

under the control of the principal

investigator and free from manipulation or

tampering of any kind. They cannot be

reproduced. For this reason they must remain

at the investigative site and review of them

requires that the reviewer travel to the site

to verify the data and its integrity.

We are proposing a technology

whereby the initial data can be captured

electronically in the same manner as it is

currently on paper and that the data can be

validated, verified and protected more

effectively. Additionally, this digital

source data could be reviewed by monitors,

regulators or any other authorized party from

a remote location. Adoption of this

technology would close the final gap that

otherwise would continue to exist between a

completely electronic environment and a paper

one. I'll turn it over to Doug.

MR. PIERCE: I'm in Dr. Edwards'

presentation. Which I'm going to do it

again, just in case you didn't get it.


(202) 234-4433 WASHINGTON, D.C. 20005-3701
MR. PIERCE: Bear with me here one

second. While I find this, I'm going to let

you know that I did not attend the why

meeting and I have no why's. Thank you. Can

you all see? I can't see, I'm at the podium.

I didn't hit the why meeting. However, I was

at a meeting that said that when you start

these kind of addresses, you should blaspheme

famous authors. So for those of you who have

read Shakespeare's Julius Caesar, I'm sorry.

We have gathered here not to praise paper,

but to bury paper.

(Sound system interference)


MR. PIERCE: But this isn't the

first time we've gathered together to bury

paper and paper is proving to be puzzling. I

want to talk about the puzzling persistence

of paper. And I think the photograph below

the title says a lot. And it says, it shows

what we do maybe every day. We sit at a


DR. LITTLEJOHN: We've got a

keyboard, a mouse and we shove them aside to

grab a legal pad and a pen. And you notice

that the person is starting a list. We have

three very different ways to input data. And

it seems to me that we have to examine the

power of paper and the paper interface and

understand it if we're going to in fact bury

it. So we'll be talking about the puzzling

persistence of paper today.

Benefits of electronic data

capture we all know very well, especially in

this room. We have rapid access to data. We

have higher quality of data. We have fewer

queries and faster resolution of the queries

that we do get. We have better tracking of

adverse events, less time to data log and of

course substantial cost savings. And

obviously those are the reasons why EDC is

gaining momentum in the market and that's a

wonderful thing.

However, there are barriers to

(202) 234-4433 WASHINGTON, D.C. 20005-3701
adoption. The rate of adoption is not as

fast as we would like. It's not as fast as

many people thought. There are technological

constraints that are holding us back. For

example, if it's a web-based EDC we have

internet access or do we not have internet

access. Do we have broadband? Is it dial-

up? Is it every other day? You know, that

sort of things. We have connectivity issues

within a research site. There are hardware

and software deployment issues involved. All

those things limit and slow the growth of


There's also personnel constraints

and I almost said behavioral constraints

because I think that's probably a better

word. We have attitudes toward work and work

flow and what traditional responsibilities

have been and which will change when EDC

systems are brought onboard. There's a

certain level of technological

sophistication, of savvy, of comfort that EDC

systems seem to demand. But the interesting

question to us is even when the technological

constraints and the personnel constraints

have been met, in the very best setting why

is it the paper source docs are still being

used? Why is it that even with some of the

best EDC systems you can get the coordinator

initially captures the data on paper? Well,

to understand that we have to understand the

power of pen and paper. If it was a computer

system, paper and pen, we would say the

modern paper and pen interface is the result

of 5,000 years open source global

collaboration. It's the most well developed,

shared, open source way to gather data ever.

The early data wasn't good. You've seen it

on the cave walls, but we quickly moved from

the rock and the chisels to paper and we've

been developing it forever. And almost every

human being on the planet knows what to do

when you hand them paper and a pen or pencil.

So it's been around a long time and it's part

(202) 234-4433 WASHINGTON, D.C. 20005-3701
of us. It's the original

graphical user interface. It's flexible. If

you forget something on your list, you can

add it. I could have added "wise" to my

thing here, but I can't now. It's familiar.

It's friendly. No one's intimidated. When

you hand a legal pad to somebody, they don't

go oh, I don't do legal pads.

You don't have somebody looking

around trying to find their legal pad like I

was doing. So it's very friendly, familiar

and it works very, very well. Now, let's

take some of these ideas and look at paper

source documents. Paper source documents

allow for structured and free-form entry of

data. In other words, the paper source docs

that you've made following the protocol have

blanks and check boxes for structured data,

data that you know the protocol asks for.

But the beauty of paper is if you need to add

something on the fly, you just write a

marginal note. You circle. You sign. You

can add data on the fly. They're simple.

They require minimal training. Data capture

on paper source docs is a very unintrusive

thing. Coordinator and patient are there in

a room. The coordinator is taking data down.

No one has to turn around and type in data.

There's no machines to wheel in, et cetera.

And the most important reason we think paper

source documents have been so hard to kill is

that there's just a lot of very important

data, important for the case history,

important for the trial that is not a data

point to be entered into the data management.

It's not even and it shouldn't be on the

electronic case report form, but you've got

to have it. And when we've talked to

coordinators and monitors, they insist on

seeing it. So there is lots of data, what we

call data, and by "data" it may be marginal

notes that have to be captured, but that

aren't passed on downstream that force people

to use paper source docs which keeps us sort

(202) 234-4433 WASHINGTON, D.C. 20005-3701
of you know tied to paper.

So what can be done about that?

Nothing. No, here we go. We think one

solution is to recreate paper source

documents keeping the 5,000-year-old

interface, the 5,000-year-old data entry

methods that we know and love, but change the

paper. It's not the interface, it's not the

way we enter data that's the problem, it's

paper. So what we propose doing is replacing

paper with a tablet PC which of course you

are very familiar with. Recreating the

source documents that look and feel just like

paper source documents that contain

structured data, check boxes, pulldown lists,

et cetera, but that also allow you to write

marginal notes, to ink comments about blood

pressures, et cetera. The data itself is

then - the data points extracted from the

forms and sent through a secure web portal

straight into the ECRF that the sponsor is


In addition to the data being

passed through, images of the form itself in

PDF version are also captured and sent to the

secure web portal. These are the actual

forms. These are the actual forms with the

checks and the marginal notes, the forms that

have been assigned, and they reside there on

the website where they can be reviewed from

anywhere in the world by the sponsor's

monitor, by regulators from the FDA, anyone

who has authorized access can look at the

source documents, the actual ones. If they

have any questions about data downstream they

can go back up to the source, the wellspring

of the data and look at what was done. In

addition, if it's an adapted trial, for

example, and the source documentation needs

to be changed, it can be changed in one place

and pushed down to the local devices deployed

in the field.

When the study is over, research

sites have the problem of keeping up with

(202) 234-4433 WASHINGTON, D.C. 20005-3701
their paper source docs. There are, as you

well know, warehouses scattered around filled

with paper source docs that supposedly are

kept in a, you know, secure way. One good

storm does those in. So since the source

documentation are digital they can be moved

to a long-term storage where again the forms

and the audit trail can be reviewed at any

later date. The benefits of this sort of

digital source documents are data is entered

only once. Now, most of the time, even when

EDC is in place it's actually entered twice.

It's written on paper first and then it's

translated into the ECRF.

Point of entry protocol-specific

data validation. Before the patient leaves,

data is validated. If there's a problem, you

can correct it there on the spot. Simple,

intuitive, familiar interface. Unobtrusive

data capture. As you all know, tablets are

very unobtrusive. You can work on the fly.

You can set them down. You can take a vital.

You can pick it up. Nothing gets in the way

of the coordinator-patient interaction. A

secure audit trail is of course provided and

that audit trail can be viewed from anywhere.

And you can see exactly what was changed,

when it was changed, who changed it and why.

We believe a system like this

would produce significant barriers to

fraudulent data entry. All the research

sites nationwide globally would all be

looking at the exact same source docs and you

can trace not just the data that's passed

downstream but what they actually did and who

did it. Who signed what. You can see where

they signed it. It's mobile. It works in a

disconnected state. That means if you're not

connected to the internet, you're walking

around your office, you take your data. When

you get back to your internet connection you

sync up and the data is automatically loaded

up to the website.

If data standards are adopted

(202) 234-4433 WASHINGTON, D.C. 20005-3701
freely, it plugs into any ED system. This

system is only concerned about capturing

initial data and then passing it on. This

would allow monitors, data managers,

regulators to verify source documents from

wherever they are. The savings to the

sponsor would be huge. It would solve the

sort of dilemma of things like fixed unit

pricing versus 100 percent source data

verification. How can we verify enough data

and yet save money and meet the budget? So

we think that a system such as this that

focuses on just the initial capture of data

and that does it in a way that mirrors this

old, old paper interface that we're used to

using, but does it in a way that's modern and

that allows us to verify data and securely

transmit data would go a long way toward

finally burying paper.

Now, the FDA's role. Clearly, a

system such as this and any EDC system

requires clear, consistent guidelines for

data validation. We all have to be

validating data in the same way. Things like

the eSDI initiative are crucial. We have to

have compliance market-wide. Right now as

you all know of course there's islands of EDC

systems. Some can communicate with each

other. Some can communicate with EHRs and

EMRs, others can't. There's got to be from

the top down a consistent set of data

transmission standards and we all have to use


We believe that a system such as

this supported by well established shared

data transmission standards would go a long

way toward helping investigators at the site

level. It would make it easier for a new

investigator to enter this line of work. It

would be clearer. What was called upon for

him or her would be clearer. It would also

make post-market surveillance easier, safer

and more affordable. So for all those

reasons we would like to see us move in that

(202) 234-4433 WASHINGTON, D.C. 20005-3701
direction and really address the power of

paper and then bury it. I have time left.


DR. WOODCOCK: Thank you. Are

there questions from the panel? Armando.

DR. OLIVA: Can you be more

specific? Exactly what exchange standards

are missing or are needed in this area?

DR. LITTLEJOHN: Well, I think the

standards that the CDISC group is working on

now where data is gathered in the same format

and then can be transmitted into whatever

back end system is being used, XML standards.

You know, it doesn't really matter on what

the standard is as long as we're using the

same one and it's an appropriate data format

to transmit via the net and share. So I

think the work's being done now, I just think

we need to keep pushing it. We need to make

sure - and at some point there's got to be

teeth. I mean someone has to say this is

what we are doing. This is the standard that

we are going to use. Otherwise I think we're

going to go a long time in having partially

compatible data standards in data

transmission standards.


DR. BUETOW: Yes. The

longstanding debate of whether one should

have structured data at point-of-collection

or later in translation after collecting

unstructured data. Do you want to come down

on that, on what side of the argument you

come down on on that?

DR. LITTLEJOHN: Well, both. I

hate to straddle the fence here. I think the

beauty of digital source docs is that you can

in fact have both. You should have both.

There's a lot of data that the protocol calls

for that is structured data. Data has to be

captured and captured in a certain way. At

the same time there's got to be room to enter

free-form data. I was talking to a monitor

last week and a very simple but clear

(202) 234-4433 WASHINGTON, D.C. 20005-3701
example. She was talking about blood

pressure. And we take a patient's blood

pressure, it's really, really high. Well,

what you always do is you just wait. Maybe

they just ran in, maybe they're late, maybe

they're anxious, six cups of coffee. You

just wait and you take it again. Well, if

there's not room to write a note, literally a

note saying, you know, I gave her five more

minutes, she seemed anxious or nervous. I

took it again and this is what I got. Then

you're left, what you do is you grab paper.

I mean, you aren't going to not record that.

Open up a chart, there's sticky notes

everywhere. And as long as we have paper and

sticky notes we're still chained to paper

source docs and this trail. So I think it

has to have both. I think the only solution

is we have a system that can accommodate



DR. LUTTER: Can you elaborate a

little bit more on what you mean by

validation guidelines?

DR. LITTLEJOHN: Absolutely.

MR. PIERCE: Can I address that?

What needs to happen for something like this

to be adopted by industry. For this to be

adopted by industry they need to know that

the device, the technology, the software that

we're developing is CFR 21 Part 11 compliant

and that the FDA feels confident that these

devices if constructed in the proper way

would be acceptable, would pass an audit, for

example. Some sort of collaboration,

cooperation from the agency to look at these

type devices as they're being developed to

reassure the potential customers which would

be any particular sponsor would want to adopt

the technology. So we would make a plea that

this sort of sharing of information and you

know working together would help move this

along greatly.

DR. WOODCOCK: Is there an issue

(202) 234-4433 WASHINGTON, D.C. 20005-3701
with the current definition of "source

documentation" that exists within the FDA?

MR. PIERCE: I don't think there's

- a source documentation is pretty clear.

It's the initial capture of any information,

whether it's an EKG, a blood pressure, you

know and wherever it exists is the original

source. So I don't think there's a problem.

Something that happens with electronic

medical records, for example, because it

doesn't lend itself easily to a clinical

trial protocol, it's not uncommon for data to

be written down on a piece of paper and then

entered into the electronic health record.

Well, what was on the paper is the actual

source document according to your definition.

So I think we eliminate that problem with

this interface.

DR. WOODCOCK: So what you're

saying is that the Part 11 issue is there

needs to be clarification there, but

conceptually there's not a problem with the

definition of source documentation, it's more

the compliance, Part 11 compliance.

MR. PIERCE: Right. We would just

like to see the guidelines to be a little

more clear, concise, unambiguous and that

there not be significant cost constraints to

adopting them.

DR. WOODCOCK: Other questions

from the panel? Ken.

DR. BUETOW: While we're on Part



DR. BUETOW: So, would you

actually argue partially in answer to the

question right before Dr. Woodcock's, you're

arguing that we need additional specification

a la Part 11 for the specific systems, for

their definitions and validation over and

above what's already in Part 11?

MR. PIERCE: Well, I would submit

that the guidelines in their present form are

rather vague and rather ambiguous and don't

(202) 234-4433 WASHINGTON, D.C. 20005-3701
give I think industry comfort that spending a

lot of money developing a system like this

without knowing ahead of time that it would

be acceptable is a problem. So there aren't

- it's not that the guidelines are wrong, we

just, we need more clear and concise

guidelines I think. More in depth.

DR. WOODCOCK: Any other questions

from the panel? Thank you very much.

MR. PIERCE: Thank you.

DR. WOODCOCK: Our next speaker

will be Mr. Ron Celeste from ThinSpring.

MR. CELESTE: Good morning and

thank you very much. I'd like to thank the

FDA very much for the opportunity to present

this morning. I am going to, having only

really 15 minutes and five minutes for

questions, I'm going to go through these

slides some of them very rapidly. What I'd

like to do is start with a historical

perspective of these submissions and get into

what the current industry standards landscape

looks like, then talk primarily at ThinSpring

servicing over 60 medical products

manufacturers. Primarily going to address

the issues that our clients see in adopting

electronic submission technologies across the

board. Talk about submission lifecycle

management, the concept of a third party

facilitation for these submissions and then

again highlight some specific challenges we

see in industry.

You all remember computer-aided

new drug application guidelines in the `80s.

When I was at Baxter Healthcare the

technology we had at that point in time

really was not very conducive, was not

conducive at all to real-time discussions

between the reviewer and the applicant or

sponsor manufacturer. They did enable

asynchronous communications where files could

be sent back and forth and questions be

answered and then files updated and sent back

using graphics programs like Harvard

(202) 234-4433 WASHINGTON, D.C. 20005-3701
Graphics, the old blue screen WordPerfect

word processor and databases on the desktop,

286 processing power desktops. We didn't

think we'd need anything more. DB3+. Those

tools were not integrated.

We didn't get integrated desktops

until the `90s with things like Microsoft

Office. The portable document format, file

format from Adobe didn't come out until 1994.

Then Al Gore invented the internet and

everything changed. We really view

extensible markup language. My second key

point is that extensible markup language as a

file format is the key technology that can

enable efficient and effective electronic

submissions in any e-submission data


We've seen an explosion - this is

my alphabet soup slide - an explosion of

initiatives since the internet came to be

with the browsers and the desktop

capabilities that are available through a

browser. I don't have time to go through all

of these acronyms. It's full of TLAs and

FLAs, 3-letter acronyms and 4-letter

acronyms. It would take me all 15 minutes to

get through all these, but key point at the

end of this. Moving into the future you see

RPS. RPS stands for regulatory product

submissions and RPS is under the auspices of

- in a standard controlled by HL7. We

believe that in all XML ECTD will be an

efficient file format to deliver effective

electronic submissions throughout industry.

Key challenge for industry. These

purple 3-letter acronyms and 4-letter

acronyms on the screen are existing

enterprise systems that commercial

manufacturers have in-house today.

Enterprise resource planning systems,

electronic document management systems,

enterprise content management systems. The

key is that their electronic submissions

tools and the tools that they use to

(202) 234-4433 WASHINGTON, D.C. 20005-3701
communicate with the FDA need to integrate

tightly with existing legacy information

systems and solutions that are in-house

already at these manufacturers.

There is a roadmap that we've been

following. We've talked about guidelines

that have been published and mandates that

have come out of the FDA starting with the

electronic labeling rule in 2004. This is a

timeline that we're following at ThinSpring

in supporting our clients moving forward. We

do see ECTDs eventually being mandated, a

potential proposed rule coming out of this

meeting, becoming a final rule sometime

possibly in 2007, late 2007. We also see

RPS, the regulatory product submission,

regulated product submission standard being

adopted industry-wide. One of the challenges

we'll get to later though is harmonization of

standards between the U.S., between the E.U.,

between Canada. XML is a data file format

that we believe is going to enable effective

e-submissions. However, there does need to

be an agreement on standards and there does

need to be more harmonization than we've seen

today between ICH, HL7, CDISC and a variety

of other standards bodies out there.

Key challenge for the

manufacturers, the sponsor applicants is that

not all of them, especially the smaller and

mid-sized manufacturing organizations have

today the ability to manage XML data-centric

and document-centric information. So an XML

infrastructure in-house that integrates with

their existing enterprise solutions is a

challenge that a number of the manufacturers

are facing today. XML is a great technology

for improving data portability and data

archiving. Dr. Edwards mentioned we need a

technology that has a long enough lifecycle

to at least match the lifecycle of a medical

product. XML technology and its ability for

archiving, I can open 20 years from now an

XML file with a simple text editor. We have

(202) 234-4433 WASHINGTON, D.C. 20005-3701
one client who is literally storing a deck

vax with a clinical data management system

from 30 years ago in Iron Mountain so that

they can get access to that data when they

need it. With an XML file, you can open that

file 20 years from now with a simple text

editor. So from an archiving standpoint,

again, we firmly believe XML is the -

extensible markup language is the technology,

the file format technology that enables an

all-electronic e-submission platform.

Hear a lot about what extensible

markup language is. I wanted to talk a

little bit about what XML is not. XML is not

a single software package that you can buy

and install and start using like Microsoft

Word. It's a structured content markup

language that facilitates data exchange.

It's not a single electronic submission

standard. There are multiple standards.

There are XML standards for clinical data

that are managed through CDISC. HL7 has the

RCRIM structure for defining medical product

data. ICH has a number of standards. The

electronic common technical document today,

the backbone for the ECTD is XML, defined in

extensible markup language. A key point for

the manufacturers and the people who are

hands-on as Dr. Edwards mentioned, looking at

that screen for eight hours a day is XML is

not a single file. XML is not a Microsoft

Word document, it is not a .pdf file, and

today again most manufacturing organizations

have document management systems that are

geared towards managing the lifecycle of a

single flat file. XML needs to - you need to

have configuration management version status

accounting tools that enable you to manage

the structure, the content and the style or

formatting of your submissions. Another key

point is XML content does not replace printed

literature and that is an interesting

scenario when you start talking about

advertising and marketing materials and how

(202) 234-4433 WASHINGTON, D.C. 20005-3701
content that is approved in extensible markup

language file formats translates into printed

material through different formatting and

style sheets.

This is a little difficult to see,

but this is making the point again that XML

is not a single file format. This is an

example of a structured product label file

folder which contains content files,

structural files and presentation or style

sheet files. So again the industry challenge

is for the manufacturers is that we need to

have tools that allow us the configuration

management control for structure versus

content versus style. We can have that same

structure - I'm sorry, the same content that

gets approved again using the SPL example,

the boxed warning structured product content

gets approved by the FDA and posted on the

National Library of Medicine. We can have

multiple formats or style sheets for

presenting that approved content back at the

sponsor manufacturer's location. They may

have a PDF style sheet to publish that

information on their website. They send it

to perhaps an encapsulated postscript file to

print that boxed warning on a physical

container label when they do a product run on

the shop floor. So the key is we need to be

able to maintain configuration management

status accounting version control of the

content, the structure and the presentation

of our XML-based submissions.

Full lifecycle management. Dr.

Edwards also mentioned that in their internal

system at TAP they're looking towards

improving the management of the meta data.

Meta data is data about data. The meta data

and the content contained in their

submissions throughout the entire product

lifecycle. Again using the structured

product labeling example, most manufacturers

are through the path of - the challenge of

getting existing Word documents, existing PDF

(202) 234-4433 WASHINGTON, D.C. 20005-3701
converted into XML and now are looking

towards full lifecycle control. What do we

do after the submission is approved?

The third party electronic

submission facilitation, an important slide

here. This is just another version of a

slide I presented back in 2004 at the

Structured Product Labeling Challenge that

the FDA published. The technology exists

today now much different from the early `80s

and the mid-`80s to allow both real-time

and/or asynchronous communication between the

sponsor applicant and the FDA. Web-based

technologies and XML data file formats are

those enabling technologies.

Industry adoption challenges. A

key is minimizing system design complexity.

We've heard earlier and we've had a bit of

conversation about validation and having to

re-validate every time there's a change in

the tool set. Minimizing system design

complexity minimizes the validation

challenge. And I have a final slide to

discuss that in more detail. Component-based

content creation training and education.

Again, remember XML is segregating structure

from content from format. That is a

different paradigm for the users at the

sponsors, the authors of these submissions at

the manufacturing sponsor organizations to

get used to. So there is a good deal of

education and training required.

Intellectual property protection

is a big concern of the manufacturers,

especially when we're dealing with a new drug

application and proprietary drug technology.

Technology such as SAFE, Signatures and

Authentication for Everyone. It's a

consortium that was formed to address

electronic signatures and authentication

within industry can be applied to help

address this concern. Regulatory agency

standards harmonization I mentioned earlier.

Very important that we see a higher level of

(202) 234-4433 WASHINGTON, D.C. 20005-3701
harmonization between ICH, HL7 and CDISC.

And then communication, communication,

communication. That's what we're doing here

today. I think this is an outstanding forum

and I think we need to be doing more of this.

My final point on system

complexity. Here's an equation we've been

using for a number of years but you do need

to see that complexity and the validation

challenge of any system design increases

significantly and exponentially as the total

number of software packages and technologies

used to create the solution are employed.

And that's all I had. We have about two

minutes for questions.

DR. WOODCOCK: Thank you very

much. Are there questions from the panel?

Anyone? All right.

MR. CELESTE: Thank you for your


DR. WOODCOCK: We will now take a

break. We will take a break until 10:35 by

that clock right there and also by my watch.

So you have a few minutes, but we will resume

promptly at 10:35. Thank you.

(Whereupon, the foregoing matter

went off the record at 10:17 a.m. and went

back on the record at 10:35 a.m.)

DR. WOODCOCK: Joining us this

morning on the panel sitting in for Dr.

Skirboll will be Dr. Barbara Mittleman also

from NIH. Our next speakers are Dr. David

Hardison who's Vice President of Life

Sciences at SAIC and Wayne Kubick who is

Senior Vice President, Lincoln Technologies.

Ready to go.

DR. HARDISON: Good morning. I'd

like to thank Dr. Woodcock and the panel for

this opportunity to speak today. Wayne

Kubick will be joining me for the Q&A after I

give my remarks.

Clinical Data Interchange

Standards Consortium, better known as CDISC,

wishes to commend FDA for the regulations

(202) 234-4433 WASHINGTON, D.C. 20005-3701
guidance documents and specifications and

their overall support for standards as part

of the efforts to move towards electronic

regulatory submissions. CDISC heartily

endorses the FDA proposal to move to an all-

electronic submission environment for

regulatory information and the creation of an

electronic platform for enhanced information

management across departments and divisions

of the agency.

CDISC is a global, open,

multidisciplinary non-profit organization

that has developed standards to support the

acquisition, exchange, submission and archive

of clinical trial data and meta data. CDISC

is the leader in the development of data

standards for clinical research. CDISC

participants and stakeholders include more

than 175 organizations representing academia,

pharmaceutical companies, technology and

service providers, institutional review

boards and others interested in streamlining

biopharmaceutical product development and

improving clinical data quality in

healthcare. CDISC also has joint memberships

with HL7, PIMS, EMEA and CPATH Institute.

Please check out for additional


The mission of CDISC is to develop

and support global platform independent data

standards that enable information system

interoperability to improve medical research

and related areas of healthcare. The CDISC

standards are in different stages of their

lifecycle. Several of the standards are

written as specifications in FDA guidance and

we continue to work towards finalizing and

improving these standards with your help.

CDISC depends heavily on its volunteers.

Data flow using CDISC standards

including its link with healthcare. After

the presentation of Dr. Littlejohn and Mr.

Pierce today we'll have to modify this to

make sure we better illustrate capture at the

(202) 234-4433 WASHINGTON, D.C. 20005-3701
point of care in our diagram which is where

everything begins from the data standards

world. CDISC supports the belief that

submissions in a standard electronic format

will facilitate regulatory review processes

at FDA and enable the use of common review

tools that improve reviews. I underscore we

support the belief that these standards will

improve regulatory review. Because of the

lack of a robust infrastructure in which our

sponsor companies can support their

electronic submissions, especially with

respect to the CDISC standards. It makes it

very difficult to have a test bid - I knew

the gremlins were going to come in at some

point in time. It makes it very difficult to

test and ensure that an implementation of the

standards can actually be received at FDA.

So therefore having an infrastructure in

place will accelerate and improve the

standards that we are working on so hard

right now. In addition, standards for

electronic submissions enable data

aggregation and the population of cross-study

and cross-product databases that will vastly

enhance the FDA's ability to perform safety

assessments, identify trends and conduct

product evaluations. Such knowledge

repositories would not be feasible without

the submission of logically and semantically

consistent data and information by

manufacturers in a standard electronic


The value of standards accrues to

manufacturers as well. When applied from the

startup stage of a clinical study or program,

standards have been shown to improve data

quality and substantially reduce cost and

time in the product development process for

sponsors and other stakeholders in the

industry. We will speak to these specifics

later. Most importantly, CDISC believes the

transition from paper to standard electronic

submissions will lead to safer and more

(202) 234-4433 WASHINGTON, D.C. 20005-3701
effective drug products to improve public


One of the impediments to

transitioning to an electronic submission

environment is the requirement that this data

or information must be in a standard format.

Fortunately industry has been actively

involved with the agency over the past decade

to help develop and implement industry

standards for the purpose of supporting

electronic submissions. CDISC is the leader

in the development of data standards related

to clinical research. Our members have

contributed collaboratively to the consensus-

based CDISC standards development process.

Most stakeholders in the industry are either

in the process of implementing or making

plans to implement these standards. Broad-

based technical knowledge and experience in

the application of these standards are needed

among all stakeholders, including industry

and FDA personnel. CDISC is proud of the

fact that we're working with FDA to provide

adequate training in these standards in the

coming year. Fortunately, both XML and

expertise and knowledge of the CDISC

standards are increasingly prominent within

the workforce. Education and training

courses are readily available at a reasonable


While the implementation of

standards-based electronic submissions

requires an investment of time and money, it

also reduces operating costs and time-to-

market once the standards are in place.

CDISC recently participated and partnered

with the Gartner Group with support from

PhRMA to develop a business case analysis

that estimates the costs and benefits of

standards implementation. While these costs

are very difficult to estimate particularly

because of the absence of baseline

information, the business case metrics

indicate that there are substantial

(202) 234-4433 WASHINGTON, D.C. 20005-3701
reductions in cost and time related to

capturing, cleaning, analyzing and reporting

clinical trial data, especially when

standards are used at the startup stage of a

clinical trial or program. Similarly,

regulatory reviewers will be able to spend

less time on data manipulation and more time

on the science when data are submitted in a

standard format. Communication will also

improve between the agency and manufacturers

such as reducing time for follow-up queries

once a submission occurs.

Over 50 percent of clinical trials

involve a contract research organization,

CRO, and an increasing number of trials

involve electronic data capture vendors.

Communications and data exchange among

sponsors, CROs, EDC vendors and even project

team members become much more cost-effective

and easier when standards and electronic

processes are employed. This is especially

true as we've seen in earlier presentations

if we can harmonize standards across clinical

research and with healthcare so that data are

entered only once at the site. This will

help relieve a lot of the burden of clinical

investigators and create more incentives for

their participation in clinical research.

Therefore, CDISC heartily supports the 2-year

industry transition time that is stated in

the proposed rule for electronic regulatory


Making the transition from paper

to electronic submissions is hard work.

CDISC would like to offer several

recommendations to ease the transition. FDA

must take the strong position to drive

manufacturers to make all electronic

submissions in XML containing data in

specified standard formats. The majority of

industry organizations is prepared or is in

the process of preparing to utilize CDISC

standards for submission of clinical data for

regulated biopharmaceutical products. This

(202) 234-4433 WASHINGTON, D.C. 20005-3701
includes the CDISC study data tabulation

model, the operational data model and

define.xml. SDTM can be submitted to FDA in

the ECTD or the HL7 regulated product

submission format.

We would like to see the FDA adopt

the ODM for electronic CRF data submissions

in addition to using it as a transport format

for define.xml meta data and SDTM and

analysis data. This transport standard is

already familiar to most industry

stakeholders. It can also facilitate auditor

reviews of electronic data capture

environments at investigative sites.

CDISC is driven by a mission that

supports electronic submissions and data

exchange that will enable translational

research. To that end CDISC is working with

industry and FDA on standards to support

electronic submissions for devices, genomics

data and animal data. CDISC has already

published the SEND standard, the Standards

Exchange of Nonclinical Data. Both SEND and

STTM are cited in study data specifications

for the final ECTD guidance.

Many of you - translational

research is more rapidly bringing innovations

from bench to bedside. For over five years

CDISC has formal relationship with HL7 to

enable a link between clinical research and

healthcare through standards. We strongly

encourage the FDA to continue its support of

these standards development efforts which are

critical success factors for translational

research and electronic submissions and data


Variation in terminology is also a

barrier to effective collaboration, data

aggregation and multidisciplinary research.

CDISC collaborates with the National Cancer

Institute in a number of ways, including the

development of standards relevant for

oncology. CDISC through NIH roadmap grants

in collaboration with Duke Clinical Research

(202) 234-4433 WASHINGTON, D.C. 20005-3701
Institute has work under way to develop

therapeutic area terminology standards with

the initial focus on cardiovascular and

infectious diseases. FDA should continue its

support of terminology standards.

CDISC is leading a collaborative

group devoted to achieving Critical Path

Opportunity Number 45 to standardize case

report form data collection consistent with

the STTM format. FDA should provide

additional support to complete this work

which will provide tremendous value to

investigators, clinical research associates

and monitors, project leaders and others who

work at the front end of the clinical trial

process. As mentioned earlier, these

benefits will cascade to provide significant

downstream benefit for electronic


It goes without saying that the

infrastructure to receive standardized

electronic submissions is critical. So are

standards. However, the lack of an

infrastructure, its absence, is a barrier to

standards development. We need a test bed.

We need to be able to demonstrate that these

standards, once developed and implemented in

our sponsor companies, in our member

companies, will actually be acceptable to

FDA. Therefore CDISC supports clinical

research information exchange as it's

currently being proposed.

Additional standards. Before

specifying additional standards, CDISC

believes it is important to first apply the

existing standards effectively and to work to

harmonize standards to support semantic

interoperability within clinical research and

between research and healthcare. Significant

progress has been made in this area with the

Biomedical Research Integrated Domain Group,

or the BRIDG modeling efforts. This was

initiated by CDISC. It's now an open source

model governed by FDA, CDISC, NCI and HL7.

(202) 234-4433 WASHINGTON, D.C. 20005-3701
Further harmonization support is

needed in specific areas, including the

following. To harmonize the multiple

standards related to adverse event reporting.

At last count there were six. Clearly a

requirement of biopharmaceutical companies to

report globally, having to do that in

multiple formats is a problem. Doing

government-sponsored research and have to

provide adverse event data in one form, why

does it have to be different than industry?

And I could go on and on. This is a

significant area of concern given the

importance of drug safety and surveillance.

We also need to complete - ensure

compatibility with implementations of the

CDISC HL7 protocol representation standard,

including trial design. Further, we need to

harmonize the protocol representation

standard and SDTM with the needs of other

regulatory agencies and global organizations.

For example, EMEA's EudraCT and the WHO's

International Clinical Trial Registry

Platform and various results reporting

projects, all of these need to be harmonized

so that we can adopt the standards to be more

robust to meet the multiple needs of these

different stakeholders.

Global pharmaceutical companies

will appreciate the efficiencies they will

gain if there is harmonization of

requirements and a single consistent set of

standards used across multiple groups such as

FDA and I might add within FDA. It's

important to - if there were clarity and

consistency within the agency and I know that

is a priority. Also with EMEA, WHO and NIH.

These groups are requesting trial registry

information and results reporting subsets for

multiple purposes. If these data formats are

designed to support semantic interoperability

there will be cost and time savings for all

stakeholders globally. This is very

important. We ask that FDA recognize and

(202) 234-4433 WASHINGTON, D.C. 20005-3701
support CDISC's continued leadership of these

standards enhancements and harmonization


CDISC supports the use of a third

party entity, especially if that accelerates

the decision by FDA to require electronic

submissions in a standard format. We believe

that the benefits to patient care and all

stakeholders far exceed the initial cost.

CDISC asks that FDA recognize and support

CDISC's continued leadership in these areas

for the enhancement and harmonization of

standards for clinical research. Thank you

very much for this opportunity to add our

comments today on behalf of CDISC and its

board and all of its members. We're very

grateful and now we'll take questions, and

Wayne if you'll join me.

DR. WOODCOCK: Thank you very

much. Are there questions from the panel?


DR. LEVIN: First, I'd like to

thank the CDISC organization for their

efforts on moving the data standards forward.

They've done a lot of good work and I

appreciate all the work that you've been

doing on this. One of the things that you

noted on your presentation were that some

companies are being proactive but others are

waiting for a mandate. Can you elaborate on

that, why they're doing that and what are the


DR. HARDISON: I'll make a general

comment and, Wayne, you can add some

specifics. Implementing standards requires

an investment. And I think that industry and

especially this industry who's risk averse in

its nature are waiting for an indication

that, (a) this is an absolute requirement,

and (b) the agency is actually ready to

receive electronic submissions. So that's

the general comment. Wayne?

MR. KUBICK: I think there are a

couple of other points as well that are

(202) 234-4433 WASHINGTON, D.C. 20005-3701
relevant. One is companies are concerned

that maybe the standards environment isn't

terribly stable yet, that things will be

changing, they'll be making investments and

then they'll be forced to change again. And

so we need to have a uniform message that

we're going to hold to these standards,

they're going to be backward compatible and

that work you put in now won't be lost in the


I think the other issue which is

pretty important is that it's a very complex

standards environment. A lot of people are

looking out there and concerned that there's

competition between some of the standards,

the HL7 versus the CDISC standards for

example, and we're working very closely to

keep those harmonized. We're working very

tightly within the RCRIM environment. But

the perception out there is not quite as

clear. And so there are concerns that again

something else will come along right after

I've gone, put the effort into implementing

what's there now. And that's one of the

messages we need to get across is a uniform

stable standard that we're all supporting,

we're all working on the same direction.

DR. LUTTER: Thank you. Could I

just follow up on that mandate question that

the other Randy asked? And that is do you

see that as necessary or would alternatives

such as incentives, think reductions in user

fees or maybe changes in performance goals or

options like that might also be an

appropriate substitute for mandates? That's

the first question.

The second one is in that same

slide on key messages for management from the

PhRMA-Gartner-CDISC project, could you

elaborate more on the nature of the cost

savings? In that slide you had an average

per study cost savings of $9 million. What

type of study is one where one might expect

savings of that nature? Thank you.

(202) 234-4433 WASHINGTON, D.C. 20005-3701
DR. HARDISON: Okay. I'll try to

address the first question around incentives

versus mandate. Basically I think that a

number of companies who see the benefits of

implementing the standards upstream in the

whole lifecycle of clinical research at the

beginning around - even with the protocol and

especially when they're designing their data

capture approach, the case report form

annotation, data management plans,

statistical analysis plans. Companies just

see the benefit of doing that according to

consistent standards globally. And then they

say well which standards? Well, there's the

CDISC standards. It just makes good business

sense. So a number of companies no matter

whether FDA ever requires or mandates

electronic submissions in the standard

format, you just see the business benefit of

moving to the standards. So a lot of

companies do not need a mandate for that.

Then there's whatever proportion,

I can't give you an exact calculation of

this. A number of companies are - like to

sit on the fence. They're aware of the

standards activity, but they're waiting for a

clear signal from FDA that there will be

benefit from them making investments in

implementing these standards. So what

signals do they have that this is going to be

something that is worthwhile? One, clearly

having reference to the standards and

specifications in guidance documents is a

good step.

Mandate versus incentives, I can't

answer that since I'm not a sponsor company,

but when I used to work for a sponsor company

clearly incentives are good because the cost

of drug-to-development is so high. So

therefore I would think that it would be a

driver. But there's still nothing short of

being able to point to a rule that says, you

know, you must do this if you want to get

your new drug application approved. The

(202) 234-4433 WASHINGTON, D.C. 20005-3701
strength behind that I think outweighs

whatever you might be able to do with


Now, the other question, second

question. Do you have anything to add to

that? Okay. Second question, around the

Gartner-PhRMA-CDISC business case. The

savings - problem with doing any kind of

business case industry level and also being

trained as a statistician early in my career,

you have problems with averages. The

average, that number, the $9 million savings,

was based on average IND clinical trials as

reported by - I think we used CMR data as one

of the primary sources for this. Then we

also used Tufts data who estimate that the

average - once again, average cost of drug

development is I think $37,000 a day. So

when you look at the - and the folks that

participate in the business case study, major

biopharmaceutical companies who have adopted

CDISC standards. We were able to extrapolate

the savings that they accrued down to the

average investment required for clinical

trials to come up with that $9 million on

average savings. So typically these would be

the large multi-center clinical trials,

pivotal studies, Phase III is where you would

start to see that amount. Smaller studies

you'd accrue less savings, bigger studies

more savings. Hope that helps. Ken.

DR. BUETOW: So I think several

speakers - and again, you all have

acknowledged this challenge of both ambiguity

and inconsistency of the current collection

of standards. And in a certain sense we find

ourselves maybe caught in this sort of

chicken and egg sort of universe right here

as acknowledged by you all as people don't

want to adopt the standards until they're

stable and harmonized, yet we don't have

harmonized standards. So can you help us see

a path forward that we could very rapidly get

to harmonized standards that seems like a

(202) 234-4433 WASHINGTON, D.C. 20005-3701
prerequisite for this?

DR. HARDISON: I'll let Wayne

answer this after I just make one general

comment. It's that we've got to clarify

first, when we say "harmonizing standards" be

clear on which standards we're talking about.

And the second comment is within those

standards that CDISC are producing, it is a

number one priority to make sure our

standards are harmonized internally. And

from the board's standpoint we've invested

significant dollars to put, you know, put

something behind our words. And Wayne is

leading that effort and can say more about


MR. KUBICK: There's a couple of

different points. I think first of all

different standards represent different

constituencies and so the more broader you

make the scope the tougher the problem

becomes. Within CDISC when our original

scope of supporting standards that support

regulated clinical research among the

commercial pharmaceutical industry, I think

we feel our standards are pretty solid.

There's work to be done to tie them together,

but we think they're usable and that we're

committed to making sure that there's going

to be a clean, upward migration path for any

work done forward.

The second part of the issue is

harmonizing with the broader world and we're

working together with NCI and with HL7 RCRIM

on the BRIDG model, the bioresearch

information development group, domain group,

the BRIDG model. And we're making sure that

we're representing all the standards and the

semantics for all the standards and the

semantics for all the standards in this

common domain analysis model. We've been

very successful to date without having to

really make any structural changes to the

existing standards, but making sure that we

make the interconnections between them. That

(202) 234-4433 WASHINGTON, D.C. 20005-3701
really is a critical centerpiece moving

forward to make sure these activities stay


DR. BUETOW: Maybe a riff on that

same thread then and it goes back to the

early question of mandate versus consensus or

incentivize. Would you argue then there's a

strong role for mandating specific standards

for the deployment and then other things will

harmonize or lock in around those?

DR. HARDISON: Well, selfishly,

mandates are always good for standards

development organizations.

MR. KUBICK: But a mandate doesn't

necessarily have to take the form of

legislation. A mandate that, you know, we

need this in order to achieve our principal

mission of delivering better healthcare to

the world and ensuring patient safety is a

strong mandate. If we keep that consistent

message and make sure that the right things

to do in support of that mandate I think we

can make a lot of progress.

DR. HARDISON: And if I may add,

that's why having an organization of

public/private partnership that provides an

environment for actually demonstrating the

efficacy of these standards if you will is so

critical. Because that then would allow us

to more rapidly pilot and move from the

actual development of the standards to

implementation of the standards in an

accelerated way.

DR. LEVIN: I have another

question if I could, going back to when we

were talking about the timelines on

investment. Can you expand on that a little

bit? How would that work? Can you say a

little more, like if you're going to now go

in to adopt a CDISC standards, move it into

your system just at a company level, what are

the timelines that are associated with that?

Or how is that?

DR. HARDISON: Well, it depends

(202) 234-4433 WASHINGTON, D.C. 20005-3701
very much on the size and complexity of your

organization, your legacy environment versus

you know a small startup biotech where you

have a green field. So it's going to vary

tremendously depending on your starting

point. Now, Wayne, you've been involved with

several implementations. Do you want to

comment specifically on that?

MR. KUBICK: Well, I think again

partially as a consequence for some of the

other issues we said earlier, a lot of

companies have been adopting standards pretty

much at the end stage process. You know,

taking what you get out and mapping it and

that's seldom an effective way to do things.

The critical issue is moving things upstream.

And it really isn't necessarily a huge issue

if you start out with the standards embedded.

I think the problems we have is sort of

unlearning the years of legacy processes and

individual standards that have been done.

Some companies make the transition very

quickly in a matter of months. Others,

larger companies certainly take longer times.

You know, that's the transition

implementation path that has to be followed.

DR. LEVIN: If you're starting off

with a new study, I mean that would be the

easiest transition, I guess.

MR. KUBICK: Absolutely.

DR. HARDISON: Yes. I have

clients that I've worked with who, once they

made the commitment to the standards, they

required a mandate in their organization that

said all new studies starting January 1, 2007

will use the standards. And then you have to

make a decision well what about ongoing

studies. Given where they are in their

lifecycle, is it cost-effective to convert

them or not? And you know, that's just a

business decision that you have to make on a

study-by-study company-by-company basis.

DR. WOODCOCK: Thank you very

much. Our next speaker will be Mr. Tom Klaff

(202) 234-4433 WASHINGTON, D.C. 20005-3701
from Surety.

MR. KLAFF: I want to thank you

all, especially the panel for letting us

speak today on the electronic submission

standard and what Surety's perspective is on

this. My name is Tom Klaff. I'm the CEO of

Surety and we are a trusted third party time-

stamping company. Some call us a digital

notary. And I stand here to represent not

just my company, but also an industry, a

burgeoning industry called the trusted time-

stamping market. And only about three years

ago it really didn't exist, but as many folks

who are familiar with record management and

migrating from paper records to electronic

records and also through what's happening

with the Federal Rules of Evidence and the

Federal Rules of Civil Procedure, the need to

prove integrity of data is really very

important and especially when organizations

rely on this electronic information that

constitutes roughly 70 to 80 percent of their

asset value, it's very, very important.

What I want to talk about today in

a very short time is just give you a brief

introduction about who we are and then go

into some of the obstacles faced by going all

electronic. We have a lot of experience with

this. We have several customers who have

made this decision or are making this

decision and it required the need to prove

independently of their people, processes and

systems that all of their electronic records

and forms and meta data and audit logs had

never been altered either maliciously or

inadvertently. I want to talk about also

from the perspective of a third party

provider. I know part of the submission

standard is who would be able to provide this

kind of service from an archive standpoint.

Surety has a lot of experience working with

third party archive service providers because

it attacks a wholly different problem. When

you're a custodial data provider, you're

(202) 234-4433 WASHINGTON, D.C. 20005-3701
managing other people's electronic records

and there's a need to mitigate that risk and

to prove legally that during its chain of

custody those records were never maliciously

altered. I want to talk about the industry

standards surrounding trusted third party

time-stamping and as I indicated before, I am

also a representative of what's called the

Information Assurance Consortium which is a

body of time-stamp vendors who have come

together to form a standard working with the

American National Standards Institute, ANSI,

in what's called the ANSI X9.95 which is

endorsed by many of the financial service

companies that you know today including also

NIST and NSA. And then lastly I want to just

give a brief implementation scenario about

how this would work inside of a live


So Surety is an IT security

software and services company. We're based

in Herndon, Virginia and we provide software

and we operate a third party trusted time-

stamping service called AbsoluteProof.

AbsoluteProof enables our customers to

protect and defend their intellectual

property by independently proving the

integrity of their electronic records from

the point that they were created ongoing for

permanent preservation. The issue that we're

addressing as part of third party trusted

time-stamping is legal defensibility. And

underneath the legal defensibility angle

there's intellectual property protection and

litigation readiness. Many companies we work

with in the life sciences market have made a

conscious decision to move from a paper-based

record management system to an all-electronic

records management system. Some have

deployed electronic lab notebooks. Some are

going with electronic records management

systems and building their own custom

applications but in each instance they've all

recognized the need to integrate the ability

(202) 234-4433 WASHINGTON, D.C. 20005-3701
to prove that their documents, their

electronic IP was pristine. And especially

when it comes into litigation, and especially

considering the recently amended Federal

Rules of Civil Procedure. During meet-and-

confer sessions those files could be turned

over in electronic record discovery and there

is a need also from the electronic file

standpoint to prove that those haven't been

altered. So authenticity is very, very


A number of reasons why data

integrity is a specific issue. First and

foremost about 93 percent of all business

records today are electronic. And everybody

considers the outside attacks being a clear

threat to infrastructure, and clearly it is,

but what people don't realize for the most

part is that for the most part, 60 percent of

IT security breaches happened inside the

enterprise with trusted insiders. And those

are the databases and the record management

systems and the electronic assets that are

really very, very important. Also with the

advent of more publishing software it's very,

very easy to tamper with electronic records

if you are an insider, but very, very

difficult to detect the manipulation of those

records. And lastly as I indicated, data

integrity is really a key component as part

of anybody's electronic vital legal defense

if they're going electronic, and that's going

to be a very, very important issue to

consider going forward.

So what does time-stamping

provide? Well, it's really an added layer of

assurance to what companies are doing in the

electronic world. Clearly we're not talking

badly of electronic signatures or digital

signatures. We're looking at this as really

an addition or an additional layer of

assurance because at the end of the day

digital signatures could expire, they could

be compromised. There needs to be a

(202) 234-4433 WASHINGTON, D.C. 20005-3701
cryptographic binding of an electronic record

with a secured audible time source and that's

what we're advocating. So really from the

standpoint of trusted time-stamping it's the

what and the when in addition to the who that

makes a very compelling argument to defend

legally your electronic records.

Lastly, there's a need to prove

irrefutably that those documents that you

possess, those forms that you are archiving

and managing are trustworthy. Sandra Lambert

who was the editor of the X9.95 standard,

quote, says, "Trusted time stamps will

replace mechanical, manual or system-

generated time stamps and provide a tool that

will offer an evidentiary trail of data

authenticity." Anybody that has a laptop

knows how easy it is to turn back the date

and the time on your laptop. And we have

several examples at Surety where we can

demonstrate that even with a PDF file with a

backdated time clock and a digitally signed

document for that matter can actually show

that a record that existed - even an article

from the Wall Street Journal that was

published on January 17 could be signed by

somebody with a digital signature on January

5 and then validated. So from a legal

standpoint that is a clear piece of evidence

that could be challenged and could be thrown


So why do customers choose our

service? One, and first and foremost really

from the life sciences standpoint and from

the semiconductor industry and the

manufacturing industry and the aerospace

industry and even the government is to be

able to prove that your intellectual property

is yours. It's an effective deterrent with a

time-stamping service integrated into your

business process to deter others from

actually going in and manipulating electronic

IP. Secondly, it's the ability to really

monetize the value of going fully electronic.

(202) 234-4433 WASHINGTON, D.C. 20005-3701
A lot of organizations go halfway. We call

them hybrid. They actually want to go

electronic but also use paper printouts as a

way, as their redundant record management

backstop. And really that is a very costly

endeavor indeed. So the ability to prove

non-repudiation is important and with having

the ability to attest the integrity of that

document similar to a wet signature is

really, really important.

We talked about legal

defensibility and authenticity and then the

ANSI X9.95 standard is what I want to get

through next. AbsoluteProof application

areas, other time-stamping providers can

actually seal any sort of data. It could be

Word documents, PowerPoints, it could be

audit logs, meta data, it could be audio,

video, it really doesn't matter. At the end

of the day it's asset data that is important

to the end user. And if it's important to

the end user, it needs to be protected.

From a third party service

standpoint if I am an archive service I need

the ability of three key things. One, I need

to have the ability to have long-lasting

protection because if I'm managing a lot of

infrastructure, there's no question that my

infrastructure will obsolesce over time.

It's the ability to constantly upgrade that

environment and know that the records that

are being managed and stored will not lose

their ability to be proven authentic.

Secondly, it's the ability to be

independently verifiable. And when I talk

about independent verifiable, I mean

independent of your own customer's people,

processes and systems and of yours. So these

all could be challenged legally.

And lastly, it's the need to go

with standard solutions, and that's what I

want to talk about now. The ANSI X9.95

standard is a standard form for trusted time

stamp management specifically endorsed by the

(202) 234-4433 WASHINGTON, D.C. 20005-3701
financial services community. It was

unanimously approved back in 2005 and it

allows for choice. There are a number of

technologies that at one point were

proprietary and now they are part of a

standard so that there is vendor neutrality

and the ability for customers to know that

their investment is protected in time stamps.

Under the standard there are four specific

technologies that are proved and endorsed.

One is a digital signature type of time-

stamping, two is a message authentication

code, three is link token, which is what we

do, and the fourth is transient key. And

within the standard there's a comprehensive

set of processes, procedures that would allow

organizations who wish to deploy these time

stamps to be complicit.

We are often compared to an RFC

3161 standard which really technically isn't

a standard. It's just a request for comment.

But the X9.95 trusted time stamp standard

does include digital signatures which RFC

3161 supports, plus the other three

techniques. What's also key is that there is

policy and practice statements involved in

the standard. There's audit evaluation

criteria and the process is well spelled out,

including time calibration and auditability

of those time stamps that come from secure


With the last remaining seconds I

have left I just want to go through an actual

implementation scenario where an organization

has migrated from a paper-based record

management system to an electronic record

management system and has used a third party

time-stamping service provider like Surety to

seal and validate the integrity of their

electronic IP. And this happened to be

integrated into an electronic lab notebook

infrastructure also using electronic

signature. So from really the concept of the

idea until the preservation of the idea there

(202) 234-4433 WASHINGTON, D.C. 20005-3701
is a seal that is assigned to that specific

file. That file can be stored in any record

management or archive module. And then at

some point in the future if that file or if a

patent or a trade secret is being litigated,

that piece of evidence can be sent along with

the trusted seal to a third party for

validation. So the key point here is that

it's very, very important when considering

electronic submission systems and management

systems that you consider the legal

defensibility angle and know that down the

road 10 - 20 years from now when challenges

do occur and the people who were originally

there aren't there anymore that those records

have the ability to stand up in court.

In conclusion, there are a number

of ways that you can go about doing this. We

believe that as organizations go through the

electronic record management process and

decide for themselves that paper isn't

necessarily the way they want to go, that

there needs to be a mechanism in place for

being able to prove the integrity of their

electronic records. And Surety's

AbsoluteProof time-stamping service is one of

those mechanisms. And we've been able to

work with organizations who wish to protect

their IP, but also organizations who from a

third party standpoint wish to mitigate their

risk and be able to prove the chain of

custody has never been altered. Thank you.

DR. WOODCOCK: Thank you very

much. Are there questions from the panel?

Yes, Kevin.

DR. FAIN: Yes, I have one

question. I was interested in you mentioned

that the time stamps have been used in the

context of private litigation for

authenticity of documents. I was wondering

in terms of FDA and FDA inspections, are you

aware of those time stamps being considered

or used by companies, especially when FDA is

inspecting records? And I guess a follow-up

(202) 234-4433 WASHINGTON, D.C. 20005-3701
question to that would be are there any

unique issues for FDA to take into account in

considering these time stamps?

MR. KLAFF: Well, I think that I

mean from our company standpoint we haven't

engaged with the FDA in any sort of

commercial endeavor. But I would add that

the time stamp technology that we use really

is - if you're familiar with secure hash

algorithms, they're NIST-approved so there

isn't really anything proprietary about them.

From a litigation standpoint typically

technologies aren't challenged as much as the

processes are challenged. So from that

standpoint all we're doing is we're helping

organizations who have specific processes in

place know that in a litigation standpoint

you can surely challenge their processes, but

when it comes to the actual data there is

irrefutable proof that nothing has been

changed. And we're advocating that if you're

looking at standards from an electronic

submissions standpoint, you might want to

consider the ANSI X9.95 standard.

DR. WOODCOCK: Any other questions

from the panel? Thank you very much.

MR. KLAFF: Thank you.

DR. WOODCOCK: Our next speaker is

Nancy Smerkanich and she is Vice President,

Regulatory Affairs, Octagon Research


MS. SMERKANICH: Thank you Dr.

Woodcock, members of the panel for first of

all for pronouncing my name right. I don't

get that very often. I appreciate it. Thank

you for allowing me to speak today. A little

background about myself and why I wanted to

present today. I've been a regulatory

affairs professional for 20-some odd years.

I started off my career in big pharma. I

worked as an independent consultant and for

the last six years I've been at Octagon. My

primary responsibility in all that time was

compiling and maintaining both

(202) 234-4433 WASHINGTON, D.C. 20005-3701
investigational and new drug applications. I

am certainly not only a believer in

electronic submissions, but probably some

people would consider me a bit of an

evangelist about them. I spent the first 10

years of my career saying there's got to be a

better way to do this and then I found the

better way and I want everybody else to do it

that way as well.

For purposes of full disclosure I

should tell you that I am working at Octagon.

I'm not only an employee but a stockholder.

And we are a software and services provider

to the pharmaceutical and biotech industry,

primarily focused on electronic submissions

but also the data management and data

integration that's involved in creating those

deliverables for electronic submissions. So

the benefits may seem obvious to you if

electronic submissions become mandated, but

to me it means I get a whole lot busier. And

I'm already very busy.

But the most compelling reason

that I asked to present today has to do with

the fact that we sent an email blast out of

all of my regulatory contacts which numbers

between 200 and 300 people about this meeting

because a lot of people have trouble keeping

up to date on the docket and so we sent out

the email blast and the response to some very

simple questions that were in that email was

overwhelming. In addition to the responses

to what I'm calling an informal survey, I

received numerous voice mails, emails, people

even tracked me down on my cell phone to ask

me to present information on their behalf

today because especially for small to mid-

sized pharma there's a lot of regulatory

people who wear multiple hats who knew they

wouldn't be able to attend today. So while

there were many great questions I think in

the docket I'm going to confine my remarks to

transitioning to and implementing electronic

submissions from the point of view from these

(202) 234-4433 WASHINGTON, D.C. 20005-3701
responders. Hopefully I'll be able to answer

some questions from my own experience. And

then there was also a number of questions in

the docket about cost and time of doing

electronic submissions so I have a couple of

scenarios that I can present to you.

Hopefully you'll find them useful. They

involve not only the cost of the production

to do an electronic submission but also the

time involved.

So we took the questions in the

docket and kind of boiled them down to a

little bit leaner, meaner set of questions

basically asking folks have they implemented

electronic submissions within their

organization and if the answer to that was

no, why no. And then asking those who had

what were the major challenges of

implementing electronic submissions and what

were the major benefits. And then

specifically trying to get at some of the

concerns around cost we asked if those costs

associated with doing this was burdensome.

And then we left the field open for

additional comments. And I have to tell you,

you'll see in the next few slides where there

were some people who didn't answer any

questions but wildly used the additional

comment field. So I've tried to summarize

those as well. So I received 77 responses in

seven days plus all the emails and the voice

mails. So this is obviously a hot button

issue for a lot of folks out there. The

companies represented were either tier one

big pharma, tier two you know kind of mid-

sized pharma or small pharma as well as some

additional participants who represented CROs,

medical devices and vet meds.

And then the answers were kind of

split amongst have you implemented electronic

submissions within your organization. These

are overall percentages. The tier one

companies, 10 yeses, they equated to 13

percent, six nos. The tier twos were split.

(202) 234-4433 WASHINGTON, D.C. 20005-3701
The tier threes had 26 companies that had and

28 that had not, and the others were split

equally amongst them as well.

So when we asked the nos why no,

why haven't you done electronic submissions,

some kind of interesting information perked

out of this. For tier one companies they

cited lack of internal resources and

expertise, lack of upper management support

which in big pharma is an endless chain at

times. They cited costs and they cited

ingrained processes. And I think a number of

people today have addressed the fact that big

pharma can be very hard to move to a new way

of doing things. It has a lot to do with

change management. Tier two companies were

both undergoing pending acquisitions. I

don't know if that was of their companies or

of their drugs, but it certainly puts you

into a holding pattern if you don't know what

your future is going to be. As for the

little guys, the tier threes, they cited

time, costs, expertise and resources. And

again, we'll get into a little bit more about

what I think time, cost, expertise and

resources mean, but in fact a number of them

were in the stage of assessing how to go to

electronic submissions and a number of them

were also implementing. They too had

management constraints. The fact that

electronic submissions aren't mandated was

definitely a barrier for some of them. Some

of them it was just an issue of timing. They

didn't actually have a major submission that

would warrant it. All of the companies that

were in the ancillary area were in the

implementation stage so I think that's

encouraging. And as I mentioned, you'll see

34 no responses. It just means they went

directly to the comments field.

So for the folks that had gone to

electronic submission we asked what the major

challenges were. And you'll notice that tier

one and tier three have identical responses,

(202) 234-4433 WASHINGTON, D.C. 20005-3701
but I think it has a lot to do with what time

means to big pharma and what time means to a

smaller company. Time to change existing

process to integrate with numerous other

tools for big pharma and lead time for small

pharma. Just having that built into their

development process. They both cited cost.

Again, cost if one of those things, for big

pharma it's a lot like renovating an old

house, but for new pharma it's like new

construction, and there are a lot of costs

associated with both. The fact that both

cited expertise I think is interesting and it

kind of speaks to where the workforce is and

where the experience level is with electronic


Training, obviously very important

and resources. And I think a lot of time

resources for a small company can mean

everything from an IT infrastructure to an IT

support person as opposed to dedicated

resources in a bigger pharmaceutical company.

Both cited process change, but again I think

they mean different things. I think for tier

one pharma you truly are talking about

changing existing process, but for smaller

pharmaceutical companies you're talking about

no process most of the time.

There were some technology issues

cited by the tier two companies and some

additional things for the tier threes which

was they just didn't even know how to create

compliant documents and didn't have document

standards. They dealt with a lot of legacy

study information and again, probably most of

it outside of their control. And that they

couldn't communicate to the management that

these challenges could actually be time-

consuming and costly.

The major benefits. Really they

validate everything I've been saying for the

last six years so I really am happy to

present this slide. Both - actually across

all of the companies the major benefits of

(202) 234-4433 WASHINGTON, D.C. 20005-3701
electronic submissions were seen to be the

ease of use within the organization, the

speed of compilation. As one of the end

people it is faster, but it isn't a push of a

button. Compatibility with other systems was

very important for big pharma as were

harmonization mostly across sites and across

subsidiary companies, reviewer friendliness

and document reuse. I love the fact that all

three tier companies said no paper because

that certainly was recognized as a benefit.

And it was interesting to me that the tier

one companies perceived that there was a

quicker agency approval with electronic

submissions although we've never been given

any metrics to that effect. I'm looking at

Dr. Levin when I say that. For tier three

companies again they stated that the same

benefits and so I think there's a real

consistent message there in terms of what are

the benefits.

In response to the question about

costs it was a very interesting split here.

Tier two companies you'll remember from the

last slide were the ones that had technology

issues and they clearly felt that there was a

burden to the cost of implementing electronic

solutions. But tier one and tier three

companies were relatively split amongst them

as were the others and the others were

actually more favorable in terms of there not

being additional cost burden on creating

electronic submission deliverables, but I

suspect that's because they were from CROs

and so they basically have electronic

databases and electronic documents so they

weren't implementing a whole enterprise-wide


Fifty-one of the respondents

provided additional comments, some of which

were very lengthy. It was a good venting

session for some people I have to say. And

so to try to summarize them was actually a

little bit difficult. But since everyone hit

(202) 234-4433 WASHINGTON, D.C. 20005-3701
on cost I did make a point around summarizing

that, that most if not all of the companies

in some way or another cited costs as an

issue but felt that the cost of implementing

electronic submissions would be worthwhile in

the long run. And I have to say I take this

quote and I'm probably going to reuse it

again because it's a good one from a small

company. "We build and collect things

electronically. It makes sense to submit

them electronically. It's the cost of doing

business." Some companies felt that the

impact of the costs actually hurt them in a

competitive environment. And I think that

this comment also related to some issues that

especially small pharmaceutical companies

face when they have limited resources in

terms of head count and they have to choose

someone to place on the clinical team or

someone to place in regulatory operations.

Guess who usually wins? During the

development phase clinical usually wins.

Some companies felt that finding the correct

technology that's reliable and current is not

only an issue but will continue to be an

issue. I remember one of the comments

actually said getting the right people and

the right tools at the right time was

probably their greatest challenge. And there

were at least seven if not more companies

that responded that they really wanted to see

ECTD mandated because it's the only way they

felt they were going to get buy-in from their

upper management and to move their

contributors away from a paper paradigm. So

again, that's just a sampling of the survey


Since everybody talked about time

and cost I thought it would be maybe

beneficial for the panel to see some

time/cost information. It's based on an

outsourcing model because that's what I have

access to and it's very important that the

outsourcing model be considered as part of

(202) 234-4433 WASHINGTON, D.C. 20005-3701
this because I think it is where all of the

time, the costs, the resources and the

expertise lie. You're basing these costs on

having a validated model, validated software,

a pressure-tested process and emphasis on

pressure, and a trained staff. And so making

sure that that's your baseline that you're

using created a couple of scenarios around

this, one of which is that you get all

submission-ready documents. For people who

live in this world they know that's a bit of

a dream, but content and format having been

reviewed and approved upstream of you. It's

also assuming that documents and data are

provided on a rolling basis. It's not this

giant bolus of work at the end. And for

Scenario 1 I used a smallish submission,

SNDA, a small 505(b)(2), even an IND around

150 volumes with about half of it coming from

an electronic source versus a paper source.

And you can see that assuming that

infrastructure is in place, electronic

submissions can not only be cheaper, about 15

percent, but you can also do them faster.

I created another scenario based

just on a bigger submission but again on the

same model and you can see that the total

cost again for electronic ended up being

cheaper, not as big a difference, about 11

percent. And also easier to compile, faster

to compile. I want to hover on this slide

just for a minute, though, so that everybody

knows. This is not a cheap thing to do.

Either way you look at it you're talking

about over a half a million dollars, and this

combined with a user fee that's creeping up

to around $900,000 is a tremendous amount of

money for a small company. And so that was

one of the points somebody asked me to make

and so if you consider some of the

incentives, you may also just want to

consider the overall cost. And this doesn't

involve any of the costs around doing any

data integration or data standardization.

(202) 234-4433 WASHINGTON, D.C. 20005-3701
So just to conclude and leave a

few minutes for questions, the major

impediments and the challenges kind of across

the board have to do with time from different

vantage points. Certainly there's cost

there. There is more expertise needed, but I

think there's also more expertise being

gained. Training is always an issue.

Getting the same folks out to meetings and

out to training can be difficult if they're

wearing multiple hats within a company.

Resources and I think process change is

probably underestimated. But electronic

submissions has delivered on its promise in

terms of major benefits, ease of use, reviews

both internal and external, the speed and I

think the one I'd like to highlight, the

accuracy which people doing electronic

submissions felt their submissions were

better. And I think that's certainly maybe

the takeaway from this is that you know we

have accomplished better submissions and so

that was certainly the point of all of this.

So I have a few minutes left for questions.

I do hope the panel found the information


DR. WOODCOCK: Thank you very

much. Questions from the panel? Randy


DR. LEVIN: Thank you for that

presentation. It was very helpful. And

thank you for all your help in electronic

submissions over the years in all the

different venues. You've been doing a lot of

work to help people move to electronic

submission so I want to thank you for that.


appreciate that. The template of the slides

now is called Levin-esque. You can find it.

DR. LEVIN: There's a little bit

of color in this slide.

MS. SMERKANICH: Well, there is.

More than you would have liked I would


(202) 234-4433 WASHINGTON, D.C. 20005-3701
DR. LEVIN: The question about the

cost and going back, and what about

implementing over time and reducing your

cost? How does that work or can you expand

on that a little bit?

MS. SMERKANICH: Well, since a lot

of the cost is based on implementing the

software, a lot of times that's going to be a

one-time cost so this is kind of taking that

out of - this is truly just costs around

production, doing the reviews, doing the

post-production changes. So it would be the

same consistently. In terms of time and cost

to implement the software and the people and

all of that, there's a lot of variables that

go into that and it changes from whether

you're a small company with a blank canvas to

a big company that has a lot of systems to

integrate, certainly.

DR. WOODCOCK: Randy Lutter.

DR. LUTTER: Could you go back to

the time and cost Scenarios Number 1 and

Number 2? I'd just like a little bit more

clarity if you would about exactly what these

numbers are supposed to describe. These are

the costs of preparing an NDA?


different size NDAs from the point of where

the documents are ready for if you want to

call it the publishing step. And electronic

in this case was both - an average actually

was almost a wash of doing an electronic NDA

with PDF tables of contents versus an ECTD.

It was pretty much a wash with that.

DR. LUTTER: So excluded from this

is any improvement in interactions with FDA

as a result of having submitted an electronic

document as opposed to a paper document?

MS. SMERKANICH: Yes, actually we

did have a minor couple of lines in there

about the ability to answer questions from

the agency based on a paper submission and

the ability to do re-analysis. And clearly

those are what bumped up the paper costs

(202) 234-4433 WASHINGTON, D.C. 20005-3701
because they tend to be faster and quicker

and more efficient interactions with the

agency if you do have both the SAS data sets

in the proper format as well as the

electronic submission with all the

navigation. So we did include that, yes.

DR. LUTTER: Thank you.

DR. WOODCOCK: Other questions?


DR. OLIVA: Is there anything

upstream that the company could do before

publishing or coming to you for publishing

that would help reduce these costs, either

paper or electronic?

MS. SMERKANICH: Yes. I think

that's an excellent question and I think a

number of the speakers, especially the

gentleman from CDISC have addressed that. I

mean, the more standardization that occurs

upstream in terms of the data and the

documents, the more efficient the downstream

process. And so you see companies that have

good standards, employed standards, standards

that everybody has trained on. They can

actually create submissions a lot faster than

companies that have to go and re-map data or

start with documents that have, you know, no

consistent field. But having said that, all

of those upstream costs aren't even in these

scenarios and they can be considerable as



DR. BUETOW: So from your

outstanding boots on the ground experience

here, what would be some critical things we

could do to facilitate this? So in other

words, obviously you're an evangelist, so

what are, you know, what is keeping you from

proselytizing at this point?

MS. SMERKANICH: Well, nothing

keeps me from proselytizing. A lot keeps the

message - well, I mean a lot of it is this is

perceived to be a nice-to-have I think by a

number of people. So when you see tier one

(202) 234-4433 WASHINGTON, D.C. 20005-3701
companies saying they don't have upper

management support, that speaks to me that

this is perceived as a nice-to-have. I think

again to speak on behalf of the people who I

claim to be here for, there are so many small

companies that their focus is - it's a day to

day thing. It's get this study done. It's

get my database clean. It's get my database

locked. It's get my tables and listings, you

know, done. And for them to see the big

picture and to see the end product, sometimes

they don't even consider it until they're

sitting at the pre-NDA meeting and someone

says to them, well, is your data going to be

in CDISC format or is it just going to be an

electronic submission. So a lot of times

it's a matter of priorities for them. They

have to spend their money wisely. So much

drug development is being done at such a

lower level now than when I started it's

amazing to me, and those people have a

totally different focus. They're focused on

the drug discovery and the development piece

and they're not necessarily thinking about

the marketing application.

DR. WOODCOCK: Randy Lutter.

DR. LUTTER: One follow-up. These

data give such a compelling argument for

industry to submit electronically.

MS. SMERKANICH: As long as you

have that top piece.

DR. LUTTER: Rather than via

paper. What additional action do you

recommend for FDA in particular? Are there

mandates or incentives that you think we

should provide?

MS. SMERKANICH: I think that, you

know, again I'll kind of echo what Dr.

Hardison said, that incentives I think for

small companies probably hold more weight,

but mandates for bigger companies hold more

weight. I think that also in terms of just

communication issues, knowing that the FDA

was 100 percent all review divisions,

(202) 234-4433 WASHINGTON, D.C. 20005-3701
reviewers behind this would also help a lot

because there still is this fear out there

that, well, my reviewer doesn't like this, my

reviewer likes paper, my reviewer isn't going

to use the data sets. So I think if we can,

you know, that's certainly - that's an easy

win as far as I'm concerned, but then they're

not my thousand people, so.

DR. BUETOW: Just real quickly.

So other than communications, are there

barriers? I mean because again, as someone

said there appears to be from a cost model,

you know.

MS. SMERKANICH: You're going to

pay one way or another.

DR. BUETOW: It's a no-brainer.

So what other barriers would you perceive

actually as a person who's literally doing

this that are actually slowing this down

other than communications?

MS. SMERKANICH: All right, I

think it's knowledge. I think if more people

saw what they look like at the end, and

especially with ECTD, which is my new thing

I'm evangelizing. If they saw the long-term

benefits of starting their INDs in ECTD and

moving forward with them I think that they

would, you know. And the fact that

redundancy goes away. I think that's one of

the things that has driven me absolutely

insane over the years is the amount of

redundance in the information that we submit.

DR. WOODCOCK: Other questions?

Thank you very much for mounting that survey

and presenting.

MS. SMERKANICH: You're welcome.

DR. WOODCOCK: All right. Before

I introduce our final speaker of this morning

I would like to reiterate and ask people

please do not wander off into the office

suites here, okay? We don't have a facility

that's secured, you know, between the public

access and our employees. So please stick to

the halls and so forth, okay? Our final

(202) 234-4433 WASHINGTON, D.C. 20005-3701
speaker this morning is Mark Scheineson who

is with Alston & Bird, LLP. Mark?

MR. SCHEINESON: Good morning.

Again, thank you for the opportunity to

present to this august panel. I'm

representing some of the little guys in

proprietary and generic drug development.

It's an honor to be back here with my former

FDA colleagues. I'm going to try to go

quickly both because Nancy's presentation

that you just heard was quantifying some of

the points I'm going to be making here and

because I'm standing between you and lunch.

The goal obviously that all of us

are discussing today is extremely laudable,

especially if it expedites reviews and allows

cross-application comparisons of safety data.

I would like to just get to the bottom line

and answer Randy's question or Dr. Lutter's

question which is, you know, mandates versus

incentives. From our perspective as

regulatory counselors to companies, we

believe what's holding back a flood of

electronic information that's ready to go is

a little bit of uncertainty on how to prepare

those applications and uncertainty on how

reviewers and uncertainty on how

investigators or inspectors would handle that

information and database. So from that

perspective, you know, any uniform standards

or guidance that the agency could issue that

helps specify some of those open questions as

well as the prospect of quicker reviews and

approval is enough to unleash a flood of

electronic data.

From the smaller companies'

perspectives there must be recognition that

the tasks involved in a transition remain

daunting, especially for those smaller

companies. Some of those tasks have been

noted before and are listed below in training

and installation of a validated document

management system that would be acceptable in

GMP inspections as well as in regulatory

(202) 234-4433 WASHINGTON, D.C. 20005-3701
reviews. It's also difficult to find

qualified vendors. I think the little guys

are concerned that the experts in the field

would be captured a bit by the bigger guys.

And I think we've seen that whenever new

rules are imposed or new requirements are

imposed that finding the experts, whether

it's, you know, NLE labeling or you know

written pedigrees or RFID or other, you know,

there are a limited number of vendors that

are qualified and they quickly get

inaccessible to the smaller guys.

Other elements to consider are the

large capital investment that is required to

put these databases into place. Most of

these companies would probably utilize CROs

to a certain extent to try to do some of this

work of them, but not everyone can afford

Parexel or Quintiles or some of the bigger,

more sophisticated vendors. Additional

employees would have to be hired. There's

somewhat of a cultural shift that's necessary

as was highlighted by management sign-off,

but once they hear that the best way to get

an application reviewed promptly or quicker

is electronic database, it wouldn't be hard

to get management sign-off in our experience


This is a cultural shift. It's

not unlike, you know, PDUFA for FDA and the

changes that that made to a culture and to a

methodology. Global harmonization is also

key. You know, if any standards that are

used can be global or harmonized that will be

an additional incentive. And the benefits

are largely a bit in the future and not

immediate. There's also a need for

universally acceptable standards as has been


It would be good that in GMP

inspections that FDA would rely solely on

electronic information. I know it's

important sometimes in the audit trail to see

changes and deletions and things, but you

(202) 234-4433 WASHINGTON, D.C. 20005-3701
know as all of us know that use Blackberries

or email, I don't know if that's reduced the

paper flow or people just make copies of

their emails. And you know, that's a

lifeline that would have to be cut both at

FDA and in industry.

The deadline that was suggested,

or a January 2008 date might be a little

optimistic. I'm going to quickly review

three issues and the thoughts of those. If

it's mandatory especially there needs to be

adequate transitioning, especially for those

smaller companies that don't have the

resources, but for large pharma as well.

That just won't allow sufficient time for

small and middle-sized companies to perform.

As was noted, the cost for

achieving electronic submissions is probably

higher and more burdensome, at least in the

short run. I think a lot of the responders

to Nancy's survey were polite in their

listing of code words, like "expertise" or

"cost" or "time." That "expertise" and

"cost" and "time" is probably on both sides

of this transaction at FDA as well as in

companies. And the pricing of these

services, especially if they're limited

vendors that have that capability would

likely be what the market will bear. And

that isn't going to be insignificant. In

addition, you know for submissions that are

already in progress there needs to be some

transitioning and some transition rules to

allow the scanning or converting or just use

of the paper until new applications can be

presented. It's both time-consuming and


If the need to consolidate safety

data is driving this, and that's very

important to be able to look across

applications, FDA might consider, and maybe

it is already using this, some kind of

standardized internal electronic reviewer

software or checklist that could be utilized

(202) 234-4433 WASHINGTON, D.C. 20005-3701
that would draw together some of the data

that the reviewers are compiling in their

application reviews. That could be done

internally without any need for rulemaking or


Secondly, time is an issue. The

timeframe must be realistic and should be

based on outside research. And this kind of

Part 15 outreach is a first step in that

regard. But there needs to be ample time to

hire consultants, to select the appropriate

submission software, to ensure and validate

its capability to roll out that software

before the submission preparation can begin.

You know, further delays could occur as I

pointed out from just finding the right

vendors. And of course for companies that

don't control it internally and some that do,

they can't always control the timeframe and

the priority that their submissions are

given. FDA hopefully will meet with the

vendors regularly to evaluate and to motivate

the process if it opts the mandatory route.

But new rules should be flexible based on the

progress in the marketplace. So from the

perspective of these small companies, they

believe that it should be no sooner than

three years if there is going to be a mandate

on which the agency - you know, three years

from the time the agency issues a final rule

or final guidance on electronic submissions.

Last issue is seeking some sort of

staged implementation. The agency should

allow for an ample implementation time.

There should be a built-in phase-in period

that's pretty specifically prescribed and the

agency should consider accepting paper to

accommodate those submissions. However, FDA

should ensure that equal treatment is

accorded to the paper submissions during that

phase-in period. Finally, FDA should ensure

that the electronic submission does not

become a vehicle for requesting companies to

include additional information or data beyond

(202) 234-4433 WASHINGTON, D.C. 20005-3701
what's already required. That was part of

the problem I think in the guidance that was

issued for the original Part 11 is that you

know, what records were covered by that

guidance became a concern.

So you've asked us a lot of

questions. Some questions for the agency to

go back and consider I guess would be some

kind of uniform specific software or systems

for submissions that could be recommended.

Uniformity obviously will ensure efficiency

by streamlining your processes internally as

well as the preparation of very complex

applications. On the legal and regulatory

side is what mechanism you'd use. I think it

would be a lawyer's preference to use the

rulemaking process, realizing how burdensome

that can be, but it would be great to go to

Part 11 and get all the information you need

as opposed to digging up, you know, numerous

guidance documents and other documents which

aren't really binding anyway on either side.

We also do believe that this is a substantive

rule in how it's likely to be structured as

far as having standards and otherwise

recommending uniformity. And in any

rulemaking there's always a small business

assessment, and it would be great if this

small business assessment could be based on

consulting externally. Small businesses and

kind of not speculating on the time or

resources that might be required. I remember

the HIPAA rule that we're all familiar with

that said it would cost, you know, two days

and $10,000 or whatever the cost would be of

that methodology. Also, the question is how

can this be harmonized internationally the

most quickly for companies that most of which

market overseas as well as in the United

States. Again, I thank you for the

opportunity to present these views. I'd be

happy to answer any questions that you have.

DR. WOODCOCK: Thank you very

much. Any questions from the panel? Kevin.

(202) 234-4433 WASHINGTON, D.C. 20005-3701
DR. FAIN: One quick question. I

was just wondering about the rulemaking

option. Mark, if you have any thoughts about

are there some significant legal issues we

would need to consider in writing such a


MR. SCHEINESON: I think you'll

avoid a lot of those issues by going the

rulemaking route and a lot of, you know,

potential litigation from companies that deem

the requirements to be too burdensome that

would delay the process. I think there are a

line of cases, Shalala cases in the late `80s

that kind of had a 5-prong test of what it

should include in the substantive area and

why this kind of rulemaking would be

substantive and not interpretative. Thank

you very much.

DR. WOODCOCK: Well, are there any

other questions? Trying to get away from us.

All right. Thank you all very much. This

concludes the morning session. Again, please

restrict yourself to the hallways and doors

and so forth. And we will reconvene and

begin promptly at 1 o'clock. Thank you.

(Whereupon, the foregoing matter

went off the record at 11:56 a.m. and went

back on the record at 12:57 p.m.)

DR. WOODCOCK: All right. We'll

be having three presentations at this point

and then we'll be having a break around

quarter of 3:00. Our first presentation is

from a set of presenters. I'm going to read

all their names and they're going to do an

ensemble presentation. Debra Bremer, Vice

President, Development and Medical

Informatics at Pfizer. Diana McKenzie,

Executive Director, Information Systems,

Amgen. Sue Dubman, Vice President of IT and

Informatics, Theravance, Incorporated. Dr.

J. Robert Beck, Vice President and CIO, Fox

Chase Cancer Center. Diane Paul, Patient

Advocate. Bob Renner, CEO, Liaison

Technologies. And Mark Adams, Senior

(202) 234-4433 WASHINGTON, D.C. 20005-3701
Associate, Booz Allen Hamilton. They will

have about 45 minutes and we look forward to

what they have to say.

MS. BREMER: So in one of the

previous presentations they talked about a

survey and summarizing the survey. We

brought all the survey respondents here.

Good afternoon. My name is Debra Bremer and

I'm the Vice President of Development and

Medical Informatics at Pfizer. And I do

thank you for the opportunity to speak this

afternoon on clinical information exchange

and electronic submissions, and especially in

the most coveted time slot of the day. I

guess we drew the short straw. We'll try to

keep this moving along. Maybe that's another

reason for the presenters, we'll keep it

flowing. And I'd ask respectfully that you'd

hold Q&A until all of us are done, our

presentations, and we do believe we've timed

it in such a way that we should allow time

for questions and answers.

So today I am presenting on behalf

of the CRIX community. CRIX, which stands

for the Clinical Research Information

Exchange is a secure shared technologies

standards-based service platform. We are

developing this exchange to deliver value-

added services to the biopharmaceutical

industry and for use in information exchange

between research organizations or business

partners and regulatory authorities although

CRIX can potentially deliver similar services

to support other regulated products.

We as a community formed CRIX

International as a not-for-profit entity just

this month with the vision of a public-

private partnership in mind. An entity such

as this is essential for the initial as well

as ongoing success of a clinical research

collaborative environment. The board of CRIX

International will be composed of

representatives from both the public and

private sectors and CRIX International will

(202) 234-4433 WASHINGTON, D.C. 20005-3701
govern and oversee the operations of the

services. While we've taken the initial

steps to form an entity that embraces a

public-private partnership in its very

structure, if desirable it could readily be

merged into an existing strategically aligned

non-profit organization. And I would say

that for more specifics on the role of CRIX

International we have submitted to the public

record for this hearing a CRIX executive

summary document where you can get more


We believe CRIX services will

deliver to users workflow efficiencies, lower

operational costs, reduced infrastructure

development expenses and faster drug

development times. CRIX can also contribute

to accelerated achievement of the goals of

the FDA Critical Path and NIH Roadmap

Initiatives. For example, a shared public-

private infrastructure provides a platform

for streamlining clinical trials and

potentially assisting with adverse event data

mining. In later testimony you will hear

examples of exactly how CRIX can streamlining

clinical trials.

The CRIX community encompasses a

broad range of stakeholders. Represented

within our community are hospital groups,

patient advocates, government agencies,

academic research centers, trade and

standard-setting organizations and industry

service providers. The CRIX community is

constantly growing and our goal is that it

will embrace all organizations involved in

supporting or overseeing clinical research.

CRIX is a collaborative shared

technology exchange. So it is fitting that

it grew out of two previous collaborations.

Both of the preceding collaborations were

looking to enhance and accelerate the process

of developing diagnostics and therapeutics.

In 2003 FDA and NCI initiated the Interagency

Oncology Task Force, the IOTF, to streamline

(202) 234-4433 WASHINGTON, D.C. 20005-3701
the development of cancer drugs. In 2004 the

IOTF focus expanded to include a shared

technology information exchange. At around

the same time the PhRMA trade association was

engaged in an effort called SEBIX, exploring

how to save time and money through the

development of a shared infrastructure and

means of information exchange.

Last year, NCI launched CRIX as an

initiative to demonstrate the value of a

shared infrastructure, leveraging the

momentum of the IOTF and drawing on lessons

learned from the SEBIX experience.

Development then began on the first service

module for CRIX, the Federal Investigator

Registry for Biomedical Informatics Registry

Data, which is why we call it FIREBIRD. To

date, FIREBIRD has been developed under NCI's

Center for Bioinformatics with knowledge,

technical expertise and resources contributed

by the broader CRIX community. Going

forward, we propose FIREBIRD as well as

future service offerings be developed and

overseen by the formal public-private

partnership represented by CRIX


The CRIX community is active and

thriving with broad stakeholder group

representation. The community holds the

shared goals of lowering drug development

costs and speeding new therapies to patients.

We believe that an information exchange such

as CRIX is essential to achieving those goals

and we believe that CRIX has the financial

backing, organizational sponsorship and the

right governance and operation structure to

make it succeed. We have demonstrated that

an exchange can work through the FIREBIRD

pilot and we propose to release the

production version of FIREBIRD in 2007. With

the governance structure that we're putting

in place through CRIX International, we are

confident that CRIX will continue to grow its

services within clinical research and could

(202) 234-4433 WASHINGTON, D.C. 20005-3701
be extended to support other regulated

products. CRIX will continue to grow its

community and deliver value to community

members and patients.

Next, members from the stakeholder

community will provide their perspectives on

the challenges of drug research and

development in the current environment and

the value of a third party information

exchange in addressing those challenges.

They will also provide information which

supports CRIX as an ideal model for

regulatory information exchange and

collaboration. Six speakers will give their

insights on CRIX, each bringing a different

perspective on the need for, benefits of and

governance structure of CRIX. We'll hear the

viewpoints of representatives from large and

small biopharmaceutical companies, academia,

the patient advocacy community, a provider of

exchanges and an external industry sector as

well as a service vendor. Thank you and I

will now hand over to Diana McKenzie from

Amgen who will provide the large

biopharmaceutical company perspective on


MS. MCKENZIE: Good afternoon.

I'd like to thank you for the time to speak

this afternoon also on the value that we

think CRIX offers to the broader research

community and to Amgen in particular. My

name is Diana McKenzie and I'm here to

represent Amgen. Amgen is a leading human

therapeutics company that serves the needs of

millions of patients worldwide. We strive to

be an entrepreneurial science-driven

enterprise dedicated to helping people fight

serious illness and we believe that by

aligning with the CRIX Initiative we

demonstrate our commitment to this


I believe we are all familiar with

the industry challenges we face in addressing

both the costs and complexity of research and

(202) 234-4433 WASHINGTON, D.C. 20005-3701
development. In addition to these, Amgen has

experienced significant and positive growth

over the course of the past several years.

In light of both these industry challenges

and our own growth, we are required to find

ways to improve our business processes and

our supporting IT infrastructure so that our

increased scale does not slow progress with

developing important therapies. These

improvements will help us to reduce the cost

of clinical research in an effort to maximize

both our investment in identifying new

therapies and making those accessible to


FIREBIRD serves as an excellent

example of the type of improvement that not

only addresses challenges for the broader

clinical research community, but also has

value for Amgen internally. We expect that

with the successful deployment of FIREBIRD as

displayed in the slide here we could reduce

the cycle time associated with capturing the

1572 NCV during investigator site initiation,

which is a common industry process, from

eight weeks to mere hours. In addition, we

believe that these reductions in cycle time

also lead to higher quality information and

improved ease of review for our regulatory

colleagues. Lastly, we recognize that for

our investigator community we demonstrate our

commitment to easing the administrative

burden for them with participating in

clinical trials, thus allowing them to focus

on the patient.

Our internal business case for

FIREBIRD, while conservative, demonstrates

the proposed reduction in clinical trial

costs at Amgen for both the current phase,

which is limited to the 1572 and investigator

CV, and future phases where we expect to see

greater return when expanded to encompass the

entire site initiation packet. We have not

incorporated additional CRIX capabilities

such as the patient recruitment matching into

(202) 234-4433 WASHINGTON, D.C. 20005-3701
our business case, however we believe that

these capabilities will only increase the

value of CRIX to the research community.

In summary, we believe that CRIX

is an effort, one of many being pursued in

the area of health information technology

deployments which benefits all stakeholders.

I've had the privilege of working in this

industry for 20 years and I have learned a

great deal about what it takes to

successfully deliver IT-enabled solutions

within research and development. I have been

actively engaged in discussions to advance

the exchange concept since 2003. It is

because of the diverse CRIX community

represented here today, the quantifiable

business case we have developed and our

collective desire to ensure all stakeholders

benefit in some way that we believe the CRIX

Initiative can successfully help us to

address the rising challenges of speeding new

therapies to patients and reducing the cost

of research to maximize their access to those

therapies. Thank you. I would now like to

turn the presentation over to Sue Dubman from

Theravance who will provide the small

biopharmaceutical company perspective on


MS. DUBMAN: So I'll lower this a

little. I used to be taller. I would like

to thank the FDA and the NIH for the

opportunity to speak here today. My name is

Sue Dubman and I know quite a few people in

the room. I was actually one of the

originators of CRIX when I worked at the

National Cancer Institute and have been a

supporter of the need for CRIX from its early

beginnings in 2004 and actually suggested

that we do a conga line around the room

because this is an important initiative.

Currently I'm Vice President of Information

Technology and Informatics at Theravance, one

of approximately 900 mostly small to medium-

sized biopharmaceutical companies in the San

(202) 234-4433 WASHINGTON, D.C. 20005-3701
Francisco Bay Area. Where it happens to be

38 degrees this morning.

So just some facts. Small to

medium-sized biopharmaceutical companies like

Theravance play a critical role in new drug

development. And according to statistics

compiled by PhRMA and bio trade associations

as much as 70 percent of new drugs in major

disease categories are being developed by

biotechs, and many of these, if not most of

these are being developed by small to medium-

sized companies. As such, any new

capabilities that allow a small

biopharmaceutical company and its many

partners, CROs, large biopharmaceutical

companies, academic research institutes and

others, to improve their ability to bring

safe, effective drugs to market sooner and

for less expense are a win-win-win for

everyone, but especially patients.

So talking with many other small

biopharma and having just submitted our first

NDA electronically to the FDA, I believe that

CRIX has the potential - and it really is

potential - to not only streamline electronic

transfer of data to the FDA, but also to

improve the processes for collaboration with

industry partners in the future. I think

CRIX can provide cost savings and scale

efficiency if CRIX supports use and adoption

of common standards for regulatory submission

by all stakeholders. That's industry,

government and academia. Today we find a lot

of inconsistencies and overlap among the

various standards and the interpretation and

use of these standards by multiple parties.

So CRIX with its vision of a shared

infrastructure and tools used by all has the

potential to address these issues, thereby

increasing the efficiency of exchange and

making interactions and decisions easier,

faster and cheaper.

Also, CRIX because it is

standards-based can also potentially make it

(202) 234-4433 WASHINGTON, D.C. 20005-3701
easier and more cost-effective to collaborate

with our many partners. Standards by their

nature facilitate the exchange of

information, that's one of the reasons you

use standards, and so - but today there's a

very steep learning curve and cost to adopt

these standards.

If the shared CRIX infrastructure

can provide capabilities to create and use

data in standard formats and make validation

of compliance with standards easier, then

CRIX will have a favorable impact on the cost

and time it takes to bring new treatments to

market. Finally, the CRIX community has the

potential to ensure that the standards

address the needs of all stakeholders, not

just big pharma. And so I think it gives a

voice for small biopharma, and I think that's

going to become important in the future.

One side benefit of CRIX that I

don't think we even understood when we

started this effort is that there are other

services - once you have an electronic

infrastructure for regulatory submission and

for collecting information in standards

structured format, you have the potential to

leverage those systems in the future. So

CRIX also provides potential gains in

interoperability of clinical systems in the

future. And so for example FIREBIRD, the

CRIX clinical investigator registry

potentially provides and authoritative source

of investigator, site and protocol

information for connecting patients to

trials. And this is critically important as

recruitment of patients to clinical trials is

a huge problem. According to market

research, upwards of 40 percent of clinical

trials costs out of nearly $6 billion spent

annually on clinical trials is tied directly

to patient recruitment, and greater than 80

percent of clinical trials have major

recruitment delays. And these costs aren't

going away. I have more detail here in my

(202) 234-4433 WASHINGTON, D.C. 20005-3701
testimony, but I won't read it. But we need

to find a new way. I think that's the bottom


We need to find a better, more

cost-effective ways to do drug development.

CRIX addresses one piece of the overall

problem. With the CRIX Initiative we have

the potential to stay focused on the science,

the medicine and most importantly the patient

rather than the information technology

infrastructure. And this is critical to all

of us because some day all of us, our family,

our friends and our colleagues will be

patients. When that happens, we will need

and want the best possible, most cost-

effective and safe treatment. And as my

mentor at the NCI Dr. Ken Buetow always says,

the question is not why now, but why not

sooner. With that I will hand over to Robert

Beck who will provide the research center

perspective on CRIX. Thank you again for

allowing me the opportunity to speak.

DR. BECK: Thank you, Sue. I

would also like to thank the FDA and the NIH

for the opportunity to speak today. My name

is Bob Beck. I serve as the Vice President

for Information Services and the Deputy

Director of the Population Sciences Division

at Fox Chase Cancer Center in Philadelphia.

I'm also an outside director of IDM, a

development stage public biopharmaceutical

company. I'm familiar with the CRIX

Initiative through activities in the

strategic planning workspace of the Cancer

Biomedical Informatics Grid, or CABIG. I'm

speaking as an individual although attempting

to represent the academic health research

community. We believe that the Clinical

Research Information Exchange offers an

unparalleled opportunity to strengthen the

academic clinical research community.

Managing clinical research at an

academic cancer center has logistical,

regulatory and management challenges. At Fox

(202) 234-4433 WASHINGTON, D.C. 20005-3701
Chase we handle over 150 active protocols

from individual investigator-initiated

projects to industry-sponsored trials, large

NIH cooperative group trials. Each protocol

of course has its own forms, procedures,

addresses, rules, regulations. Investigator-

initiated protocols are particularly

difficult. Although the sine qua non of

clinical research at an NCI-designated cancer

center, these are quite challenging to export

beyond the single institution where they're

developed due to lack of standardization

across institutions or protocols. Sponsored

trials, while offering some limited

standardization within a corporation's

offerings, generate their own problems. As

one colleague comments, every time I work

with a new sponsor it costs me money. That's

because new sponsors mean new forms, new

organization of trial data and other

inefficiencies that standardization of course

could address.

We recently celebrated the 20th

anniversary of Fox Chase Partners, a

consortium of over 20 institutions in

Pennsylvania, New Jersey and Delaware. Our

partners participate in clinical research,

add many patients to clinical trials and form

the substance of a community clinical

oncology program, or CCOP. However, the

problems of clinical trial management are

multiplied twenty-fold when extending them to

this community. This group could function as

a single collective entity with improved

automation and standardization of protocol

management and investigator data.

CRIX, in particular the FIREBIRD

initiative, offers several positive features

for busy academic clinical research

organizations. Cost savings can be achieved

by reducing paper shipment, reconciliation

and storage costs, by creating a single

electronic master repository for investigator

regulatory documentation for the entire

(202) 234-4433 WASHINGTON, D.C. 20005-3701
institution and its partners, and by

minimizing the number of documents the

coordinator or investigator must manage.

Increased standardization can arise through

the establishment of a single industry portal

available for use between all sponsors, all

sites. This would particularly help our

partners, our CCOP. Standardization across

sponsors of technical management of

regulatory documentation would be a plus as

would the standardization of

idiosyncratically designed forms.

Establishing safe credentials that can be

used for other systems would represent a

further important step. Also, improved cycle

time is of import not only to industry but to

busy academicians. Reuse of frequently used

information leading to reduction in the

number of errors is part of FIREBIRD's

appeal. Also, accelerated regulatory

document package completion will shorten the

time required to initiate a new study,

whether initiated by an individual

investigator, a cooperative group, or an

industrial sponsor. Most attractive to us is

the reduction of redundant sponsor requests

and data entry for routine contact

information. We of course want to enter and

maintain our static data once for all

sponsors. When changes then occur, updates

through a central portal would provide

accurate information to all our sponsors


Automation of the clinical

research enterprise at the federal level

allows more protocols and sponsors at the

academic site without increasing staff

resources. FIREBIRD and CRIX promise to

minimize paper and paper-handling costs and

thus to simplify the entire compliance

environment. This in turn could enhance

patient safety even as it facilitates

investigator compliance with appropriate but

burdensome regulation. Most exciting is the

(202) 234-4433 WASHINGTON, D.C. 20005-3701
potential of these systems to leverage other

information management and informatics

investments, such as the electronic health

record. Fox Chase Cancer Center as part of

its participation in CABIG is developing a

clinical data warehouse that can take results

of translational and clinical research

directly into consensus records populated by

the electronic health record. This project

can be rapidly accelerated by the sort of

standardization offered and promised through

CRIX. This project can be rapidly

accelerated with the sort of standardization

offered through the Clinical Research

Information Exchange which will leverage

institutional electronic record systems and

clinical research technologies for the

benefit of all participants. Now I'll hand

it over to Diane Paul, a patient advocate who

will provide her perspective on CRIX. Again,

thank you for the opportunity to speak today.

MS. PAUL: I'd like to thank the

FDA and the NIH for the opportunity to speak

to you today. My name is Diane Paul and I'm

a 13-year survivor of advanced ovarian

cancer. While I belong to a number of

different advocacy organizations, I am

speaking to you today as an individual. I am

a member of the CRIX steering committee and

have been for the last year and a half. Over

the last 13 years I have witnessed firsthand

the roadblocks that exist in bringing

treatments to patients. Time after time I've

seen patients willing and able to enroll in

clinical trials who have to go back on

standard treatments because of the delay in

opening of trials. I've seen patients

waiting for new drug approvals die before

those drugs get to the marketplace. When

treatments do receive approval, they are

often no longer of interest to patients

because newer, less toxic methodologies are


Prior to January of 2000 I was

(202) 234-4433 WASHINGTON, D.C. 20005-3701
employed in the Information Technology

Department at the Fashion Institute of

Technology in New York City as a systems

analyst. When I first came to FIT in 1979,

there were many problems with admitting and

enrolling students in a timely fashion. The

different silos of information which the

admissions, financial aid and registrar's

offices maintained caused similar problems to

what I see as issues in the drug development

and approval process. The first step in

solving these problems was to get

representatives from each area to work

together towards a common purpose.

Communication between the FDA, the research

community and the pharmaceutical industry is

the first step in breaking down barriers

which hamper them all. The CRIX Initiative

provides the framework for communication so

areas of agreement can be documented and real

problems can be defined and solutions

explored. With this type of framework all

three entities can work together for the

public good.

The CRIX Initiative will enable

the standardization and electronic transfer

of data. This will both speed the

development and the approval process. It

will service patients in a number of ways.

Treatment choices often must be made - sorry

about that - in a relative short period of

time in order to best serve the patient.

Patients are often unable to enroll in trials

when they and their doctors deem it

appropriate because of the delay in the

opening of the trials. Timely openings will

improve accrual rates. This will have a

ripple effect on the time it takes to

complete the trial and get information about

efficacy and toxicities to regulatory

agencies and hopefully to increase the speed

with which treatments are available for the

public. Cost savings resulting from

standardization and electronic transfer of

(202) 234-4433 WASHINGTON, D.C. 20005-3701
data should allow for more investment in new

therapies as well as savings in treatment

costs for the public. Better standardized

information from trials should provide more

meaningful treatment information for doctors

treating patients in the clinics following

the completion of trials and treatment


Currently cancer patients and

their doctors need to search various

databases in order to find trials which are

open and available. One way the CRIX

Initiative should help is to establish ways

for patients to find appropriate trials

across both the private and the public

arenas. Matching patients to trials across

the health continuum will be a big step

towards improving accrual times for trials

and the ability to search the public and

private sector at the same time will allow

better information for patients and their

doctor to determine what trial choices exist

and what is the best choice for the patient.

Patient advocates work on many

levels of research and treatment development.

The CRIX Initiative includes patient

advocates in all areas of its governance

structure to assure that patients' concerns

will be heard and met by the initiative. I

volunteer my time to the CRIX steering

committee because I believe it is of utmost

importance that we change the current system

to better service patients. I feel the CRIX

Initiative will help to do that. I will now

hand over to Dan Ruggles of Liaison

Technologies who will provide a perspective

on how other industries have dealt with the

establishment of exchanges. Thank you.

MR. RUGGLES: My name is Dan

Ruggles from Liaison Technologies speaking on

behalf of Bob Renner, our CEO, who could not

make the meeting today. I'd like to thank

the FDA for the opportunity to speak today

and to share our experiences in setting up

(202) 234-4433 WASHINGTON, D.C. 20005-3701
and operating a consortium exchange. Liaison

Technologies, formerly known as

ForestExpress, was founded in June of 200 and

is headquartered in the U.S. with operations

in Europe. We are a private for-profit

company. We act as a shared infrastructure

provider for an exchange will I'll define

later in this discussion. Liaison has been

funded by Global 1000 companies with a clear

intent to foster and promote standards with

our implementation of an exchange. Today we

operate a set of systems that process about

3.1 million messages per month with over

3,100 connected companies.

There are two types of

architectures in an exchange. They're either

point-to-point or a hub-and-spoke. The

point-to-point model involves a series of

direct connections between two unique

endpoints. Most industries have deployed the

hub-and-spoke model for their exchange. As

can be seen from the table, these exchanges

have gained significant momentum and business

model validation over time, and involve the

active participation of competitors to work

together to help drive down their own

respective costs and improve their cycle

times. Some of the common themes of these

exchanges are it does require industry leader

participation actively leading with visible

high-profile companies, and it does require a

clear separation between the standards

bodies, generally not-for-profit, and the

operating the exchange, which generally

follows a profit model. Significant

investment is required if you want to start

one from scratch. Nearly all successful

exchanges report investments of between $100

to $250 million, mostly in startup costs,

before reaching substantial benefit for the


Now I'd kind of like to talk about

the functions. There are usually three

functioning parts in an exchange. The

(202) 234-4433 WASHINGTON, D.C. 20005-3701
initial driving force is the top one, the

industry association, which sets and promotes

standards, governance and coordinates

funding. They frequently act as the voice of

the industry, marketing champion. It's a low

financial investment initially, but requires

a significant amount of time with senior

industry leadership resources. On the lower

left to yours is the shared infrastructure

operator, Liaison. We actually operate and

execute the shared platform which we helped

build and design. And last there's the

certified solution providers. This is

nothing more than a typical leverage model

using economics to speed the adoption of the

standards and in some cases the adaptation of

the standards over time based on real world

experience. Our experience has shown that

validation and enforcement activities for

industry standards can only be maintained

through a shared services model. Without the

exchange, certified solution providers and

individual companies will deviate from the

standard to accommodate their individual

needs. The exchange acts as a control point

to localize those exceptions. Companies used

to the exchange can provide advantage in

terms of cost economies, quicker to market

and improved cycle times.

I'd like to share with you some of

our key learnings. From our experience and

observation of other exchanges as well as our

own, getting started required strong industry

leadership and commitment. This model

requires a long-term view. The economic

benefits may not likely be reached for five

to seven years, although a breakeven

operation may come within two to three years

if managed carefully. The consortium

exchange is a proven model as exemplified by

my previous table. The three key

stakeholders all have to participate, which

is the association, the shared infrastructure

provider or the exchange operator and the

(202) 234-4433 WASHINGTON, D.C. 20005-3701
certified solution provider. And lastly,

it's a long-term commitment by all

stakeholders for success. Now I'd like to

hand over to our last speaker, Mark Adams,

from Booz Allen Hamilton, to provide a

service provider's perspective on the CRIX

Initiative. Again, thank you for allowing me

the time to speak.

MR. ADAMS: I want to thank

everybody again for the opportunity to speak

to the panel from the service provider

perspective. As said, my name is Mark Adams.

I work for Booz Allen Hamilton which is a

management consulting company located here in

the Washington, D.C. area. I'll be

addressing these questions in my remarks


The CRIX service delivery

providers will be responsible for the

construction, deployment and operation of

tools and components of the CRIX service.

The providers will also ensure that the

development and service provisions meet the

compliance requirements that are outlined for

them. The provision of these services is

provided to the community of users via a

contract which will provide the platform for

the service level agreements and other

activities. CRIX International would select

one or more service delivery providers to

deliver these services to the CRIX community

and would contract with these service

delivery providers providing structure,

scope, governance for their activities.

These service delivery providers must be

independent from CRIX International and can

either be for-profit or not-for-profit


One of the key recommendations

from industry is to select a single prime

contractor to act as a service delivery

provider. This service delivery provider

would then subcontract with the entities

providing the specific subcomponents to

(202) 234-4433 WASHINGTON, D.C. 20005-3701
comprise the CRIX services. The service

delivery provider subcontracts with specialty

vendors, then provide the key project

activities. Thus each subcomponent can be

supplied by best-of-breed groups without

burdening the CRIX community with the need to

identify and contract individually with all

of them. This includes software development,

deployment, hosting, support services and


Industry regards a prime

contractor as the most efficient way to

provide the CRIX collection of service

modules. This also ensures that the

interactions between those providing the

specific services to the community are

consistent and flexible, and also allows new

services and/or service providers to be added

to the process without disrupting or adding

complexity to the users of the CRIX services.

Additionally, consistency and synergy can be

realized by providing shared help desk,

software support and other shared services

for all of the tools and capabilities that

are provided to the CRIX community. In this

simplified contracting and funding approach,

all the users of the CRIX services would

execute a single contract with the prime

contractor for each of the desired modules

which additionally provides for a single

channel for the collection of service fees.

A single point of contact also provides a

mechanism for single, well-defined issue

escalation and dispute resolution mechanisms

and provides for flexibility in resolving

such issues quickly and effectively.

The funding model for CRIX

International would depend on a set of

initial contributions from industry founders.

These contributions would provide the

foundation on which the initial organization

would be formed and then service fees for

usage would provide an ongoing revenue stream

to support module operation support and

(202) 234-4433 WASHINGTON, D.C. 20005-3701
maintenance activities. These service fees

would go to CRIX International and would be

based on the levels of usage and an

organization's ability to pay. A CRIX user's

ability to pay would be measured by the

firm's global annual sales where firms with

higher sales are expected to pay more for

services. The annual fees for founders and

non-founders would be set in relation to the

projected market share in order to minimize

the capital investment requirements and

achieve a self-sustaining cash flow by the

third full year of operations. This will

ensure that the services provided by CRIX

International are available to all

organizations regardless of their size and

stature. As the CRIX organization reaches a

self-sustaining level, the fees can be

reduced or excess funds may be directed

towards the development of new services. The

process of selecting commercial suppliers for

the prime contractor and for the

subcontractors would be done via standard

competitive award process which will ensure a

fair, open and objective review and selection

of vendors in all the roles. This ensures

maximum fairness in the process and the best

value to the users of CRIX services.

Through the proposed process the

community involved in all the process and

phases of the new module development, both

that carried out internally and externally to

the CRIX International. From a governance

perspective, once the prime contractor is in

place, industry recommends establishing an

advisory board to ensure that the explicit

implementation meets the needs of the

customers as well as CRIX International.

This would provide management oversight and

leadership connecting the community with CRIX

activities. To facilitate this, it is

proposed that the CRIX International will

oversee the selection of modules, the service

modules, in an inclusive manner that ensures

(202) 234-4433 WASHINGTON, D.C. 20005-3701
that appropriate business analysis,

incubation and development take place. This

provides a mechanism for successful, cost-

effective development and deployment that

meets the customer's needs and adheres to

regulatory requirements.

So in summary, the proposed

structure yields several advantages. Third

party oversight ensures vendor neutrality and

cost efficiency. Secondly, single service

delivery provider reduces the contractual

complexities and it mitigates the risks

associated with coordinating multiple

vendors. Thirdly, the approach promotes an

open dialogue and involvement that is

inclusive of the CRIX community stakeholders

in ensuring that all interests are met.

Finally, the investment review, incubation

and development oversight ensures service

module selection is technically and fiscally


Thank you for allowing me the

opportunity to speak to you today. My

colleagues and I would be pleased to field

any questions the panel may have. And I'll

ask Deb up. Thank you.

MS. BREMER: I thank you all and I

will open it to the panel for questions. Dr.


DR. MITTLEMAN: I'd like you to

speak a little bit to how you see the role of

government in this. I mean, I think there

are a lot of enormous strengths that can be

brought by having a lot of aspects of the

private sector be involved. And how the

research community and the FDA interface

really will work here and how they can both

provide scientific input, expressions of

needs and opportunities with the private

sector folks is a little less clear to me

given the governance structure you're


MS. BREMER: Sure. Well first of

all, I think the first thing we want to say

(202) 234-4433 WASHINGTON, D.C. 20005-3701
is we think it's absolutely necessary that

the government agencies - thank you, Diana.

I appreciate that. My tall friend. But that

government agencies, particularly the FDA,

are a part of this and so I would say there

are a couple of different ways. One is

formal participation in the governance

structure. The bylaws do call for a board of

directors that has I believe up to four seats

available for government agencies. And

again, we would foresee that certainly one of

those could be the FDA, perhaps NCI as well

and others. Secondly, we would also be

looking for active participation in the

exchange service itself. And I think most

people feel, you know, in terms of lessons

learned and things like that, if that is not

a part of the model going forward we would be

less than successful. And I think that

active participation starts very soon

building on the foundation of the memorandum

of understanding between FDA and NCI with

FIREBIRD in particular. And we hope that we

could kind of exploit that, expand that and

make FIREBIRD available for commercial usage

as well.

MS. PAUL: I also think it's

critical to the public that we have

governmental involvement in this project, I

think particularly since it's going to

probably involve sensitive data. Having the

government watchdogs on the various

committees is I think important. It will

make the project much more palatable to the


DR. MITTLEMAN: To extend that

just a little bit, have you also thought

about other governmental agencies, EMEA, et

cetera, and how you want to interface with


MS. BREMER: Yes, we have. So as

it's called CRIX International, the emphasis

is to be able to ultimately operate in a

global environment. And I think it was Mark

(202) 234-4433 WASHINGTON, D.C. 20005-3701
that talked to the concept of advisory

councils and advisory boards. And our

thoughts are that essentially we would have

regional - over time we'd have regional

advisory councils so they're, you know, the

first would be what we have here in the U.S.

and then as that broadens we would foresee,

for example, the European arena having an

advisory council. Those advisory councils

also have board representation, one from each


DR. WOODCOCK: Dr. Lutter.

DR. LUTTER: Earlier speakers used

the words "mandate" and "incentives" and I

haven't heard either word used in response to

the question what role for government. So

maybe I could ask you are you opposed to

mandates or incentives, or is there some role

for such actions by FDA or other federal


MS. BREMER: I think mandates will

be on a case-by-case basis depending on the

service that we're talking about. There will

be some services that in order for them to be

successful I think it'll you know, it'll

become intuitively obvious that it would have

to be a mandate in order for it to achieve

its outcomes. Others will you know not have

those same characteristics. In terms of

incentives, I think the incentives are

essentially the benefits that we've talked

about that are to be gained from something

like this. For example, lowering the

development cost for you for independent

entities, or lowering the development - the

review costs from a regulatory agency kind of

perspective. So besides the incentives in

terms of speeding drugs to patients which is

something that has you know in our hearts

also feels good, there are the financial

incentives to the folks that are involved.

DR. WOODCOCK: Other questions.

Ken Buetow.

DR. BUETOW: I assume it's safe to

(202) 234-4433 WASHINGTON, D.C. 20005-3701
assume you guys are interested in third party

solutions? You wouldn't be opposed to that?

MS. BREMER: I think you could

assume that.

DR. BUETOW: I'm intrigued what

you all see as the relationship of an entity

such as this to the existing standards

development organizations and how this group

could, would play in the further generation

and propagation of those standards.

MS. BREMER: Bob, if you're


DR. BECK: I see us as an umbrella

organization that serves as a very large tent

under which companies, academic institutions,

the public and government can essentially

coalesce. And so standards bodies, the

governance structure will be reviewing their

work and basically we will provide a stimulus

I think to standards development. I can't

see any other role for an organization like

this to be other than a stimulant, not an -

and maybe, you know, reciprocal participant

on some bodies, but not actually doing the

standards development themselves.

MS. DUBMAN: Just to echo that, I

think you know we're going to try to leverage

whatever is out there and not reinvent the

wheel. And so we are using for example CABIG

guidelines in developing solutions so that

they're interoperable. And you know, I think

we're working you know closely with CDISC and

ICH and other organizations to make sure that

we're not, you know, again reinventing the


MR. RUGGLES: One additional quick

comment and that's, you know, when we're

talking about some of these additional seats,

I think it's envisioned that standards

organizations like CDISC and others would

have opportunities to actually be on the

governance board of the organization. So to

make concrete some of these recommendations

they'd participate.

(202) 234-4433 WASHINGTON, D.C. 20005-3701
DR. WOODCOCK: Ken, do you have

more? No. Okay, Armando?

DR. OLIVA: I recognize that CRIX

International just came into being this

month, so maybe this is an unfair question.

But there are many stakeholders that

obviously would like to see CRIX succeed and

to borrow a phrase from someone else on the

panel, why not sooner. And my question is

this. My real question is this. If a

stakeholder sees the need and maybe even has

the resources to develop a service, a shared

service, what's your advice on how that

stakeholder should proceed at this point?

MS. BREMER: So first of all we

welcome stakeholders that see a need and are

willing to get something started, to incubate

it so to speak, and FIREBIRD is an excellent

example of essentially that kind of a

concept. So we don't discourage it.

Certainly though it would be very helpful the

more stakeholders at large have an interest

in something, the better the model is going

to be in terms of usage. So if there was,

you know, a single stakeholder that wanted

something, but there wasn't broad interest in

it, it may not be a viable long-term model.

So there's a variety of different things that

we'd use to determine the order of services

to be brought into the CRIX environment. And

certainly the broad stakeholder community

represented in a large way through the

advisory councils, in a very focused way

through the advisory councils and through

other input will help advise the board of

directors in terms of what are the

prioritizations so to speak of the next

services to be provided. But we certainly -

we don't discourage that. We welcome that.

DR. OLIVA: So is there a process

or is there a mechanism right now where new

services can be proposed or discussed and

entertained for prioritization?

MS. BREMER: Back to your point of

(202) 234-4433 WASHINGTON, D.C. 20005-3701
just being formed formally as of about 11

days ago. I would be remiss if I said that

there was an active process. To date what we

have are the startings of the board of

directors with the founding members and our

next task is to round out the board of

directors, get some officers in place and

then start working on service providers. But

in parallel to that we will obviously be

working on processes. As soon as we get a

few more people onboard that is certainly

part of our plans.

MS. DUBMAN: At the last CRIX

meeting we actually did discuss an additional

new service to bring forward for CRIX and

actually the discussion actually ended up

talking about what is the process. I think

we will get those processes in place quickly


DR. WOODCOCK: Dr. Buetow.

DR. BUETOW: So to complement the

question on services, there's obviously the

interest in the prioritization of what

services come onboard, but what about - I

think from previous speakers there was

discussion about also, concern about access

to services and what would be in place to

guarantee that - especially it sounds like as

this is proposed there might be - "monopoly"

is the wrong word, but perhaps singular

source of services. How would you guarantee

that services could be provided to all of the

stakeholders in a timely fashion and that

it's not a few 800-pound gorillas that

consume all of the resources and then smaller

players are actually left on the sidelines?

MS. BREMER: So first of all I

would say again, substantial emphasis will be

placed on what the advisory council and the

CRIX community at large recommends. I can't

repeat enough, and I do represent

individually one of those big 800-pound

gorillas, but I can't emphasize enough that

we do not see ourselves being successful in

(202) 234-4433 WASHINGTON, D.C. 20005-3701
this not-for-profit kind of entity, in this

overall shared infrastructure approach,

unless there's active participation by all

stakeholders. So it is not in the big

gorilla's best interest if we do things that

preclude folks, you know, that aren't those

big gorillas from participating.

MS. MCKENZIE: And I'd also like

to add that as we think about the overarching

governance body, our expectation is that we

have government representatives on that body

I think first and foremost. And I also think

that in forming CRIX International our goal

is to make sure that there's an oversight

body that's looking out for the interests of

the community. And one of the expectations

that's either outlined in one of the slides

or in the executive summary is that we expect

to put pricing in place that is actually

tiered based on the size of the different

organizations that would want to take

advantage of the services.

MS. PAUL: And the governance

structure includes patient advocates at every

level including on the advisory board. And I

and most of the people I know who are patient

advocates take that very seriously and I

think Ken you know we will be there watching,


DR. WOODCOCK: Other questions

from the panel? Dr. Lutter.

DR. LUTTER: The services that you

offer would presumably be for sale at cost

because you're a not-for-profit. Would they

be available to any interested party, or only

people who are in some sense members or

subscribers to CRIX?

MS. BREMER: We see the services

as being offered to, as the bylaws point out,

all members of the CRIX community. And there

are different kinds of members of the CRIX

community, from founding members as we talked

about, you know, are onboard today, to

associate members. And the associate members

(202) 234-4433 WASHINGTON, D.C. 20005-3701
are essentially the users of the service. So

there is a formality to saying, not a

bureaucratic formality, but there is an

active choice of being a part of the service

and paying fees, again, commensurate with

ability to pay so to speak. So there is, you

know, an active choice. I'm not sure if I've

exactly answered your question, Randy.

DR. LUTTER: For the non-associate

members, they would have no access without

becoming associate members?

MS. BREMER: That's correct.

DR. WOODCOCK: Dr. Buetow.

DR. BUETOW: In terms of risk

mitigation, I'm curious. This is, again, a

relatively large centralized activity. What

are the strategies that actually mitigate

risk with respect to the availability of

these services if you all decide at some

point in time that you don't like this?

What's the role of open source? What's the

role of open standards and other components

in what you all would develop? Would these

services, infrastructure and other components

be able to be run by other people besides

this so-designated entity?

MS. BREMER: Certainly. If you

think about the prospect of we are looking

for initially a core service provider, again,

we would be giving that service provider

oversight in terms of the order of priorities

of services and things along those lines, but

they would be essentially running the day-to-

day operations, providing the user support,

et cetera. So if the, you know, although we

don't anticipate - we anticipate this being,

that we're all in for the long haul, but if

that should happen, the core service

provider, if the company chose to do so,

could potentially keep things going.

DR. BUETOW: So would the services

definition be in the public domain? Would

there be a plan to share the services

definition as part of this public domain?

(202) 234-4433 WASHINGTON, D.C. 20005-3701
MS. BREMER: Absolutely. Yes.

DR. WOODCOCK: I have a question.

I found the presentation from Liaison

Technologies very interesting. My question

is this set of services involves to me

something beyond what you presented as types

of services that were mainly for industrial

partners. It involves health, it involves

patients, it involves healthcare providers

and so forth. It seems to add some extra

dimensions. Have you seen any models where

there are many stakeholders with major stake

beyond the industrial partners in the

exchange? Maybe the financial networks, for

example, where they're all moving their money


MS. BREMER: Dan, do you want to

come to the microphone?

MR. RUGGLES: Yes, the financial

industry, VISA is an excellent example on the

security side of it, and that's probably one

of the better examples out there. We do buy

and sell and keep track of things in that

regard, and in terms of confidentiality, in

some cases that information has to be

encrypted. So there is some similarity, but

nowhere near some of the things that you've

talked about or discussed here today.

DR. WOODCOCK: Complexity. It

strikes me those financial networks for

example, there are users such as me. I mean,

I'm just, I'm not really a user because I'm

always using some industrial partner's

whatever, setup, but is that how it would be

envisioned here, that a lot of healthcare

types and the patients and so forth wouldn't

have to be members of the CRIX community,

they would simply be users of this network on

behalf of the entities that set it up?

MR. ADAMS: So we create, as an

example, we create services as a solution.

And one of those services could be very much

the way you do your banking, as an example.

So I'm trying to use it as an illustration as

(202) 234-4433 WASHINGTON, D.C. 20005-3701
opposed to I don't want to design or

architect anything. But yes, it could be a

web-based sort of service that's going back

to the pharmaceuticals also feed information

to security. Kind of keeps things separated

as to what you can look at and what you can


DR. WOODCOCK: Thank you. Are

there any other questions? Armando.

DR. OLIVA: Yes, I'd like to probe

a little bit this whole idea about

competition. Using the VISA example. As a

user of VISA, I can shop around for the best

rate or the lowest annual fee or the perks.

And how would that - how do you see that

level of competition? How would that work in

using the CRIX model?

MR. RUGGLES: I think the key way

in which that would work, and one of the

reasons that the organization was sort of

designed with this multiple tier aspect to it

is such that the stakeholder is participating

in it, and that includes the government

representatives. The representatives broadly

of the individual stakeholders then initiate

a competitive and an ongoing competitive

process at two layers. One for the general

overall service provider who's going to act

as sort of the prime contractor, but also

through that to the folks providing services

up through that process. So there's two

layers essentially of traditional competition

going on to ensure that both the quality of

the service meets the requirements of those

stakeholders. Maybe unlike many companies,

the stakeholders themselves are those to whom

the organization is beholden, not you know

shareholders out in the world. But those

stakeholders then initiate that competition

that can be ongoing. So essentially you're

getting competition for the services that are

then provided more or less through the

central mechanism. So it's not quite the

same where you could go to any of several

(202) 234-4433 WASHINGTON, D.C. 20005-3701
shareholders any time, but there's

competition for those roles and that's

ongoing and controlled by those who are using

the services.

DR. BECK: Two other thoughts

along that line. First, FIREBIRD itself is

an open source solution. So there's nothing

that would preclude somebody out there in the

community, in the wider community, in the

non-CRIX community from creating or modifying

or improving FIREBIRD and providing that as

an alternative to this entire enterprise.

Secondly, there's nothing in this enterprise

that would preclude the agreement to host

more than one service provider for a

particular service just like VISA versus

MasterCard at the level below - at the

operating level. So there's no reason why,

especially if we focus on some of the things

we've learned through CABIG and other

initiatives, a bias toward developing

prototypes in open source and then taking

them to the point where they're ready to be

supported or taken out. There's no reason

why there couldn't be more than one solution

at any level.

DR. WOODCOCK: We'll have to - go

ahead, and then we'll have to wrap this up.

MS. MCKENZIE: Just one more

point. I think one of the things that's made

the financial services model so successful is

the adherence to standards. And I think in

this case competition becomes better the more

they adhere to standards. It makes it easier

for them to come and fit into the service so

actually it creates just a nice momentum that

we're looking for in our industry for


DR. WOODCOCK: All right. We

thank the entire CRIX community as

represented here for your presentations. Our

next two speakers are Ed Tripp, Program

Director for eSubmissions at Abbott

Laboratories and Bill Rosen, Executive

(202) 234-4433 WASHINGTON, D.C. 20005-3701
Director, e-Health Policy and Standards,

Pfizer Global R&D.

MR. TRIPP: I feel a little bit

like following a production of A Chorus Line.

Dr. Woodcock, distinguished panelists, ladies

and gentlemen, my colleague Bill Rosen and I

would like to thank you for the opportunity

to present on behalf of the Pharmaceutical

Research and Manufacturers of America on the

subject of electronic submissions and public-

private partnerships. First I will address

electronic submissions. Then I'll turn the

podium over to my colleague Bill and he will

speak and address public-private

partnerships. The member companies of PhRMA

spent nearly $40 billion on pharmaceutical

research last year and with such a massive

investment we have been long been advocates

of data and information automation that can

lead to improvements in healthcare through

better access to information, and that in

itself could lead to better decision-making.

We wanted to make our position

clear. We fully endorse a plan to move to an

all-electronic submissions environment and to

that end we've partnered with FDA to

establish electronic capability for

individual case safety reporting, structured

product labeling, annotated ECGs and other

standard data and exchange formats. Our

reach goes to working with ICH, CDISC and

HL7. We've got broad leadership involvement

with the ECTD working group, the SPL working

group and today with the regulated product

submission, both the leadership and testing

teams. And our organization, the members are

implementers and we want to both identify our

concerns and add our voice as proponents of

the public-private partnership concept.

So as we look at some of our top

goals on electronic submissions, the first

goal deals with developing end-to-end

electronic environments. And when we talk

about end-to-end, we're talking about dealing

(202) 234-4433 WASHINGTON, D.C. 20005-3701
with process and information flow upstream

and downstream of submissions. I want to

emphasize that a common environment

facilitates information sharing and transfer

of information all the way from discovery to

product retirement. So its scope needs to

encompass all transactions throughout the

product lifecycle. As Dr. Edwards pointed

out, we need to be looking at advertising

information as well as submission data. It

needs to be the entire scope of the process.

Goal number two goes to harmonized

processes and data that increase data access

and dissemination, and that leads to better

analysis. Critical to the success, we

believe having a long-term plan with

sufficient detail in the first 18 - 24 months

and establish long-range goals running out

over a rolling five years will help

facilitate the right activities and speed

adoption. The plan or roadmap needs to lay

out the key milestones for each initiative

that will in the end lead to an end-to-end

electronic system.

The most effective standards are

broadly harmonized. We were just talking

about electronic banking. Imagine what

electronic banking would be like if national

boundaries and regional jurisdictions

dictated that they use different standards.

We would never move money out of this country

and yet that's exactly what we're doing in

the healthcare industry.

So, how would we benefit by an

electronic submissions environment? That

question depends on how it's implemented and

it assumes the assumption of effective

standards. And by "effective standards" we

mean that they're harmonized and well

established or stable. And the focus then

can be directed at the science and at

information access and retrieval, and that

information access and retrieval will exceed

our best expectations. We only have to look

(202) 234-4433 WASHINGTON, D.C. 20005-3701
to the Worldwide Web to see how

standardization of data and information can

increase our access and our ability to react.

Harmonized standards promote

consistent processes and procedures, reduce

redundancy. Standardization of clinical and

safety information and its transmission in a

globally harmonized format lays the

foundation for a future vision of a global

electronic health record. And from my

perspective that needs to be the target in

that arena as well, not a U.S. electronic

health record, but a global electronic health

record. I do not want to go to Germany and

find that my health record doesn't do me any

good there.

Now the use of electronic

submissions requires standards, but for

standards to be truly effective they must be

harmonized to achieve interoperability. If

we drive to globally harmonize standards, we

enable the use of the information not only

for marketing reviews, but for many purposes

across the healthcare domain. And working to

develop standards and processes for an end-

to-end electronic environment should be

viewed broader than an FDA objective. All

stakeholders need to be engaged to maximize

the use of the data and the information for

improvement of patient healthcare.

To the question of wouldn't all-

electronic submission environment change your

ability to initiate in a timely manner the

studies supporting your regulatory

submission. Well, from our perspective the

question needs to be re-worded slightly. Can

a sponsor improve the conduct and analysis of

a clinical trial with an all-electronic

submissions environment? And the answer is

undoubtedly yes, but. It needs to be done

correctly. Imposing standards on legacy

processes designed to utilize paper will not

speed up study startup. We need to evaluate

and redesign the process as well as develop

(202) 234-4433 WASHINGTON, D.C. 20005-3701
the standard to achieve a more efficient

overall process. We need to extensively

pilot the standard and the processes and

allow sufficient time for tools to be

developed and to mature. However, the impact

of an improved harmonized electronic standard

that supports more efficient business

processes for data collection, monitoring,

review and analysis will likely offer more

dramatic gains and speed and cost reductions

for both the industry and the agency. If you

think about it, you wouldn't look at an old

process that required triplicate NCR paper -

and we all remember that I think. Most

everybody in the room remembers that - and

automate it by having your printer

automatically spit out three copies. We re-

engineered the process. We shouldn't combine

our efforts to the development of data

standards and automating the paper processes.

We need to leverage the automation by

redefining those processes.

Not having a clearly stated

compelling business case for electronic

submissions or any of the components of

electronic submissions hinders adoption. And

this - we've gone through the discussion of

mandate, no mandate, incentives. The bottom

line is there needs for each component of an

electronic submission system there needs to

be a clearly stated business case. That may

be the gun's at your head, this is a mandate.

It may be that there's an incentive. It may

be here's a rationale and a way to look at

it, and you're going to save hundreds of

thousands of dollars if you adopt this. But

it needs to be clear and stated up front or

we get this fuzzy participation. There's

some companies that want to be on the leading

edge. There's some companies that want to be

on the trailing edge.

The benefits can be realized with

the comprehensive long-range plan where every

one of those initiatives has a compelling

(202) 234-4433 WASHINGTON, D.C. 20005-3701
business case and folks can see where the end

game is and plan where along the path that

they want to join in the process. Standards

churn, or the movement from version to

version, increases cost of implementation and

results in continual reinvestment. And we

have many players who sit on the sideline

waiting for the dust to settle before they

jump on. You only have to look at the

history on SPL to know that's the case. We

still have some folks sitting on the sideline

waiting through their first submission on SPL

because that standard has churned and turned

over a number of times.

A major impediment is a mixed

environment with too many options. Andrew

Tanenbaum, who's a Professor of Computer

Science in Amsterdam and the father of

structured programming, said the nice thing

about standards is that there's so many to

choose from. Well, we don't think that's a

very nice thing when it comes to trying to

move to a new environment and we believe that

the notice to withdraw the ENDA guidance is a

positive step.

Are there enough entities to

support the effort? It really goes back to a

question of churn. Lack of a long-term

roadmap leads to confusion and that combined

with instability of standards delays vendors

from entering the market and causes industry

to take a wait-and-see attitude. So if

there's a lack of testing before introducing

a standard that can erode its value. And

although the time to develop a well-tested

stable standard is longer, the overall cost

to implement ends up being much less. Faster

is not always better and through piloting

standards before implementation we can obtain

stable services and tools and actually get

faster adoption and lower cost.

Additional costs associated with

implementing a particular format of standard.

Best guess is the move from ENDA to ECTD is

(202) 234-4433 WASHINGTON, D.C. 20005-3701
or will be more costly than the move from

ECTD to RPS, but that in part depends on how

good a job we do testing and developing the

RPS standard and the processes around it.

There are also hidden costs dealing with

behavioral change and that happens both at

the agency and the sponsor. We still get

requests from reviewers to provide paper.

That sends kind of a funny message when we're

trying to move our organizations along to an

e-environment. And our scientists are

reluctant to move from a paper environment

because they have to learn to evaluate not

just the content, but the bookmarking of

documents, the hyperlinking of documents.

The analogy is everybody in this room can

pull out their PC and type up a several-page

report and print it out. How many of you can

turn it into a webpage? And that's what we

ask folks to do when we create an electronic

submission. So there are costs associated

with finding the right resources. There is a

shift of skill set. There are costs in

modifying document management systems to

accommodate changes in structure and meta

data every time we shift a standard. And

then there's costs of new publishing tools

because not all - we can't count on the tools

that we have for one standard being developed

and made available in the right timeframe for

a new standard.

Are there parts more costly to

convert? And the answer is yes. I think we

heard this also from some other speakers

earlier today. Any change in electronic

format that ripples downstream into our

internal data collection processes actually

has an enormous cost associated with that

change. There are, at least for mid-sized to

large companies, there's a substantial

investment in data collection systems, and

when the format changes, there's a large

reinvestment to change those systems.

Sometimes we get lucky and it happens to hit

(202) 234-4433 WASHINGTON, D.C. 20005-3701
at a time where we're looking to upgrade

those systems anyway and other times it's

totally unexpected. Once again, a well-laid

out 5-year plan will help us better stage

that kind of investment, but it is possible

for the fully implemented change to be more

efficient than the previous environment. And

that requires all parties to be involved in

the redesign.

How much time? The answer is not

a number. You've got to have the right

processes in place regardless of the time it

takes. We learned a number of lessons

implementing SPL and one of them was that the

lack of scenario-based development and

testing led to frequent changes and

additional complexity. You need to crawl

before you walk and walk before you run and

that means engaging all the stakeholders in

developing the new process, then develop the

standard, do scenario-based testing, pilot

it, be comfortable with the results of the

pilot and finally document the implementation

requirements well, then take it into

production. And if we do that, the uptake

will be faster and a lot less problematic.

Problems arise when we skip these steps to

compress our timeframes or when legacy

methods for exchange aren't withdrawn. And I

want to thank the panel again for the

opportunity to address them, and now I'll

turn the podium over to Bill Rosen.

MR. ROSEN: I too would like to

thank the panel. I'm vertically challenged

here. I too would like to thank the panel

for giving us the opportunity to speak today.

I'd also like to thank the multitude of you

who bothered to get up this morning and get

down and get into line as early as 7 o'clock,

and I hope that the turnout will show the FDA

and the NIH the interest that the topics that

we're talking about today have for the user

community. I'd also like to encourage that

community to please get your comments in by

(202) 234-4433 WASHINGTON, D.C. 20005-3701
the deadline and judging by the number of you

that are out there this morning, I hope that

there will be plenty of them.

Now, last night on the way down

here, I don't have a slide for this one

folks, but on the way down here I happened to

start thinking about the history of e-

submissions and some of the beginnings of the

public-private partnership. Now, the good

news is that I can remember it at all. The

bad news is that I do remember it. Going

back to 1984 Research Data Corporation and

Abbott collaborated to submit the first

CANDA, or Computer-Assisted New Drug

Application to the FDA on a VAC server, if

any of you can remember what a VAC server is,

that was not located on FDA property but

rather was located at RDC. Unknowingly, they

delivered the first electronic submission via

a trusted third party entity. During

the late `80s NDAs were delivered

electronically at the FDA via something

called ONDAs, or optical NDAs that consisted

of scanned documents that were not easily

searchable as you can imagine, but they were

in electronic format. In the early to mid-

`90s CANDAs multiplied proliferously but

without standardization within our cross-

sponsors, the scope, features, hardware,

software, they all varied wildly. Sponsors

paid for one-off development. Agency

reviewers had to learn a new system each time

a new application was submitted. The entire

process was not efficient. There was a sense

of frustration on the part of both sponsors

and the agency due to the lack of standards.

However, there was agreement that electronic

submissions could increase access to

information, speed access to information.

These early CANDAs could not be leveraged,

however, on an international basis.

In 1992, the initial FDA CANDA

guidance manual was published. During 1994 -

1995 the Submission Management and Review

(202) 234-4433 WASHINGTON, D.C. 20005-3701
Tracking or SMART project was established by

the FDA. The goal was to provide standards

and a system for industry to submit text,

data and image files in electronic format.

In the mid-`90s Document had emerged as the

de facto document management standard while

Adobe PDF emerged as the de facto output

standard and SAS transport format became the

de facto format for submitting data

electronically. In 1997 the Aris pilot for

e-safety reporting was initiated and the FDA

also drafted a guidance for e-submissions.

In 1998 ICH, the International Conference on

Harmonization, began to focus on a means to

create and transport an e-submission that met

the definitions of the common technical


In 1999 the e-submissions guidance

for CDER was approved in January and for CBER

in November. And by 2000 a brokerage site

was presented at the DIA annual meeting and

PhRMA began discussions to consider something

referred to as InfoBroker which was intended

to establish a shared environment via a

trusted third party. In 2001 FDA speakers

mentioned the innovative use of third party

repository at the DIA annual meeting and

PhRMA developed a white paper titled Issues

in Developing and Implementing an InfoBroker

for the Biopharmaceutical Industry and

Regulatory Authorities. Also in 2001 of note

over 70 percent of the NDAs included at least

some electronic component.

In the spring of 2001 the ECTD or

electronic common technical document

specification was published, establishing the

format for a globally harmonized e-

submission, quote unquote, or the uncommon

technical document. By 2002 PhRMA began an

effort to build the concept then known as

SEBIX, the Secure Electronic

Biopharmaceutical Information Exchange that

was mentioned earlier by my colleague Deb

Bremer. In 2004 all knowledge harvested as a

(202) 234-4433 WASHINGTON, D.C. 20005-3701
result of SEBIX was transferred to the

National Cancer Institute for further

development of the concept and we thank you.

You will notice during this brief history I

have used the words "brokered site" "Info

Broker" "shared infrastructure" "shared

environment" and "SEBIX"

anonymouslysynonymously for public-private

partnership. Now so that I don't step on my

tongue throughout the remainder of this

presentation I'm going to refer to "public-

private partnership" simply as P3. Thank


In today's standards environment,

it is important the terminology is

standardized as well as data content and data

exchange standards. We recognize that there

are many definitions for the meaning of

"public-private partnership" or P3.

Minimally, PhRMA believes public-private

partnership is defined as a secure

sustainable shared infrastructure that will

provide services across the biopharmaceutical

and healthcare communities. For any public-

private partnership or P3, security and

protection of intellectual property is of

vital importance. As a result, efforts must

be taken to secure the infrastructure and

ongoing testing is necessary to assure the

effectiveness of that security. Independent

audits should be considered for this purpose.

Regarding general viewpoints on a

third party entity or entities providing

services, PhRMA supports the creation of a P3

to provide a shared infrastructure for

information exchange. PhRMA also recognizes

that the FDA has jurisdiction over other

products such as devices, foods and animal

health products. We believe this common

shared infrastructure should also be scalable

to provide services to these other industries

as well, achieving economies of scale. At a

high level the P3 must implement a common

standards-based electronic infrastructure not

(202) 234-4433 WASHINGTON, D.C. 20005-3701
only for regulatory data and document

submissions but also for information review

and analysis.

To minimize any preferential

treatment we prefer that oversight of the P3

infrastructure be a not-for-profit

organization. The not-for-profit

organization should be empowered to establish

contracts with for-profit companies to

provide appropriate infrastructure and

related services. The body that oversees the

P3 would then have the authority to establish

multiple applications running on top of the

infrastructure in a profit or not-for-profit


PhRMA advocates for a pragmatic

approach where P3 has grown organically,

adding new services at a speed that P3 can

manage. Logistically, backup and recovery

are essential as is maintenance for both the

software and hardware. P3 must assure 24x7


Some random thoughts about

benefits. P3 has potential to speed safe

effective medicines to patients. The focus

of pharmaceutical research should be on

health sciences, not on managing and

maintaining technology infrastructure that

supports delivery of electronic information.

P3 in this respect is an enabler. A shared

electronic information management platform

therefore targets redundancies across our

industry by providing a single


We believe the business case for

each new application differs across industry,

health authorities and other potential users.

This has to be noted as new applications and

new ideas come up underneath the common

infrastructure. We recognize, however, that

for the P3 to be effective it must be

affordable for all, allowing for broad

participation. PhRMA envisions the P3 can

level the playing field for small to medium-

(202) 234-4433 WASHINGTON, D.C. 20005-3701
sized companies, giving them access to state-

of-the-art technology at reasonable costs.

This will eliminate barriers to process

innovation and speed adoption of standards.

The spectrum of participation

includes not only large and small pharma

companies, but also CROs, healthcare

providers, as well as others in other

regulated industries. P3 will also

facilitate two-way communication between

sponsors, their business partners and health

authorities. It also facilitates a truly

paperless submission.

It's all about information and the

shared environment can serve as a single

point of reference for participants,

permitting sponsors and health authorities to

see the same information the same way using

common tools. I must re-emphasize the need

for security, security of both the

intellectual property that might reside on a

shared infrastructure and security that

guarantees that patient-related information

is managed appropriately.

Personal health records,

electronic health records and electronic data

captured for clinical research purposes can

be integrated at a high level, providing the

right information at the right time in the

right format in the most efficient way

possible to benefit all parties involved,

providing the standards are there to be used.

Proliferating multiple P3s will not allow for

efficient cross-referencing across

submissions, nor will it allow for efficient

information lifecycle management. The

implementation of standards must be focused

on the information itself rather than an

electronic information exchange platform.

However, interoperability of standards that

enable information exchange once again I must

state is imperative.

Some potential barriers. For P3

to be successful there has to be broad

(202) 234-4433 WASHINGTON, D.C. 20005-3701
participation. I mentioned this once before.

Several other people have mentioned it as

well. Further, we recognize the potential of

a P3 across not only the pharmaceutical

industry but across the broader healthcare

community. If P3 is overly conservative as

it develops, the scope may be restrictive,

resulting in an effort that will have a

limited lifespan. Conversely, if the P3 is

overly aggressive it may spin out of control

with minimal success, also with a limited


We have a need for a long-term

electronic archive. Some products are on the

market a long, long time. I believe this was

mentioned by Dr. Edwards earlier this

morning. If we take for instance the case of

Dilantin which was first approved in 1938 and

it is still being marketed today, imagine if

that was approved today and was still being

marketed 70 - 80 years from now. Are we

going to be able to dig up the electronic

records for that and possibly is this a

service that the P3 can play for us, as a

long-term electronic archive?

Other potential barriers include a

lack of clarity as to the FDA's willingness

or ability to participate in a P3 and this

may delay or discourage full participation.

This has also been mentioned earlier. Others

may simply choose to follow rather than lead,

staying on the sidelines waiting for FDA and

others to engage.

Some conclusions. Efficiencies

cannot come at the expense of increased

complexity and costs on the side of the

pharmaceutical industry. We should all be

looking for win-win scenarios where

information can be presented and preserved in

the most effective and most efficient and

cost-effective manner possible. The agency

shares the responsibility for processes they

impose on the industry they regulate and how

those processes contribute to the overall

(202) 234-4433 WASHINGTON, D.C. 20005-3701
cost of medicine. We should all endeavor for

an optimized process that creates, develops

and implements globally harmonized standards

that increase the likelihood of success for

shared information exchange. For the

pharmaceutical and healthcare industries, we

must establish a long-range plan for managing

data, thereby creating a true information

lifecycle. Long-range, the P3 should be

inclusive of information generated from

discovery through product retirement with

integration of submissions across the drug

development continuum. PhRMA recognizes the

value proposition of P3 and that it varies

from one region to another, but extra efforts

should be made to establish globally

harmonized standards. Standards in the use

of public-private partnerships can speed safe

and effective new therapies to patients


Once again I would like to express

my gratitude to the NIH and the FDA for

allowing us to have time today to express our

opinions. I would also look forward to

working with the FDA and the NIH and the

research community on electronic submission

standards and any P3 effort. Thanks.

DR. WOODCOCK: Thank you very

much. We'll have questions now from the

panel. Dr. Mittleman.

DR. MITTLEMAN: I was wondering

what PhRMA's view on the presentation that

was made by CRIX and particularly the CRIX

International kind of model, how does PhRMA

react to that?

MR. ROSEN: I may not be the right

one to answer the question. I've been

involved in this concept since 2001. I was a

member of the group that worked on SEBIX and

I was one of the people who went over to the

NIH to do a data dump back in 2004. PhRMA

supports the concept of CRIX although I would

also have to be negligent if I didn't say

that that is not universal across the 40-some

(202) 234-4433 WASHINGTON, D.C. 20005-3701
odd companies that are members at PhRMA.

Anything to add, Ed?


DR. WOODCOCK: Dr. Levin?

DR. LEVIN: Ed, so it sounds like

there is a business case for electronic


MR. TRIPP: I believe there is. I

believe there is. I don't believe we've

clearly stated it or sold it well.

DR. LEVIN: But inside your

company there's a - you have a business case


MR. TRIPP: Inside our company

we're reacting to the reactions of the FDA

and say if we don't do something, we're going

to have a problem. So it's more of a gun to

the head than a clearly stated business case.

DR. LEVIN: So in your company

that you're not developing your own business

case as far as this is going to be a benefit

for us internally to our company and waiting

MR. TRIPP: No. In most cases it

is reactionary because the dollars need to

compete with other initiatives across the

company. And there isn't a clearly stated

and obvious business case that you can pull

forward. And so it tends to be sold -

investment tends to be sold as a matter of is

it a regulatory compliance issue. And if you

look at - we ran some numbers across the top

20 pharmas. If you look across the top 20

pharmas, over about the last eight years

they've invested a half a billion dollars in

building submissions systems and document

management systems that support that and

continue to invest. It's been a moving

target and it's getting harder and harder to

go make the ask.

DR. LEVIN: So that, so different

companies will be at different stages as far

as their business cases because some have

made a business case already that they want

(202) 234-4433 WASHINGTON, D.C. 20005-3701
to go into the electronic without the FDA

mandating something.

MR. ROSEN: I think that's

correct. Indeed, if we take a look at how

processes are changed backward of the

submission process there are clearly cases

where document management for quite a long

time has been employed and the whole concept

of submission-ready now being carried forward

to the concept of archive-ready and the

business case that enables the management of

electronic information and also for knowledge

management as well.

DR. LEVIN: So it's - and this

goes back to questions that were asked

previously about what can the FDA do to move

forward. And it sounds like for some it's

the mandate will be the piece, but why don't

you address that?

MR. TRIPP: I think if we work

together on a collective long-range plan, if

there's a target out there of when we

collectively believe we should make the move

on any of the initiatives, then I think we

ought to also have candid discussions in part

around what is the business case and what's

the business case for Pfizer is not

necessarily the business case for Abbott

Laboratory. Everybody runs their business a

little differently. But you can put together

templates that state this is how it's being

used. Plug your own numbers in or plug your

own processes in and I think we've failed to

do that effectively in the past. And I think

in order to make things move faster it will

be helpful to do that.

DR. LEVIN: And when you say "we"


MR. TRIPP: We collectively.

There's a group of us standing up here that

have been involved in this for awhile. Help

drive standards, help, you know, drive

electronic submissions and so forth and are

out in the forefront beating the drum, but we

(202) 234-4433 WASHINGTON, D.C. 20005-3701
need a little banner to go along with the

drum, and that's the business case.

DR. LEVIN: And I just, and I

question about when you're talking about

staging the investment. So let's say you

decide there's a business case for whatever

reason. What's that process like and how

long does that take to move your investment


MR. TRIPP: Well, and again it's

probably different for every company, but for

us if we make a change to - if we decide to

make a change to our document management

system for example, not changing it but

modifying it, assuming it's not a real small

patch or fixed to a bug, but it's reacting to

a change in the standard, we start planning

for the investment in May of the year before

we start making it. So the May before the

January we want to start making the advance

we start planning, building up the numbers

we'd start in January, and generally by the

time you implement it and validate it, you're

talking September or later.

MR. ROSEN: Just one comment.

Randy, I think the timeframe differs

drastically with the size of organizations.

Smaller companies are apt to be able to move

more quickly, again possibly because they

don't have established systems and standards

internally whereas larger entities are apt to

have you know a greater problem moving and

being flexible to the change. I'd also have

to say that I think there's a difference

between making modifications to existing

systems and existing processes versus doing

business process change, or the re-

engineering of a process and absolutely

turning over, let's say for instance a

complete document management system. And

that would go for any size entity.

DR. WOODCOCK: Dr. Buetow?

DR. BUETOW: A slightly more

technical question. You spoke relatively

(202) 234-4433 WASHINGTON, D.C. 20005-3701
forcefully against the concept of multiple

3Ps. I'm just curious in today's technical

universe of services-oriented architecture

and under the presumption of codified global

standards, can you further articulate why the

disadvantage of multiple partnerships as

opposed to a single centralized entity?

MR. ROSEN: Please don't

misinterpret what I said. I'm not against

multiple P3s. I think that there is reason

for multiple P3s. However, if you stop and

look at the examples that I've presented

about being able to link across a submission

lifecycle, there's obviously a problem if

that is located on different physical

entities. So I was merely pointing out some

of the drawbacks that may occur if this is

not planned and rolled out in an efficient


MR. TRIPP: One other thing to add

to that. As you look across a P3 concept and

where some of the savings are, some of the

savings and attractiveness if you have a

single disaster recovery plan, you have a

single backup and so that maintenance cost

gets multiplied depending on how many P3s.

DR. BUETOW: That may be in part

answer to sort of a follow-up question.

You've mentioned this and I just want

clarification of how tightly coupled the

service provider should be, or the end

service provider should be to the - from your

all's perspective to the 3P. Is the 3P -

where do you see the relationship in the

service providers? Are they loosely coupled?

Are they tightly coupled? Does the 3P drive

and determine who all the service providers


MR. ROSEN: I would think that it

is necessitated by the board of directors of

say, for instance, CRIX in this instance, and

the community of stakeholders at CRIX to

ascertain what applications would be running

underneath the platform. I would also say

(202) 234-4433 WASHINGTON, D.C. 20005-3701
that the quality of service, the timeliness

of service would be mandated by any contract

that would be formed between that entity to

put applications up underneath that

infrastructure. So I would hope for a

tighter relationship rather than a looser


DR. WOODCOCK: One other question.

Go ahead.

DR. BUETOW: If I could just - and

we absolutely appreciate this concept of

concern of churn of standards. So how do you

all - do you all have a proposed path that we

actually standardize the standards? I mean,

this comes back to our incentivized versus --

MR. TRIPP: Well, I think we're

collectively learning together over time. I

think the HL7 draft standard for test use

model that's been adopted by RCRIM is a good

model. We need to have the patients patience

to let it play through and we also need to

make sure that get the stakeholders engaged.

And in some cases the stakeholders have other

things that are preventing them from getting

immediately engaged. I know we talked with

our colleagues in Europe about engaging in

some testing on RPS late third quarter, early

fourth quarter this year, and we got the

answer that they can't participate till first

quarter because of other things going on. We

can't throw up our hands and say, well, we

can't wait for you. We have to wait and let

the process mature if we're going to get a

valid standard out.

MR. ROSEN: One other comment.

I'd state that the CDISC ODM model and the

BRIDG model are helping I think to bring

about harmonization of standards across the

various communities.

MR. TRIPP: Yes, absolutely.

DR. WOODCOCK: Dr. Levin.

DR. LEVIN: Yes, just two things.

You just brought up a point about the

testing, but there's a business case for

(202) 234-4433 WASHINGTON, D.C. 20005-3701
having resources for that too and so some

people it's going to be the same issues?

MR. TRIPP: Yes. I think there

needs to be a well-established test plan for

those so that we can go to those people who

are stakeholders and say, hey, we need a

stakeholder who is willing to test storyboard

one, two and three. Because they're related,

it's a short, manageable piece of work.

Who's going to step up and do that? And then

four, eight and 15 are related, we need

somebody else to pick up that. And I think

we're just collectively learning that those

are the kinds of details we need in this.

This is a relatively new process as you're

aware of.

DR. LEVIN: But the same issues as

far as justifying - going back to your

management and justifying that resource is

going to be the same issues.

MR. TRIPP: But I can measure what

that ask is as opposed to I just need an open

head count to work on this for ad infinitum.

DR. LEVIN: My question was back

to the CRIX or public-private partnership.

MR. ROSEN: "P3" works, Randy.

DR. LEVIN: Okay. Does it have to

- why is it - why did you say not-for-profit?

What about a for-profit organization running


MR. ROSEN: A for-profit

organization running the infrastructure, I

think that a number of companies would be

reluctant for fear that monopolistic

practices could evolve if there was a total

lack of competition in a for-profit way.

Then certainly the cost of services could be

driven up instead of driven down.

Additionally I'd mention that the whole idea

of the not-for-profit entity itself is indeed

to channel any profit that's generated back

into the process and thereby reducing the

cost not only for all of the founding

members, but also for the associate members.

(202) 234-4433 WASHINGTON, D.C. 20005-3701
DR. LEVIN: Now, the banking

industry with the VISA, that's a for-profit



DR. LEVIN: Their - VISA is a not-

for-profit organization?

MR. TRIPP: Yes, it is.

DR. LEVIN: How about if you say a

public-private partnership can be other than


MR. ROSEN: I guess it could be.

It would not be my desire given all the

effort of all the many stakeholders who've

put numerous hours into that effort over the


DR. WOODCOCK: All right. We're

going to have to conclude this presentation.

Thank you very much for your comments.

MR. TRIPP: Thank you.

DR. WOODCOCK: All right. Our

final presenters in this segment of the

proceedings are John Rapoza, CEO of JRRapoza

Associates, Inc., and Ari Kaliannan, Chief

Executive Member of Newtech Global Solutions,


DR. RAPOZA: Audience, I don't

have any specific presentation. I'm just

going to make a couple of comments. I'm in

the business of basically dealing with

generic drug companies and in the process of

getting them approved applications. And in

the clients that I deal with there has not

been a lot of interest in electronic

applications. Primarily the cost, although

there are some. The middle-sized companies

have expressed interest and they do have

applications they're putting together. The

primary reason I think is the motivation to

do this given the fact that the generic drug

office is so far behind in these application

reviews that they have difficulty seeing the

advantages of making a change to a different

application format. However, I think the

OGD, the Office of Generic Drugs, is kind of

(202) 234-4433 WASHINGTON, D.C. 20005-3701
making progress in that direction. I think

with the SPL, implementing and mandating the

SPL as part of the application was a very

good idea and that's how I got involved with

Ari here and he's going to talk a little bit

about his document management system.

And the other thing I think

they're doing which is most helpful is the

electronic review which essentially takes the

key elements of the application based on the

CTD format and allows a very quick review

application of the application for what the

agency would consider to be adequate

information. So I think they're going in the

right direction. I think it's going to be a

long time before you'll see, you know,

applications being submitted by small generic

companies because many of the drugs that

they're chasing just don't have the financial

margin to do that. Having said that, Ari is

going to give a little presentation about his

system which I became aware with the SPL

which is really a lickety-split system for

dealing with transfer of documents. And he's

going to talk about his systems as well.

Thank you.

MR. KALIANNAN: I hope I have

learned to use the computers. I take this

opportunity to thank the agency for giving

this opportunity for the presentation. My

experience in the license industry is two

years. I have been tasked to automate the

small generic industry with $16,000. It has

not come yet. U.S. dollars. It's not Indian

rupees. But I ask the question like what

software do you have. We have only one

commercial graphics systems, nothing else.

It's a good place to start with. So we ask

the questions to the users what you are

familiar with. Only two that are familiar

with this Office, Word and Excel. We started

from there. What it takes to make use of

these systems more useful to the end users.

Since we do not have much expertise or

(202) 234-4433 WASHINGTON, D.C. 20005-3701
collaboration with other agencies, we had to

talk to three vendors. We planned on being

Microsoft Office, talking to them, how you

can make this Excel as user-friendly for the

regulatory environment, how we can make the

Office product more useful. They said you

wait for us, Office 2007 will be the answer.

We are not going to wait for Office 2007. We

need something to be done for this year. So

we started exploring that way with standards

and that's why we took the SPL as a reference

point to start our work.

Having said that, I think that the

last two presentations speaks very well about

the industry's needs and the standards and of

the non-profit organizations. I just want to

go through this here. Basically the business

challenges are editing among the business

partners and agencies, especially when your

really small companies who do not have

dedicated resources and not only one

discipline, who has to wear multiple hats.

That's a challenge. Dedicating, locating

files across databases and applications.

It's good that my company did not have one

database system to work with. I had to start

with file servers. Difficulties enforcing

common vent, check-in/check-out processes

have to happen because there's none. We

ought to establish one. Lack of integration

and familiar cost-effective document

management systems. Any vendor you're going

to ask for it. They are not going to sell it

for $16,000. We already cleared one. That

and - that's a challenge.

IT. Even though I was given the

title of IT Director, I preferred work in the

field first two years first. Without that I

said I will not take over the responsibility.

Having spent two years on the field, I take

over the responsibility to move forward.

Seeing a couple of applications awhile in the

market. What we can do with what we have.

That sort of approach we took. We used the

(202) 234-4433 WASHINGTON, D.C. 20005-3701
Microsoft Office Word package to create the

SPL example data and we talked to our friends

in Microsoft even though they are not the

good one for security and standards, but

still you have to go with their blessings for

some work. So they had a solution called

BTSTAC sort of which basically gives a tool

to integrate the XML files. So we tried to

leverage the application, tried to integrate

whatever the possible automation we could do

within our capacity. Also we talked about

the enterprise systems, links, commercial

systems. Even though the mission was there,

there was not enough funding. So we started

out able to - XML for the form and work for

the management system using InfoPath tools

which is also available with Office, which is

very affordable. The question is who is

going to build the path, who is going to

validate it, how is it going to happen. So

these are the questions we are to answer for

the users first.

We took the initial step of doing

one thing first. That's what we did with the

SPL. And we are currently working on the

CDT. We are seeing wherever there is an XML

there is a place to go and look for. And

that is how we start the next one. Also we

are working with Microsoft and Alliance

program called BioAT trying to put the

puzzles together with the various vendors and

see how we can play a role in this

environment. That should be our next


Anyway, nothing is more important

than compliance security, especially

environment like this. Having come to this

country without knowing what is 911, I spent

the last eight years dispatching, creating

software for at least 200 agencies in this

country. With no standards you know how

difficult it is to make the systems work

together. Finally comes to SPL and escape

the global XML example. Spent two years in

(202) 234-4433 WASHINGTON, D.C. 20005-3701
understanding it. I saw that if justice can

be done with XML, why we cannot do the

pharmaceutical solutions and that's where we

are trying to look at it.

I just went through the review and

put up a couple of action items for our own

site, how we are going to deliver this.

Again, since this is a regulatory

environment, all business processes are to be

initiated and entered in the validated state

and systems. We already initiated with a

change control process. We leverage XML

data, use the built-in workflow mechanism

within the Windows 2003 environment and the

upcoming Windows Vista. I don't want to sell

Microsoft products, but somehow the features

are available which allow the vendors and

give the opportunity to derive some solutions

with the tool sets. And that is how we are

going forward developing this ECTD framework.

What we do, we do validation by integrating

various data sources, including any business

applications, including ERP systems, links,

common data systems, document management

systems, laboratories. We clear the portal.

We take the repositories in an XML format,

integrate them into portal environment so

there it's easy for remote access.

This is the philosophy behind our

two products today, SPL and ECDT, which makes

use of the Microsoft Office so it's less

training for the end users and it's easy to

use. And we take advantage of these

facilities portal. We'll do with Windows

Vista work for management features which

allows us to create a cost-effective

solution. We are engineers. We applied an

end-to-end system and product management

services from design, development, testing

and validating. This would be my brief

introduction. With this I leave the podium

for questions.

DR. WOODCOCK: Thank you very

much. Are there questions from the panel?

(202) 234-4433 WASHINGTON, D.C. 20005-3701
Any questions? All right. Thank you. Okay,

what we're going to do is we're going to take

realistically a 15-minute break. Please come

back here. We will reconvene and begin

promptly at 10 minutes after 3:00. Thank you

very much.

(Whereupon, the foregoing matter

went off the record at 2:52 p.m. and went

back on the record at 3:10 p.m.)

DR. WOODCOCK: We are going to

reconvene and get started. Our next speaker

is Mark Rutkiewicz, Director, Quality

Assurance, AGA Medical.

MR. RUTKIEWICZ: Good afternoon.

I feel a little like a PETA member at an NRA

meeting, or actually a cop at a FBI sting.

I'm with the medical device industry, one of

the companies in the Twin Cities. How many

other people here are associated with the

medical device industry? Oh, I've got a

couple of others. Yay. I'm with AGA

Medical. We make devices for making

occlusions in the heart for congenital heart

defects, but I'm going to talk today about

best practice of sharing electronic documents

in the medical device industry. I've been in

the industry about 20 years doing pacemakers,

defibrillators, external defibrillators,

implantable hearing devices at a few

different companies and one of my jobs has

been to engineer information so we can make

things faster and make change better.

I'm going to overview quickly the

rules and the history. I really want to talk

about a proposed process we've been using in

the medical device industry for managing

product information which is just as complex

as what we deal with on a submission. And

actually we are using a model that was

actually developed by the high-tech industry.

The laptops you see today cost - we can buy

them for $500 from Dell. Ten years ago they

were $5,000. So the high-tech industry has

taken a lot of techniques and configuration

(202) 234-4433 WASHINGTON, D.C. 20005-3701
management, information control and applied

it to reduce costs. One thing they've also

done is instead of building it themselves

they outsource it so I'm going to talk about

a business-to-business model that's been used

by the high-tech industry. It also can be

used by the medical device industry and used

for submissions to the FDA, but worldwide.

Quickly on the device side, I

don't know if everyone knows. The types of

submissions that we do are we do PMAs, IDEs,

HDEs, 510k's and additional supplements

associated with that. For Europe, medical

device companies do either design dossiers or

technical files and there's a lot of overlap

between those submissions. In addition,

Canada has licenses. Their submission

requirements are lower. We don't have to

submit as many documents for those types of

areas. And then Japan and Australia, totally

different animals. We're not going to get

into that.

But the rules for creating an

electronic submission for the medical device

industry has evolved over time with different

authorities and different needs. When Europe

consolidated with MDD and the AIMD

requirements, they standardized this

technical file and design dossier which then

simplified, at least for medical device

manufacturers, not having to submit to every

country in Europe. It's always been done as

books or volumes and PDFs or electronic.

We've done some versions of the PDF that the

pharma's been doing, and each company also

has tracking methods on databases, but they

were just typically tracking specific parts

of a submission, you know, dates, what kind

of submission it was, who it was to, but not

the entire submission that's tracked

electronically by most companies. A couple

of years ago I ended up doing a pure

electronic submission to the FDA. I had a

paper, but it was really done electronically

(202) 234-4433 WASHINGTON, D.C. 20005-3701
with the ENT group for an implantable hearing


But historically why the

submissions are set up the way they are is

because every department owns their

information. There's a DMR file, there's

clinical files. We've seen a lot of

technical clinical information. The clinical

side isn't as large for a medical device as

the pharma side, but there's DHR files,

complaint files, engineering report files.

They're all managed independently. And what

happens when we create a submission of

medical device, you go into each of the areas

and you put it together in a binder and you

get the submission out. Well, these things

in these other areas can change and they'd be

- it becomes very complex when you get into

medical device because of the different

process interactions and what's

interchangeable and not interchangeable, what

needs to be submitted and not submitted.

So the history has been we create

each submission book and for each country too

independently of each other, even though

there's a lot of overlap. And then databases

or manual paper methods were used to track

these. And one thing that happens with this,

if we submit - a medical device manufacturer

submits something to the FDA, you can't

cross-link documents that have been approved

already. In a medical device company like

ours, we have about five or six different

products, either IDE or PMA, but basically

the same material, but each time we submit

the biocompatibility report and each time the

biocompatibility report gets reviewed by

somebody over and over again. There's no

reuse which makes the effort by the FDA a lot

more difficult.

So what I'm going to talk about is

some strategies that have been used by the

industry, the high-tech industry and the

medical device industry. Best practices.

(202) 234-4433 WASHINGTON, D.C. 20005-3701
They've talked about - some of the other

speakers have talked about configuration

management. There is a process called

Configuration Management 2. It's an

institute that's a national one that actually

trains people all over the world. There's

tens of thousands of people that have been

trained in this. Aircraft industry is really

huge in this and military too. It also talks

about flexible end configuration management

too. You talk about flexible information

structures, products and processes. And they

haven't gotten to submissions, but it's

designed to be flexible.

I talked about information

mapping. I think that's been part of the

process to create the common technical

document. You know, defining who your

audience is and organizing the information

with XML. I'm going to go over a quick

little information map where manufacturers

control their information and talk about open

standards. And I think everyone has talked

about this. XML I think is the way to go

with this. I started using XML about eight

years ago for some information management

processes and it's still going great today.

This is a quick map-out of the CM2

process. They're an institute based out of

Phoenix. They're associated with ASU and the

University of Tennessee and they do training

on this. And some of the basic concepts here

is that you start with a master document

which you're going to submit and then it goes

through a change request, a change order and

a change implementation process. That can

happen internally or it can happen at a body

that's going to be accepting the information.

And it's called engineering change request

because they're talking product structure,

but the same structure can apply to the

submission because a submission is made up of

chunks of information. A change admin 1 is a

roll-in process that talks about do you want

(202) 234-4433 WASHINGTON, D.C. 20005-3701
to even accept this thing to begin with. And

so sort of the first step in FDA, is a

submission submittable. Second step, the

change order is done. Does each section get

approved? And third is a change in admin

where you actually implement and that's a

feedback process. This process works

internally at a manufacturer but can also

work at a regulatory body.

So it's a 3-step process. Part of

the thing with the parts of the submission,

each chunk of the submission gets unique

identifiers so you can then allow for

reusability. Part of CM2 also is status and

revision identifiers for all objects of the

submission. This allows you to then reuse

linking existing objects. You know what rev

this part number's at or this document's

number at. Understanding interchangeability

so when you make updates to these

submissions. We currently have - one of our

PMA supplements out there has 440-some

supplements because we also use it for

emergency compassionate use concepts. So

we're doing these variety of kinds of

submissions with our submission, but it's all

tied to a base PMA so to be able to re-link

that information together becomes complex

when you're talking that many supplements.

One thing too in the medical

device industry versus pharma, and we're

talking pharma having you know drugs out

there for 40, 50 years. In a medical device

company a lot of - in the pacemaker and

implantable defibrillator industry the

products were being revised every 18 months

and had to get re-approved. So the speed and

the way of making that change, these

techniques we started using at some of the

bigger companies. Then I started

implementing these concepts to smaller

companies and they seem to work really

effectively. As going out from CM2 there's a

basic concept on information mapping. I'm

(202) 234-4433 WASHINGTON, D.C. 20005-3701
not going to go into details here, but it's

basically organizing and understanding your

information types, which is a big key concept

in XML, understanding the type of information

that is in the document.

Regarding here the next slide,

what's in a manufacturer? In a medical

device manufacturer there's information

inside the company that we have control of

and there's information that's outside the

company. And part of the submission is

typically combining these things together

into one object that combines device master

records, what it takes to build a product,

design history files, the reports, some

regulatory documents that lead in and talk

about the submission, clinical documents that

are part of the clinical reporting. You

might tie in articles and papers to support

the submission. So we want to be able to tie

that all together into a common format.

So an example here is a BOM

structure. You build material. It's

creating a structure that's different than a

standard folder. In a standard I saw

earlier, some of the documents, you create a

folder and create folders within folders. In

this one I could link the same object across

multiple sections and not have to recreate

it. In a submission there's top level object

and then there's lower for each section and

you can tie different kinds of documents to

the different sections. And then you could

route, as we'll get into a little bit later,

you can route each subsection independently

for approval either internally at the

manufacturer or externally.

This basic process is what's been

done in the high-tech industry. There's

about 20 key circuit board manufacturers in

the world that make most of the circuit

boards and most of the electronics that you

have around here, your phones and stuff. But

there are hundreds and hundreds of design

(202) 234-4433 WASHINGTON, D.C. 20005-3701
people because they end up outsourcing the

manufacturing. Those key 20 manufacturers

use one software application for managing the

information that they have to create the

circuit boards, and all the design houses

then follow that standard. And then the

design houses can move from manufacturer to

manufacturer and they can get their

information consistent to any manufacturing

house. So it's a manufacturer-to-

manufacturer-services standard.

So the standard we're going to

talk about is an XML standard for managing

the structure of this information. One thing

we talk about XML is paper to web. It's also

machine readable and also it's been done a

lot for the documents. That makes a lot of

sense to take the document XML, but combine

it with the structure and change management

associated with it.

So the needs I'm going to be

talking about in this process, what the high-

tech industry has done and what medical

device companies are doing is using a PLM

system, a product lifecycle management. We

talk about lifecycle of a submission. It's

not just a document control tool. There is -

in a document control tool there's no way of

creating a building material. A BOM allows

you to link objects together that have no

relationship to each other at all. You'd

have to create separate meta data fields for

all the objects to keep creating these

different links. The BOM allows you to be

independently created. The submission then

has a - would have a lifecycle by itself too.

You know, you could create whatever you

needed to. These tools that are out there

are very flexible. I can call it whatever I

want, but pending on a review hold. Design

the information architecture of a submission

with all the document types in a submission

record. So there's clinical reports and

records and then there's design reports. You

(202) 234-4433 WASHINGTON, D.C. 20005-3701
might have - we submit things on

manufacturing process flow information.

We'll submit articles and papers. And if you

define a standard architecture for all that

information that goes into any kind of

submission to the FDA, then we can build

submissions that the FDA then can read. With

these, these tools allow you to attach any

file type to any of these objects. You can

attach PDF, XML, Microsoft. You can attach

files that nobody can read because a lot of

these tools have neutral file viewers that

can read 500 file types. We typically put in

in the high-tech industry there's the ability

to read a schematic and that's in a unique

file type that typically nobody can read.

But in a tool like this, a neutral file

viewer allows you to read it without having

to buy a $500 tool or a $1,000 tool to view


Design. The design process has

multiple submission types. Like I said, for

medical device industry there's half a dozen

different kinds of submissions but then with

biologics and drug there's different other

styles. But if they can all be built with

the same basic architecture. And the

electronic submission needs to be designed

for approval, electronic approval and


So part of this too, like I said,

some of the information that goes into a

submission, initial regulatory summaries to

highlight the overview of what's being

contained, supplements, the DMR for the

medical device master record, all the specs,

drawings, labels, IFUs that are part of the

submission, the DHF, all the reports,

engineering reports. In the medical device

companies there's a lot of product that's

software. There's software in the products

or there's software - the medical device is

software. Software instantly doubles or

triples the size of any report because of the

(202) 234-4433 WASHINGTON, D.C. 20005-3701
amount of documentation associated with

validating software that's used in a device.

Along with the clinical documents, annual

reports, postmarket information.

So the standard that the high-tech

industry is using today is called PDX,

Product Data Exchange standard. It's been

standardized with the IPC. It's the

International - IPC is the standards

organization for printed circuit boards

because they were using it. Multi-part

standard in how to create structures. It

focuses on the content between the

manufacturer and the electronics manufacturer

service providers. So tying those together

and that's the standard that a lot of work

with Intel, Dell, a lot of these companies

have used this standard for their

information. XML provides a standards way of

providing simple structure. And there's a

standard user group that - I looked this up

online. I'm not associated with this group

directly, but Barb Goldstein at NIST is

running this group.

Next thing I want to show you is

just an example of in my organization what

one would look like. We're in the process of

implementing these tools that are used by the

high-tech industry. What you see here is

this is a submission record. This could be -

I gave this a generic number. It could be

the number the FDA assigns when the PMA gets

it. That's what I've done in the past is

generically create this and then once we get

a number we change the number. Subsection

and then attach structure to it. These are

the documents, the reference documents for

that subsection. Here's another section with

different documents. And this just shows the

BOM explode. One thing by creating it once,

what happens is we ended up taking it, copy,

save as, edit it for Europe. So then we give

a copy of this then to our notified body in

Europe and they'd review our submission the

(202) 234-4433 WASHINGTON, D.C. 20005-3701
same way with slight changes. You get a

different number, but basically the same

content. With this kind of structure too you

can click on any of these records and look

back and see where it's used. You can see

how many different submissions one document's

been used for.

What happens then with this is the

BOM is created internally and released

through a change order process. Then it can

be exported out once it's approved. I have

create a PDX file. And down here is what a -

this is just a file object that's sitting on

my desktop when I created this XML. You send

the PDX to the approve body. You can send it

FTP, through the web, send out a CD,

whichever method. But basically it's an

object called PDX. There's three PDX viewers

out there in the industry, it's like Adobe

viewers, that allows you to - this one shows

the exact same structure with a neutral file

viewer. Same structures, and the files are

all attached so you can click and see them.

So this is neutral. It's not even in any

system now. It's available for everybody to

use, anybody to use.

Then the process would - we didn't

end up doing this with the ENT group. We

ended up sending them the CD with the PDX

viewer and they were able to use the paper

copy for routing, but then for reviewing they

were able to take the CD with them and look

at it whenever they needed to and just carry

it on a CD versus the whole structure. But

the process that normal manufacturers in the

high-tech service industry would use is they

would take the submission from the design

house, route, import it into their system

with the structure, break up - that's you

know approving this initially is a change

implementation as I talked about as far as

CM2. Next, break each section up into

different sections for routing for approval.

So different groups would get different parts

(202) 234-4433 WASHINGTON, D.C. 20005-3701
of the submission to approve. Then it would

have to come back together internally and be

put on another change order to release the

entire submission to an approved status. And

then it allows you to link these record types

together across multiple submissions. So if

you received a submission with a report

that's been approved previously you'd see

that when it gets imported in. And this is

what all the contract board manufacturers in

the world are doing this structure today.

So we were using a PLM tool in the

industry, product lifecycle management, not

just document control tool. The majority of

medical device manufacturers use PLM tools

because we have to have control of our

documents. All these tools, there's the

three that the industry uses is Agile, Matrix

1 and PTC from Windchill. Windchill from

PTC. A lot of people do this kind of

structure. They're all 21 CFR 11 compliant

for audit trails and electronic signatures.

Agile has a focus, the one we're using. A

lot of medical device manufacturers use the

Agile tool. There's other ones out there

too. It becomes a fairly simple tool to

track and implement. A lot of these

manufacturers can implement these things

fairly quickly. There's a case study that

Dell ended up implementing their Agile tool

in about six months. And they were 100,000

parts in 30 sites, so. A lot of these tools

are becoming easier and easier to use.

They've been around for about 10 years.

So recommendation here as from the

medical device side. Design a flexible and

proven submission process that can take into

account all the variations in all kinds of

submissions. Use industry standards and best

practices, I'm talking CM2, using some PDX

standards or XML standards. Use off-the-

shelf tools. Don't try and create something

custom. A lot of the stuff's out there now

that's available that's a lot easier to use

(202) 234-4433 WASHINGTON, D.C. 20005-3701
than even 10 years ago, so. This is my

background. So any questions?

DR. WOODCOCK: Thank you very

much. Questions from the panel? And I would

like to mention, Randy Lutter had to leave

and Clark Nardinelli who's Director of the

Econ staff at FDA is now sitting on the

panel. Any questions from anyone?

DR. LEVIN: Can you use your

system to submit something in the regulated

products submission standard?

MR. RUTKIEWICZ: What's the

specifics? You mean is there a specific


DR. LEVIN: Well, there is a -

we're working on a standard in the Health

Level 7. It's an XML standard called the

regulated products submission standard.

Maybe if you stick around for maybe a couple

of speakers will discuss that.

MR. RUTKIEWICZ: Sure. Yes, with

these kind of tools, they're designed - as

you create different classes of objects you

can create as many different - they're

designed to be very flexible. You can create

as many different data fields that can map to

existing fields, and then the question is how

do you create the standard, the output, the

XML and tag it appropriately so it matches

the standard.

DR. LEVIN: Right, because here

we're working on a - using your

recommendations, it's just not the same


MR. RUTKIEWICZ: But if it's using

XML, then you can easily translate one XML

standard to another because as long as you

define the data types and the PDX in this and

they map, you can easily create translations

that can map to the common.

DR. LEVIN: Okay, I think we'd

like to speak to you after the meeting.

DR. WOODCOCK: Other questions?

All right, thank you very much. I have

(202) 234-4433 WASHINGTON, D.C. 20005-3701
another announcement. There was a ring left

in the ladies room. If it belongs to anyone

please just come up to me and describe it and

you can get it back. Our next speaker is

Jason Rock who is CIO of Global Submit, Inc.

MR. ROCK: I can tell you with a

high degree of certainty that ring does not

belong to me. I'd like to thank the panel

for allowing us to speak and my name is Jason

Rock. As Dr. Woodcock said, I am the CIO of

Global Submit. I am also the project lead

for the HL7 regulated product submission

message. And being the project lead for the

HL7 regulated product submission messages

allowed me access to a lot of folks in the

industry as well within the agency. I have

worked with all the centers so far. We have

evaluated a lot of their systems compared to

the RPS message. We started the testing

process. We have also done this with several

folks in the industry as well.

And today we heard a lot of good

presentations on different topics. And what

we're going to try to focus today on in my

presentation is cost. And we heard a lot of

things with cost so I'm going to try to not

highlight the things that we've already

heard, try to go over some of the things that

we haven't heard yet. And I'm going to

change my presentation slightly around to

focus more on market interactions and market

forces, particularly around competition. So

if my presentation seems slightly disjointed,

I changed things up slightly because a lot of

the content that we've talked about has

already been presented. I want to give,

based on the feedback that I've gotten from

several review folks that's worked on

regulated product submissions, several

industry people that I talked to, I want to

provide new feedback to you that you haven't

heard today.

So one of the big issues that we

see is on implementing a pure electronic

(202) 234-4433 WASHINGTON, D.C. 20005-3701
standard. As you heard today, there's a lack

of resource out there. That means there's

not enough people to do the job that we

currently have. What I'm going to try to

provide is some solutions to that as well.

Then we're going to talk about, again on the

same flavor, on how to save costs within the

industry. We're going to talk about the

multiple guidances that are out there from

the FDA and around the world. We're going to

touch a little bit on the disparate

infrastructure, people having the same

systems, doing the same jobs. I'm sure the

FDA knows that they have the same systems

doing the same jobs. Abbott has the same

systems that does the same jobs. You know,

everyone else has the same systems to do the

same jobs. We're going to talk about how we

can consolidate those and we're going to talk

about getting input from the industry early

on in the process, from the broader industry,

not just people that, you know, work at ICH,

not just the people that work at the FDA and

how that's really going to help us evaluate

our systems earlier in the process. And the

whole focus of today's - of my presentation

is really going to be talking about costs.

And then I'll probably pepper in a couple of

things. I have an idea for a business case

of how we can sell this to management and so

there will be some other stuff in there as


The first roadblock that we need

to overcome is the lack of resources that we

have in the industry. And you know as most

of you know there's probably not enough hands

to do the current job that we have. And the

reason why we don't have enough hands is

because we have an industry full of

specialists, and that is a lot to do because

of the FDA process. People that submit to

devices group that creates PMAs, they're not

necessarily trained to do drug submissions so

we have specialists. Anytime you have a

(202) 234-4433 WASHINGTON, D.C. 20005-3701
specialist you have a high barrier to entry.

The high barrier to entry increases costs.

So we need to figure out a way that we can

share resources, lower the barrier of entry.

That will allow us to have more resources

coming in because we'll have a larger market

thus decreasing our cost and sharing


The next issue that we have, and

this is going along the same lines of sharing

resources and specialization. A lot of that

is multiple guidances. Multiple guidance for

the same business processes is very costly,

costly to the industry. When you have to

submit to CBER a PDF label and to CDER an SPL

label, that's not only cost infrastructure

and time when people are diverted from

different things, it's also very distracting.

So wherever possible we need to consolidate

the guidances. We also need to start working

on one submission standard throughout the

agency, one standard for all the things we're

working on, and we also need to reach out to

a global community. You know, although for

all the same reasons that I've talked about

so far about consolidating your resources

where most of these companies are global

organizations. Although you're going to get

good cost savings and good sharing of

resources working internally, we also need to

work externally as well.

And the first thing that you know

give a little story of when we started the

regulated product submissions our goal was to

create one model that could be used for all

regulated products worldwide. And the first

thing that people said was you know it

couldn't be done. And a lot of it was

because of different business drivers. You

have in medical devices, they regulate from

sunglasses to pacemakers. You have vet

medicines that have hooves, fins, beaks and

tails. You have drugs that has 100,000

submissions a year, some of them over a

(202) 234-4433 WASHINGTON, D.C. 20005-3701
million pages. You have foods that - there's

an active submission out there that's been

active for over 30 years and they're still

getting information through it. And so

people thought it was very difficult to come

together, though it was impossible.

So what we did initially was we

started off with two groups. It was the

medical device group on one hand. On the

other side we had the drug and biologic

group. The drug and biologic group had a

head start. They have already worked on

electronic submissions in the past. Based on

their experience they created up a submission

model, a draft submission model that we

started to work on. The devices group we're

doing the same. They had the benefit of

working from the drug group because they had

a head start. They said that the model

doesn't meet their needs. They then took

that model, made some tweaks to the model.

Then we presented it to the drug and

biologics group and at that point the drug

and biologics group unanimously voted that

this was a better model that served their


So what we did at that point is we

then started the two groups. We thought this

was great, we were making good progress. We

then asked both groups to come up with a list

of priorities that we wanted to do for the

project. What we found out is both groups

independently came up with the exact same

list. The orders were different, as you can

imagine. In a device submission you know

they have a lot of products, a lot of models

you know in a device submission. In a drug

submission there were issues more because

they have a standard already. We had issues

more of well we need to be able to

communicate back and forth to the sponsor.

But we found out the list was exactly the

same. One and two was flip-flopped, three,

four and five were exactly the same. So we

(202) 234-4433 WASHINGTON, D.C. 20005-3701
combined both groups and I've been sitting

around talking to everybody that's in these

groups and everyone's happier that we

combined them. We're learning that the drug

folks and biologic folks have a lot to learn

from the device folks. The device folks have

a lot to learn from the drug and biologic

folks. And I look forward to start bringing

the vet med folks as well as the food people

as well. So the one thing we learned from

RPS is that we can work together.

Now, here's the action item for

the FDA. If we're working together you know

as an industry, we need you to start working

together too. We need you to start looking

at all your processes and procedures. We

need you to consolidate your efforts because

when we - and for devices, when we have to

send the submission and then get an

application number, and for drugs the same

exact RPS test submission, we have to ask for

the application number first, it becomes very

difficult. We're going to still have

specialization and specialization increases


The next thing, it goes along the

same lines, is that the fact that I have the

pleasure of working with all the centers in

the FDA. You know I'll let you in on a dirty

little secret. I hope noone from the FDA

pulls me aside afterwards. They have

multiple systems that do the exact same thing

and that comes from their tracking system.

And you would expect that from any large

organization. I know I'm talking to industry

folks. They have multiple systems doing the

exact same thing. I've talked with members

of Abbott, letting their dirty little secret

out, and they have systems that are only for

device folks. They have systems only for the

drug and biologic folks, and they're doing

the exact same thing. And the reason why is

because the people they have to deal with on

the other end are doing things differently.

(202) 234-4433 WASHINGTON, D.C. 20005-3701
So again, if we consolidate our efforts, work

globally, we can get a larger market for the

consultants and the vendors. And it's the

competition of that larger market is where

we're really going to drive down prices.

Because what's very expensive right now is

you have a very small market, all these

different styles are very small market and

you have very few vendors willing to get into

those markets. And those very few vendors

when they do get in the market, to recoup

their costs they've got to charge a lot of

money. Okay, this stops. That's why

everyone that goes into these markets

basically almost all the companies focus on

the big guys because that's only where they

can recoup their costs. They can't justify

charging big pharma millions of dollars and

for the exact same system charging a food

company five dollars. They just can't

justify that. But if we consolidate and we

can consolidate our markets, you know, we're

going to allow more companies to come in

because they're going to see a bigger market.

The processes are going to be consolidated so

therefore it's easier to get in. And when

it's easier to get in we're going to have

more companies. More companies always drive

down costs.

And this is a couple of things

where I'm trying not to throw some criticism

any one way, but I want to make comments

about other presentations. I think for the

CRIX model it's important to realize that if

you have one company that's a not-for-profit

or one for-profit company and that's your

only vendor that you have to choose for,

they're going to cost you more money than 10

companies that are doing the same process.

And the reason why is because there's no

competition. There's no incentive to be

cost-effective. There's no incentive to be

innovative. So you're going to get worse

products for more money. So although I

(202) 234-4433 WASHINGTON, D.C. 20005-3701
believe in the CRIX model, I think the CRIX

model is the way to go, I would urge the

board of the CRIX board to allow for-profit

vendors, multiple for-profit vendors to plug

into the network so that the industry has


My last point is from lack of

industry, lack of input from the industry.

We have good experience both on the regulated

side and on the industry side. We need to

leverage those experiences and we need to

work with organizations that have the policy

and the procedures to allow us to do this in

an open environment, in a fair environment.

And that's related to standards developed

organizations. And we need to focus on

international standards development

organizations. We need to focus on

organizations like ISO where we can put our

business requirements. We need to focus on

organizations like CDISC where we can work on

clinical data. We need to focus on HL7 who

does the messaging, but we need to do it

together. We all need to do that. And the

reason why we need to do it together is

because the cost is too high if we don't.

We've all seen what happened with the ECTD.

We've all seen that it came out and there

were no vendors that were willing to come out

of the gate and say here's a product, start

using it. We saw regulatory authorities slow

on the uptake because without the input from

industry, without the input from the vendors,

without the consultants noone can support us

out of the block. So what we need is we need

to all go in the same development

organizations where we can all go and then

where we can all participate. When we do

that we can evaluate our systems internally,

we can see if the standard meets our systems.

If they don't we can make recommendations to

make them meet our systems. We can probably

do that in a low cost earlier on in the

process until we implement and we'll get your

(202) 234-4433 WASHINGTON, D.C. 20005-3701
support, your consultants, your vendors, your

software, your software companies out of the

block being able to support you. Now it's

not going to happen immediately. That's not

going to say, oh, we have a draft standard or

a final standard and we're ready to go, but

what that means is everyone's been a part of

the process and soon after that you know once

things are finalized, soon after that people

have already been developing. I've talked to

all the vendors that do the ECTDs at least in

the United States, most of them in Europe as

well, and they're all preparing for regulated

product submissions. If this development was

happening behind closed doors they wouldn't

know what was going on, they wouldn't know

the standard or where it was going and they

wouldn't be able to participate early in the

process. They wouldn't be able to support

you when the standard is ready.

Obviously we need strong

leadership. And we're going to need to reach

out to people that we've never traditionally

worked with in the past. You're going to

have to see all of the agencies, we need the

leadership you know from people like Dr.

Woodcock and Dr. Levin and Dr. Oliva to tear

down the walls of the centers and start

working together. We're going to need the

food product associations, pharmaPhRMA,

bioBIO and AdvaMed to start working together,

start interacting with each other, start

making joint recommendations to the FDA. And

we also need to start working more

effectively with our global partners. And

when our global partners are unwilling and

unable to work with us, we need to continue

to extend our hand. We need to realize that

to get our true cost savings it's really

going to come with working with our global

partners. But we also have to realize right

now they might not be able or willing to join

us. So we have to constantly communicate

with them. We have to tell them we're

(202) 234-4433 WASHINGTON, D.C. 20005-3701
working in international organizations.

We're working in open process. This process

doesn't favor the FDA, or the NIH, or any

U.S. companies. And we need to bring them

into the fold.

We also need managerial leadership

to realize that it's cheaper to work together

than to work in silos. We also need these

leaderships to come together and fund new

positions. To work in standards development

organizations is much cheaper than not to,

but it is not cheap. We need reviewers at

the standard development organization table.

We need the publishers there. We need the

managers there. We need IT at the table

working together. We need people inside

companies whose big part of their job is to

work with SDOs and we need people inside the

FDA whose a big part of their job is just to

work with SDOs and to be a liaison.

To sum up, I think I'm okay with

time. It's - to sum up real quick, we need

to first and foremost, although I'm a big

believer, we're all - the big cost savings is

going to be working with the global

community, we need to first look internally.

We need to do what we can and the first step

is really the FDA needs to take the

initiative and consolidate their guidances.

We see what's going on in Europe. They're

making a valiant effort. They want to work

together. It's been very difficult and my

opinion is the reason why they're not moving

as fast as everyone who I talked to would

like to is because they don't have one

person, one authority that can say you're

mandated, you must start working together.

Inside the FDA that's not true. You have the

commissioner that can mandate and says the

agencies can work together. So that is

something in my opinion, I know although

controversial, in my opinion is where we need

to start. We need to work within SDOs in

international SDOs. That will show our

(202) 234-4433 WASHINGTON, D.C. 20005-3701
global partners that we are serious of

considering their needs. We also need to get

the feedback from both industry and agency


And what this is going to allow us

to do, this is going to allow us to share

resources. We're going to get improved and

cheaper infrastructure. This is going to

come from competition. As I said earlier, we

cannot allow one organization, not-for-profit

or otherwise, to monopolize the space. They

won't be innovative. They won't figure out

new ways to make things cheaper. We need to

have competition. And it's very important

that we get industry buy-in early on in the

process. And if you're participating at the

table and you're making your feedback into

the standards development process and then

the standard does not work for you, the only

person that you have to blame is yourselves.

So in short I believe we do all this, we're

going to increase adoption of electronic

submissions and hopefully we can do it sooner

than later. Thank you for your time and I

guess questions.

DR. WOODCOCK: Thank you.

Questions from the panel? Dr. Levin.

DR. LEVIN: I have a comment.

Just to take time now to talk about the - in

standards development organizations it's not

easy to develop these standards and get all

the people cooperating together. And I just

want to thank Jason for the work he's been

doing in the regulated product submissions

standard. It's very difficult to get all

those groups together and you've been doing

an excellent job in doing that. So just to

thank you for that.

MR. ROCK: Thank you sir, and I

don't think I could have done that without

the support of the FDA, yourself and several

industry individuals out there such as Ed

Tripp from Abbott, Karen Sailor from

Medtronics, and so many of you.

(202) 234-4433 WASHINGTON, D.C. 20005-3701
DR. WOODCOCK: Any other

questions? Thank you very much.

MR. ROCK: Thank you.

DR. WOODCOCK: Our next speaker is

Laurie Rose, Director, Product Management,


MS. ROSE: Thank you, Janet. And

I am from SAS. I am the Director of our

Product Management and Strategy organization

for Health and Life Sciences. We looked at

the docket a few weeks ago and decided that

based on some of the experience that we've

had in the industry for over 30 years and

working with the FDA that we might have some

hopefully good contributions to make

regarding those issues of discussion. And we

do appreciate the opportunity not just to

share our thoughts, but also to hear the

perspectives of everybody else who has

presented today.

Two of the areas that we feel

probably most qualified to address are - were

in the sections around the electronic

submissions about the implementation itself.

As not being a sponsor and not having those

costs and time issues but supporting those,

we wanted to focus more on the

implementation. And then also on the section

about third party entities. We've provided a

number of standard tools for the industry for

years for analysis and data management, but

in more recent years we've also been

providing some evolution of our technology to

support things like the CDISC standards and

the electronic submissions. So we wanted to

focus on that from our experience not just

providing those solutions, but also from

hosting those for the industry.

Before you think too much about

that picture, I probably should have backed

up, but I think what most people think about

a submission they think about an event.

There's a timeline. You start a project.

You develop protocols, trial design models.

(202) 234-4433 WASHINGTON, D.C. 20005-3701
You get your statistical analysis plan. You

start running those trials, collecting data

and analyzing those. And at some point you

come to a belief or theory that what you have

is in a desired state and it gets packaged up

and it becomes a submission. And hopefully

there's been some communication with the FDA

along the way with pre-submission meetings,

but still viewed more as an event. And for

this discussion today from our perspective

we'd really like to take a look at that as

more of an ongoing process. Not just an

event, but the electronic submission as a

process itself. And part of what we want to

concentrate on is that theory about that

electronic platform. What can that be and

what can it hold to better enable this

process. We believe that from the sponsor

side you know this is something that would

basically have global accessibility. It

would be internet-based so people would

basically just need thin clients and access

the system anywhere in the world and do that

collaboration in real-time if necessary.

That might be somewhere a little bit more in

the future. I think one of the questions in

the docket was should there be a phased

approach, and certainly developing the tools

and the infrastructure to support this does

need to be phased. Exactly what would happen

in those different phases needs to be

examined I think in more detail to make

strong recommendations along that. But

basically with that platform holding things

like common tools and even a common shared

workspace between sponsors and the FDA to do

that review is a recommendation that we would

like to make to the FDA and see how that

might be able to be accomplished.

Also just from the phased

approach, if we look long-term, and I've got

a slide at the end that will sort of give a

little bit more of that futuristic view, but

initially I think just being able to handle

(202) 234-4433 WASHINGTON, D.C. 20005-3701
those components and that ECTD lifecycle and

the amendments to that, but even further down

the road using the same concept and possibly

even the same type of platform for post-

marketing and reviewing that with sponsors as


If we look at what might be inside

that box, and I'll have a little bit more

detail later, but basically there are a lot

of technology pieces that have to be put in

place and infrastructure from whoever is

maintaining those systems to create those

firewalls and those partitions to allow for

the privacy that's needed so that the

sponsors using the system can have their own

private workspace for development and

testing. And when those elements are at a

point where they want some preliminary review

to promote those to more of a shared

workspace. We're also looking at the same

type of platform for supporting something

like JANUS where the FDA has their own space

where they're collecting all of that

information. You've got probably the best

treasurer of clinical information in the

world and being able to mine and analyze and

understand that data in a much better way.

One of the things that we look at

from our own perspective around cost, and

we've been providing a hosted system as an

offer probably just in the last four or five

years now. And we are seeing more and more

of the industry asking us to provide that and

give them that service because there are so

many benefits. If you look at what a box of

software would cost if you're getting a set

of CDs and you're going to take that software

and manage it and install it and validate it

yourself, there's a certain cost to that and

it's certainly going to look lower initially

than if you look at the one that's going to

be hosted where somebody else is providing

hardware and services with that. But we have

seen not just from our own analysis, but more

(202) 234-4433 WASHINGTON, D.C. 20005-3701
from the return on investment analysis of the

industry where we're having both large and

small pharmas asking to have that hosted

because of that reduction in all the

resources, not just financial costs, but the

burden for IT primarily. I believe it was

ThinSpring earlier who mentioned the

exponential cost when you start looking at

complexity and just that one validation

component. That is something that is an

exceptional part of the puzzle when you look

at having to install and maintain and

validate a regulatory compliant environment

for FDA submissions.

I would have to say that we are a

very strong supporter of CDISC. We're proud

to have our own staff, Dr. Ed Helton, as one

of the board of directors. We also have

member on the Industry Advisory Board and a

number of our research development and

consulting staff on the technical committees.

So we would certainly want to continue that

promotion with the FDA to support those

standards. I think our comment around that

though is that we see both as a vendor as

well as working our sponsors trying to

implement those standards that they do need

to be simplified. There are many and I do, I

believe we lost our CDISC people, but very

well aware of the great efforts that have

been made to get those standards to a point

where they are today where there has been

agreement with the industry to come up with a

number of those models that they can

implement. But we see such a vast array of

difference between the CROs and the sponsors

who are implementing those standards.

Everything from the ones who are just trying

to see how they can cram everything into SDTM

at the very end so that they can do a

submission using the guidance for SDTM maybe

for the first time to those who are really

embracing what that business case might be

and using those standards throughout the

(202) 234-4433 WASHINGTON, D.C. 20005-3701
entire process, and trying to really gain

some of those benefits of using that standard

data, especially the global organizations

where they've got multiple sites, multiple

entities who are contributing a specific


I think the bottom line in what

we're trying to comment here is that there do

need to be more tools. And I'll even take a

moment to maybe kick SAS a little bit and the

other vendors and the other providers. I

know Octagon provides services to do CDISC

transformations. A lot of the sponsors don't

know what's available. They don't know what

tools are there. They're aware of many of

the review tools that have been developed

with the FDA through CRADAs, but they don't

all apply necessarily to the submission

process. And so they just don't really know

what's available. So for those of us who are

providing those tools, we need to probably do

a better job of educating the market about

those. And then the FDA, we also want to

support having those standard methodologies,

whether it's through guidances or the support

of those standards for developing an ECTD

using CDISC standards throughout the process.

Just to liven up and get away from

all that text, here's a nice colorful slide

with a lot of text on it. Don't want you to

get too bogged down in what's in all of the

boxes, but this is basically to sort of take

a step into that platform. There would be,

you know, the sponsor's environment, they've

got a lot of inputs coming in, they've got

clinical data management systems, they've got

EDC systems. The one, there's a small box

right in the middle there, that meta data

integration, and that's pretty critical

because that's where all of that information,

whether it's TOX data, PK lab, the clinical

information, whatever it is, is going through

a process and hopefully as CDISC standards

become adopted more, that'll more and more

(202) 234-4433 WASHINGTON, D.C. 20005-3701
become CDISC standard data that's going in

through that process and eventually being

merged with the documents and the other

information into an ECTD.

This shows kind of a handoff

between the sponsor world and then this

shared world where there is this compliant

repository, where there are capabilities like

audit trails and electronic signatures, a

meta data registry that allows that access

into the data and the analysis and the

results, and then it's all being hosted by a

third party system. Now this picture I have

to say I kind of cheated a little bit. This

is not a SAS picture, it's really just a

visual, but SAS has experience with something

that's very similar to this. I hope most of

you are aware of the FDA pilot that was done

with CDISC recently where there was a mock

submission. There were a number of industry

players who provided a lot of work and a lot

of input, and they actually used SAS's

electronic platform to host all of that

information. So these different parties from

different areas across the country were able

to load and put the information in there.

And then the one thing that was different is

that it was not a shared repository. There

still was that package and handoff at the

end. But that mock submission was also done

in a matter of months. It wasn't done over

years. And it was done very quickly, and

hats off to everybody who was involved in

that project because it was really amazing

how much they accomplished I think between

December and September of this year when they

presented at the CDISC interchange. But we

certainly would promote that this type of

shared environment we do believe would have

great benefits in reducing that overall cycle

time in running a trial and getting to a

submission standpoint.

Let me switch gears a little bit

to the third party entities. And I'm really

(202) 234-4433 WASHINGTON, D.C. 20005-3701
- we're really approaching this from how we

have been used as a service provider in

hosting a system for the sponsors and for the

industry. The first bullet point there,

neutrality, what I really mean by that is

that the person hosting that system does have

to be neutral. In other words, there need to

be some definitions around what can be

provided by the person doing the hosting. So

you're hosting, you're providing

infrastructure, you're not accessing our

touching the data in any way. So that's

really what we're meaning by "neutrality."

The experienced third party is

just critical because there are risks, very

high risks, that are assumed by that third

party when they take on that responsibility.

But one of those is to take the risk off of

the people using the system. So for example

if you're concerned about audits and whether

you're going to pass an audit, you need to

have all the standard operating procedures in

place. You've got to have the ability to do

that validation and do it correctly to

alleviate that risk from the people using the


On the business process modeling,

we believe that that third party can

certainly be a facilitator, but should not be

dictating what processes are in place

especially if we look at a shared environment

between the FDA and the sponsors. Those

processes need to be agreed upon by both

parties from a business modeling standpoint.

And just on the collaboration side, excuse

me, I'm hitting a mouse here accidentally.

The framework for collaboration - I'll just

skip down to that part just a little bit.

The security mechanisms have to be put in

place. I've heard a lot of people say

security, security, security as well as

communication. But that is important, not

just from the physical security of what's in

that data center, but also privacy, user

(202) 234-4433 WASHINGTON, D.C. 20005-3701
access, things like that. And the user

profiles themselves. Whatever is in that

electronic platform, there are going to be a

number of different tools and applications.

It's not going to be just one system. So

what types of users will be accessing it and

what level of access do they need. Do they

need programming access, do they just need to

be able to do simple reporting, but

understanding those profiles of the people

accessing the system. Managing things like

change, the change management procedures and

the standard operating procedures in place

for that, the whens and the wheres and the

hows. And then also having some agreement on

the governance.

On the services side, and Jason

really - he hit it hard. We really believe

that whatever services are provided, there

should be some very standard services for

providing the infrastructure, providing the

implementation of that and providing some

training around the implementation of that

platform. But additional value-add services,

consulting on how to do things more

efficiently using the platform, that's fine

for that third party entity as long as it is

an openly competitive environment. So Jason

hit that one hard, and thank you Jason. I

agree wholeheartedly with you on that one.

The technical services, I believe we also

heard some notes about this earlier, but

certainly things like disaster recovery and

backup and all the planning that are

associated with those types of technology

services that need to be provided as well.

These are some of the attributes

that we see that are necessary. The first

one just being 21 CFR Part 11 compliant, and

that sort of goes without saying except that

there's some complexity in managing a system

that has that. So that third party needs to

be able to provide that type of service and

that validation. Security, once again,

(202) 234-4433 WASHINGTON, D.C. 20005-3701
whether it's at a user level or a data level

within the physical entity itself. And then

the scalability and performance. And

certainly as we have gone through some pilots

with customers who want to expand users very

quickly. We know that you have to be able to

have the ability to grow. The third party

entity needs to be able to expand hardware.

They need to have data center capability.

They also need to be able to meet the

expectations on performance, response time,

hitting a button, how long does it take for

something to happen. If this electronic

platform is going to enable people to move

more towards electronic submissions it can't

become a bottleneck so performance is very


I think we talked about the

registry and repositories, basically having a

meta data-driven system that allows for data

loading and exploration and analysis,

reporting. Some of that goes back to those

user profiles. What are their needs going to

be? What capabilities need to be in there?

A couple of additional attributes,

and these have been requests coming

specifically from the industry. If there is

an electronic platform, they do want to have

that global access, it needs to be web-based.

It does need to integrate with different

types of tools. It needs to have some

extensibility and openness. And I know

within our company we're looking at things

like service-oriented architecture. We have

capabilities within our own system right now.

We need to be able to expand those into other

areas and other integration with other open

tools as well as other industry tools. So

all of those capabilities need to be part of

the system as it grows. A lot of people are

also requesting more work flow. A lot of the

smaller organizations, the biotechs, are not

- for many of them it's the first time

they've done electronic submission and

(202) 234-4433 WASHINGTON, D.C. 20005-3701
anything that you can build into a platform

to make that process easier so they have to

maybe pay less to those consulting resources

that help them is very valuable. We also see

a lot of the larger pharma who would like to

have some automation just to speed things up.

We also see that it's very important to

support that end-to-end process of

information flow from HL7 to CDISC and

through the ECTD process. And then the

sponsors themselves also want that common

area supporting the standard data that they

are trying to adopt themselves in their own

organizations to do cross-study review.

A little bit about that future

look. What's in green is a little bit out.

Probably not very many people are using that

kind of methodology right now. It's more

what's happening in the yellow. It's just a

little bit of a different view of what we've

looked at before, but what we believe is that

with the electronic platform, as CDISC

continues and the organizations really help

start building the standards for the e-

protocol and trial model design, that

electronic platform is going to be fed with

really automatic feeds that are set up with

data and meta data coming into the system so

that there is a meta analysis. There are

automated procedures for doing that analysis.

And the safety and efficacy results of that

analysis can then be used to feed back into

those systems things like adaptive trials to

really help make the whole process more

efficient, to take some of that guesswork out

each time, to use that information and

improve those processes, improve those design

models and those protocols as we move


Any questions?

DR. WOODCOCK: Thank you very

much. Does the panel have questions? No?

All right. Thank you very much again. Our

final scheduled speaker today is Terence

(202) 234-4433 WASHINGTON, D.C. 20005-3701
Zagar who is Chief Technologist at Northrop

Grumman Information Technology Health

Solutions/Life Science.

MR. FISHER: Thank you, Dr.

Woodcock. We have a slight change to our

speaker list. My name is Harry Fisher and I

am the General Manager for Life Sciences at

Northrop Grumman. And I'm accompanied by our

Chief Technical Officer Rim Cothran and Terry

Zagar, one of our program managers as well.

So with that being said generally we'll be

talking about the same things. But based on

how the day has gone, and I will say I think

I heard someone, I think it was Debra said

she had the worst speaking slot. But as I

look at the empty seats I say hmm, I wonder.

But I think those empty seats are going to

see that the best was left for last hopefully

and we can add some value here.

So one of the questions we get and

one of the areas we're going to address is

first off very quickly the question always

becomes why is Northrop Grumman here. Don't

you guys build the B-2 bomber? And yes, yes

we do do that. But we have about an $8

billion IT practice of which about a half a

billion of that is for health solutions. And

what I'd like to offer today is not some

theory around some things that might be

interesting. I think there's a lot of `Mom

and apple pie' here that absolutely makes

sense from an e-submissions standpoint and we

certainly support CRIX. What we'd like to

offer are some specific points regarding the

third party entities that we have direct

involvement with today and share some

experience that could be directly germane to

the activities that CRIX could be doing or

any potential third party entity.

So some of the questions that we

wish to address. I'm not going to read

through these, but are around models for

setting up a third party entity. I believe

it was Bill Rosen said P3, public-private

(202) 234-4433 WASHINGTON, D.C. 20005-3701
partnership. What does a P3 look like?

What's the experience with it? What does it

cost to ramp up? What about this thing

called monopolies? And is cost-effective

mostly in a competitive environment, a

different environment, et cetera, et cetera.

So these are the questions that we wish to


Why is this important for us to

talk about this? Well, I've highlighted five

programs that I think can show some various

ways to look at the analog of CRIX along with

any other type of submission agency that

would be involved here. The first one is

AHLTA which is something we do for the

Department of Defense and the second one is

the Federal Bidirectional Health Information

Exchange which we do for the VA. Those two

you can look at together, but what's

important about those two? What's important

is that Northrop Grumman runs the largest

electronic healthcare record system in the

world. It's live. It operates today. So I

know there's been a great deal of talk about

EHRs and how they work and how they wouldn't

work. We do this today and on a conservative

estimate we have 10 million beneficiaries

globally. On an extended beneficiary list we

have 20 million. The difference being is

that on the one hand for the VA we handle

things like the technical aspect of how this

works. We don't tell them the process. We

don't tell them the content. We allow for

bidirectional communication. And while in

certain cases in certain environments people

will say the technology is the last thing you

need to worry about. Well, what we find is

if the technology doesn't work nothing else

works. For the Department of Defense we do

something a little bit different. We deal

with the processes, the content, the

strategies and the technologies underneath of

that. So we deal with eight to ten separate

proprietary EHR systems on a global basis and

(202) 234-4433 WASHINGTON, D.C. 20005-3701
try to figure out how to make them all work

and talk so that DoD gets what they need for

their beneficiary base.

The two other ones beneath this,

PHIN, NHIN. Rim will talk a little bit more

about NHIN, but we have over a thousand

people in Atlanta working on activities like

this, a public health information network,

and what that means and what the benefit

would be to the populace in the U.S., but

also because it's a globally-based activity,

how do you enforce that. How do you make

that happen.

And then finally what's of

interest here and it might be very germane

for this discussion as well is MedDRA. We

are the maintenance and service and support

organization for MedDRA. So for anyone out

there who deals with MedDRA, has a

subscription to MedDRA, you've dealt with

Northrop Grumman. Northrop Grumman maintains

that solely and in completeness. And we do

that under the organization which we'll talk

about a little bit later which is a not-for-

profit organization.

So when we think about what that

means to us and we think about some of these

questions, what do we see? Well, when we

think about the question of what are the

things that are important, multiple

stakeholders. It's almost essential that we

see a public and private entity

participation. And we see that very clearly

because in the cases where things actually

have to happen, if we don't have both support

in the public and private sector then it

stops. And for one hand on the public side

what we see is that's where the mandates

come, that's where the regs come from. But

if we look in the private sector, if there's

not an associated ROI it stops dead in its

tracks. If we go down to different

stakeholder needs, it's very, very different.

But what you have to always promote is a

(202) 234-4433 WASHINGTON, D.C. 20005-3701
collaborative interoperable environment.

Now I'm going to use the case

example for what we do, and we do a little

bit more of this in MedDRA, but in MedDRA we

have stakeholders globally. We have a 25-

person management board. They're made up of

government individuals, NGOs, industry

associations, private partnerships, all over

the globe. And I can tell you there are some

groups in there who will say no one should

ever make money on anything. So how can we

actually charge people for MedDRA? Then

we'll have other people who say well you know

what, we should charge them as much as they

possibly can bear because then we can invest

those funds in other things as well. Now,

we're not in that discussion with the voting

member. We're in there to help facilitate

the discussion, but the fact of the matter is

we're the ones who implement what comes out

of that. So in terms of the different

stakeholder needs we see that.

We see workable models for a

public-private partnership in many different

ways and some of those we've already

discussed. But if you get down to the bottom

line, there are multiple workable models and

we're experiencing that right now. So I'm

going to skip this. Rim will come back to

this. But the model that you see here, and

I'm not going to go through all of this, is

related back to what we do with MedDRA right

now. Now what looks very complex there can

indeed be very, very complex. And rather

than go through that model, you can read

through that. How did that evolve? It took

years for that to evolve. So we're in Year 5

of this agreement. We've just mutually

extended the agreement through 2011. And

what that means is we've gotten really good

at doing this. And it took a long time to

figure it out. And it cost a great deal of

investment. And so there was a - and I don't

have the graph for this, but you know if you

(202) 234-4433 WASHINGTON, D.C. 20005-3701
look at your investment and finally where you

get to the breakeven point, it took a great

deal of time to get there. But here's the

benefit. The benefit is that in this area,

that model that you look at, the costs for

the average subscriber have decreased by 20 -

50 percent over the last two years. Why is

that? Because we figured out how to do this

model. We figured out how to efficiently get

membership information in to change the

schemas as required, to go through the

process of medical evaluations and process

evaluations, all the different things

involved in this. So in any event the way I

look at this is there's something out there

that makes sense.

And then finally from my side,

these are the things that I think will help

in terms of separation of responsibilities.

One of the things that we don't do both in

our public work with MedDRA, what we don't do

with our work with DoD and with VA, we don't

have a vote. I mean, if the board decides to

go in a certain direction, if the DoD decides

to go in a certain direction, we have to

carry those orders out. And they have to

make sense, and obviously we have influence

over that because we have data and we have

observations. But the reality is as you look

at this there's a separation of

responsibilities throughout. But the key

that I would leave you with is that it is not

a vendor relationship. In this model if we

go forward the stakeholders are vastly

different. There's certain competitive

advantage. There's certain non-competitive

advantage. But if you think that you can

have a vendor perform tasks for the lowest

possible price you will get exactly what you

pay for. You must invite partnership in this

type of scenario because the folks who will

be closest to how things actually work will

be the ones who can help you make decisions

as to what's the best public interest both

(202) 234-4433 WASHINGTON, D.C. 20005-3701
now and in the longer term. So with that

I'll turn it over to Rim Cothran to talk a

little bit about our experience with NHIN.

MR. COTHRAN: Thank you, Harry.

Before we get started on this slide I think

the first question you might end up asking

yourself is why are we talking about NHIN. I

mean, NHIN is - the Nationwide Health

Information Network is a mechanisms for

exchanging clinical information among

physicians for point of care. And yes that

is what the primary goal is, but

fundamentally what we're talking about here

is an information exchange that takes into

account the business processes, regulatory

environment, privacy needs and security needs

of the health industry. And I think that if

we talk about how NHIN is being developed and

the public-private partnerships that are

being put in place there to support its

development, it gives us at least one model

that we ought to look at for other parts of

the health industry as well. Also, if we

think about NHIN which is represented as the

central circle in this diagram here, it's

usually the solutions themselves that people

focus upon.

Well, what the Office of the

National Coordinator for Health Information

Technology within HHS put in place was really

a number of parallel efforts that all worked

in conjunction with each other in order to

develop an overall solution. So we'll start

at the top of the chart here and talk a

little bit about practices and policies.

They established an organization that their

charge was to look at the practices and

policies associated with privacy of patient

information. And it's important whether

you're looking at regulations that are

established at state level as it is for

clinical care or more broadly at the national

level as it is for the regulatory industry,

it's important to establish those policies

(202) 234-4433 WASHINGTON, D.C. 20005-3701
and to continue to maintain those policies.

Electronic submission, electronic exchange of

health information allows for different

problems, different issues than we've had in

the past and it's important to continue to

monitor what policies need to be in place to

facilitate it.

As we move clockwise around the

circle next we reach certification. And what

they also put in place is a separate

independent organization charged with

certifying systems that end users would end

up using. The primary reason for that was to

help those end users understand the

functionality in a competitive environment.

We've already talked a little bit about why

it's important that that competitive

environment exists, and it is, but it's also

important for the end users to understand how

different products address the needs in that

competitive environment. And certification

and a body that supports certification is

important in order to realize that.

So we continue to move around to

standards. What they also put in place was a

public-private partnership to identify main

standards for information exchange and the

implementation guides that should be put in

place. One of the things you can say about

the health industry is there are plenty of

standards today. And in fact, the HITSP that

is the embodiment of the standards

organization identified well over a thousand

standards for health information exchange and

need to identify exactly which standard were

properly maintained, how they were

maintained, identify which ones met the needs

for information exchange and how they would

be interpreted to best implement health

information exchange in a uniform

environment. As I said, that's a very good

example of a public-private partnership in

that that organization is made up of 200

different independent organizations. It

(202) 234-4433 WASHINGTON, D.C. 20005-3701
includes standards development organizations,

it includes private industry, that has a

stake in this industry and is interested in

providing services for it. And government

agencies that are both interested in

regulating, promoting as well as

participating in that exchange.

And then finally we reach back to

the Center for Architecture Exploration for

NHIN and that is really to bring about

solutions for the exchange of health

information. What's interesting about the

approach to development of the Nationwide

Health Information Network, the NHIN, is not

that government was interested in putting

that in place or that they were interested

necessarily in establishing a single vendor

that would own NHIN and that's who you would

approach. But instead, inspiring industry to

take on this as an endeavor by funding early

organizations to explore different solutions

and in a public format describe what those

possible solutions were. And in fact, the

culmination of this program is coming up soon

in January where the public is invited to see

demonstrations of the NHIN that have been

developed by a number of vendors working in

partnership with all of these organizations.

What's important is that these organizations

don't only operate as public-private

partnerships themselves, but the

interoperation between them is also

important. The certification body takes

standards that are developed by the standards

organization in order to produce the

guidelines for certification. And both of

those bodies have to understand the solutions

that are being developed as part of the NHIN.

If we move on to services, now we

talked a little bit already today about the

different services that might be developed or

might be offered by a third party provider.

There are really two different buckets that

you might put these services in. The first

(202) 234-4433 WASHINGTON, D.C. 20005-3701
listed here is provider services that are

primary services. The electricians and

plumbers that are actually doing the software

development, doing the interface development,

establishing the infrastructure, et cetera,

and we've talked a little bit about that and

the need for that to be a competitive

environment. Also, the organizations that

maintain the standards as participants in

standards development. There's also a need

for secondary providers, certification

bodies, auditing bodies, et cetera, that

monitor the exchange of information.

What's also important though is

there is this second bucket. There is an

opportunity for providers of services that

actually establish and facilitate the

exchange of information. This is a

developing concept within NHIN and ONC today.

But the idea that there is a place for an

organization that acts in the same way that

your internet service provider does, that

establishes connections between organizations

and facilitates the movement of information.

Some of the services that are necessary to

facilitate that for NHIN are listed here and

I won't go through all of those because this

is specifically about the movement of

clinical information, but a few of them are

probably interesting to highlight. The first

is the translation. Really what we're

talking about there is an arena where the

standards are developing, standards are

changing and not yet established and there

are legacy electronic systems within the

organizations that may want to exchange

information. There is a place for an

organization that provides as a service a

mechanism to translate between different

standards, to transform information from one

standard to another to facilitate the easy

exchange of information. It takes the burden

off of the exchange in organizations and the

receiving organizations, but still promotes

(202) 234-4433 WASHINGTON, D.C. 20005-3701
the use of those standards.

Many of the other services that

you can imagine here would support routing of

information to the right organizations that

provides querying of information, et cetera.

What you will see missing from this list

though is it's not necessarily that this

third party organization build a repository

to hold all of that, that it is possible to

just offer services that facilitate the

exchange of information without needing to

own that. You can think for instance, again

as I turn to your ISP. Your ISP does not own

your financial information, but facilitates

the transmission between your bank and your

own financial application at home without

actually storing any of that information

centrally. And there is a place within the

health industry also for an organization that

we've begun to call network service provider

that exchanges that information without

actually owning it. Terry?

MR. ZAGAR: So there actually is a

Terry Zagar, as everyone knows. What I'd

like to do is to start to wrap up here and

finish up the day. One of the questions I

think that was posed by the FDA was what

kinds of things can the FDA do to actually

support the establishment of a third party

entity and service. I think when we consider

the FDA, there's really two roles that we

see. One role is really as a user of a

system. FDA will be a point node if you will

in the contemplated electronic information

exchange and so there's specific requirements

and those requirements are going to actually

represent diverse points of view across the

FDA. We also though see that the FDA as a

regulator is very much involved in the review

of the system. They will be reviewed in the

validation activities associated with the

platform. They will be reviewing the

processes that are being used by all the

people who use the system. There's a

(202) 234-4433 WASHINGTON, D.C. 20005-3701
question of that guidance from the FDA also

being consistent in terms of how it gets

provided. In both of those areas what we

would recommend and what we would like to see

is that the FDA speaks with one voice. And

this is important in terms of how this comes

across from a standards perspective, but also

in terms of what it communicates to the body

at large. I think industry does not want to

have to deal with different points of view if

you will and the extent to which FDA can help

to facilitate that by having coordinated

positions and bringing those forward I think

would be very powerful.

Another area I think where we

think FDA can contribute here is really in

recognition of some of the stakeholders.

We've talked about patients as being one of

the stakeholders ultimately when you talk

about the efficacy and safety of a product,

but one of the - if you look at where a lot

of the source data comes from, it's from a

medical practitioner who's performing and

supporting clinical trials, collecting

information and validating that information

is reasonable and rational and that they

actually performed or administered in

accordance with the protocol. By the same

token though, those same medical

practitioners are also involved in other

health-related activities over and above just

clinical trial support. They're dealing day

to day with patients. To the extent to which

you can provide a service that supports both

those medical practitioners in the clinical

trial realm as well as in their day-to-day

activity realm would be very powerful in

terms of getting them to buy in. If I looked

at stakeholders today and I said well, who

really represented the medical practitioner,

I didn't see that voice really here as part

of the comment. And we think there's a good

opportunity for some of that discussion as

Dr. Cothran talked about what's going on with

(202) 234-4433 WASHINGTON, D.C. 20005-3701
the NHIN. There's opportunities to leverage

that existing investment that's going on at a

national level to say what additional

functions can be taken advantage of and

leveraged for some of the contemplated

electronic exchange here.

The final point I think we'd like

to address is this whole question, and I

think it's come up several times today, of

mandates versus incentives. If we go back

and look at MedDRA as a model, if you look at

the data in terms of adoption rate from when

we started on day one with the MedDRA MSSO in

distributing the terminology back in `98 and

where we are today, you know it's gone from

zero to 1,500 subscribers. But there's been

some very significant spikes if you will in

adoption, and those spikes corresponded

almost exactly with mandates from the

Japanese Ministry of Health, Labor and

Welfare that people would use the MedDRA

terminology as part of their submissions.

Also corresponded to the similar mandate from

the E.U. in terms of the European Medicines

Agencies requirements. So I think mandates

have a role here in pushing adoption. I

think I have to agree with some of the

comments that were made today though that in

mandating the use of electronic submissions,

the FDA should do that in the context of a

window, a timeframe in which that should

occur. It should also exist with a waiver

process for those organizations that for

whatever reason are not able to actually

accomplish the transition in that timeframe.

At least they can establish a plan by which

that will happen. We've used that very

effectively in some of the activities we've

done that I've been associated with with the

SAFE-BioPharma Association in trying to bring

in certain providers to support credentialing

for a lot of the pharmaceutical companies.

On the incentive side we also see

that incentives can play a very strong role

(202) 234-4433 WASHINGTON, D.C. 20005-3701
here. To the extent that there's a business

case I think that helps a lot of the large

pharmaceutical and the smaller pharmaceutical

companies to participate, but we also see

that incentives play a very significant role

here in terms of can you reduce - you know,

if you submit electronically do you get some

benefit in terms of somehow your application

or submission goes to the head of the queue.

I think those types of activities would be

very important to facilitate adoption and to

promote it. I don't see that you know

necessarily that the time for review is going

to go down with an electronic submission

platform. I think more time, you know the

time box is the time box and that will be

used for quality purposes rather than for

sorting through paper purposes and so you

will get better results. And to me that's

the ultimate goal everybody wants.

To sort of wrap up, in terms of

the question of the concept and the

feasibility of a electronic platform

facilitating information exchange, we think

that's very feasible and it can be

demonstrated. We also think that this whole

question of - I think Bill coined the P3, the

public-private partnership term. We very

much think that that's required for success.

All the organizations do need to participate

and all the stakeholders do need to be

represented. And the structure I think, as

Harry talked about such as for MedDRA where

there's actually a structure whereby

different points of view can be accommodated,

but also no single point of view necessarily

is pushed over another.

And then finally you know is it a

single entity that should do this or multiple

entities. We're of the opinion that multiple

entities are required. You may start off

small with a single entity because there's no

basis if you will from a scalability and from

a business case to say we need more

(202) 234-4433 WASHINGTON, D.C. 20005-3701
providers. But at some point if for instance

this is mandated, then I suspect you would go

very quickly to multiple entities to support

this. If it's not mandated and has to grow

slowly I think you'll start with a single

entity and that will lumber along for some

period of time. Again, I think there's

analogs that we've seen out in industry where

that's been the case. And with that if

there's any questions.

DR. WOODCOCK: Questions from the

panel? Go ahead.

DR. NARDINELLI: Yes. I'll work

backwards. It's not too surprising that when

you get a mandate you would get a spike, but

would you say that that's had much effect on

your overall trend? I mean are we talking

about just a general upward trend with an

occasional spike, or are the spikes, the

mandates, driving the upwards trend?

MR. FISHER: What we've seen,

generally speaking, is it would look like

this. If you looked at the trending it would


DR. NARDINELLI: Right, that's

what I'm picturing.

MR. FISHER: So it wasn't -

there's no step function activity.

DR. NARDINELLI: It's not a step.

So it's really you know, it could well be

described that it's just speeding up what's


MR. FISHER: Right. And what you

also see in that regard too is that we do see

even today, we see folks drop out. But we do

know generally for a fact, I mean we can't

say for certainty, but any organization

involved in drug development has a MedDRA

subscription now. So I mean that's not

really up for debate at this point. And so,

but the growth period is pretty much over

unless we're talking, which we are, about

going into the other geographies.

DR. NARDINELLI: And then I have

(202) 234-4433 WASHINGTON, D.C. 20005-3701
one other question. You have actual

experience with falling costs. What is the

source? Is it happening at the platform

itself, or is it at users becoming better at


MR. FISHER: Well, it's

interesting. From the MedDRA perspective,

which again it's the one that's out there,

it's a live one, is that the users themselves

don't actually do a lot in terms of the

MedDRA process. They take it and they

consume it and what will typically happen is

that in a major or smaller or solo

practitioner, they simply have - they can

interact with the MSSO and then they get the

information they need. But what we've

managed to do is we've gotten the process

down to where the expectations and what we'll

do and how fast we'll do it and then also

when we will update the standard, depending

on what the basis is, we've managed to get

all the stakeholders on the same page so we

don't chase that 20 percent with 80 percent

of the cost anymore. That's really where

it's come out. And so we've learned how to

do this better and so the systems underneath,

I mean we've commoditized those. So the

technology underneath, the platform, the

hosting, all that, that's being commoditized.

DR. NARDINELLI: Okay, thank you.

DR. WOODCOCK: Additional

questions? Ken?

DR. BUETOW: Could you drill down

a little bit deeper for how you all worked in

public-private partnership and implemented in

the Federal Health Information Exchange with

the VA and the Public Health Information

Network? Because not to take anything away

from the NHIN, but that feels much more

conceptual yet to me, whereas I know you guys

actually have boots on the ground and there's

actually working systems in PHIN and Federal

Health Information. So what is the public-

private relationship, how is it deployed, how

(202) 234-4433 WASHINGTON, D.C. 20005-3701
do you actually deal with the delivery of


MR. COTHRAN: Well, both of those

two are very different programs. The

FHIE/DHIE is a longstanding program. It's

been up for about seven years now in

operation and as you say, where NHIN is still

very much conceptual in the pilot stage, FHIE

has been operational. That whole program

came about with the VA recognizing that they

needed an exchange of information. The real

need was movement of military personnel from

the DoD into the VA system and a need for the

VA to exchange among its different clinics

and hospitals. The VA established the

program itself, identified contractors to

develop that system and there is a single

system that's put in place at the VA, largely

operates itself now. We support that through

technical refreshes, et cetera, but that's

primarily a VA-owned system. Go ahead.

DR. BUETOW: So basically in that

model it is a single system. It is a single

contractor who's actually executing and

running that particular facility.

MR. COTHRAN: That is correct.

Now, as it moved to bidirectional, now we're

looking at two agencies interoperate with

each other, with the DoD and having a

completely separate architecture than the VA

system and needing to interoperate among

themselves. It turns out that we are now the

single contractor that supports both of

those, but that happened through an

acquisition. And the truth of the matter is

is it was two separate organizations, two

separate contractors that needed to work with

themselves and two separate federal agencies

that needed to work between each other in

order to bring that about. And that was

established before the acquisition, so it

wasn't the acquisition that facilitated that.

Now if we move to the PHIN, the

PHIN is also - it's hard to say when the PHIN

(202) 234-4433 WASHINGTON, D.C. 20005-3701
started. Often you'll hear people talk about

two or three years ago is when the PHIN was

actually established, but the underlying

systems that make up the PHIN today, many of

them are longstanding. Our support of the

PHIN was again in a very tight partnership

with CDC. CDC owns that network. We did not

develop it for them and hand it to them, but

it was a give-and-take among both of us.

That doesn't mean that while CDC is managing

the project that they also architect it,

design it, et cetera. We had architects and

designers that were part of that. We would

help establish policy. We'd help establish

standards that were associated with that.

And we weren't the only contractor there. As

Harry mentioned, we are the largest and we

support everything that the PHIN does, but

there are other contractors that are part of

that as well to produce a single product, a

single integrated family of applications, but

working with a number of different

contractors that each interact with each

other and with the CDC to bring that about.

Now if you give me just a second

to talk about the NHIN, it is a very

different construct than this. And what HHS

is proposing there and what ONC has been

promoting is that there is an environment

there not for a single solution, but for

multiple solutions that need to interoperate

with each other. And they started that by

awarding four contracts to four different

vendors for only one of them and having them

independently establish solutions that would

meet the needs of specific organizations that

might want to connect. Because all of those

organizations may have different

philosophies. And as we move into next

year's activities within NHIN, you're going

to continue to see independent solutions

there that will probably interoperate with

each other. I think the model there that you

should be thinking about is how cell phones

(202) 234-4433 WASHINGTON, D.C. 20005-3701
work, that there are multiple vendors, but

whether I am a Cingular or a Verizon

customer, I can still call you and it's

because those vendors taught the standards

necessary that allow them to interoperate

with each other. And that's how I believe

NHIN is going to grow up, so it will look

different than FHIE will.

DR. WOODCOCK: Any other

questions? All right, thank you all very

much. This concludes the scheduled speakers

for today's hearing. Are there any members

of the public who would like to make some

comments at this point? If you could take

the microphone there and identify yourself.

MR. BARD: Good afternoon. My

name is Bob Bard. I'm with Aastrom

Biosciences. We're a small biotech company

in Ann Arbor, Michigan. I know we try to get

some small industry conversation, but as the

panel reviews what they've heard today, and I

know it's a lot of information over the last

eight hours or so.

We stand at about 16 people.

We're in cell tissue and gene therapy. We're

a cell company. The ECTD which was talked

about in great detail is a very rigid kind of

document and for us to try to fit in there

has been very difficult. In fact, we started

and then pulled back because it just didn't

fit. When you listen to some of the people,

especially Edward from PATH, he's a good

proponent of this, but he misses part of the

things that I have had conversations with him

about. We're a small company. He's got

maybe five or six or seven or whatever number

of people. In my whole regulatory department

there are two of us. I have a clinical

department of three. When we start looking

at these kinds of things, it's very, very

difficult. We're very much in favor of

electronic documents. In fact, the mandatory

part of it is one of the ways I am going to

push it higher and higher. Think about this

(202) 234-4433 WASHINGTON, D.C. 20005-3701
in great detail though. There are costs in

personnel training, cost of acquisition of

software, cost of acquisition of hardware.

There's a multitude of sins involved in this

that haven't been brought to light in this

meeting and it's not just ECTD. What I came

here for is I though this was a global

discussion of electronic submissions. I

remember being at the L.A. District Office

I'm going to guess about maybe eight or nine

years ago. We were talking about

registration being all-electronic. That

hasn't occurred yet.

I'm in great favor of making as

much of this electronic as possible, but for

small companies, implementation is very, very

difficult. What we have to learn as we go, I

have a very big distrust of the vendors that

supply this software. I don't know if

they're going to be around tomorrow and I

spend anywhere from $20,000 to $100,000, for

a small company that's a big expense, and

then I have to hope that they're going to be

with us. As you go through this, I'd like to

see some way of getting this down to a much

simpler style. XML is fine. Does it have to

go through a vendor? Something that could be

more generic, something that would allow us

to work. I remember this started off as an

Acrobat system where we had an Acrobat

backbone, an Acrobat document. We've gone to

- and it's much more difficult. Can we get

to something that works in XML that anybody

can pick up and work with it easily because

this is getting to a point where you're going

to want us to do this and I think as soon as

the FDA mandates it, EMEA is going to mandate

it, HAN's going to mandate it, everybody that

utilizes the ECTD format is going to fall in

line and we're going to have electronic

submissions as the requirement.

I applaud this group coming

together, at least giving us an opportunity

to listen, but I think there's a lot more to

(202) 234-4433 WASHINGTON, D.C. 20005-3701
be had here. Taking it all the way and

getting rid of paper is a great idea, I just

hope we can do it at small companies because

when we start off small we have big hopes.

Thank you.

DR. WOODCOCK: Can you stay at the

microphone? Does anyone on the panel have

questions? No? All right, thank you very

much. Other members of the public? If you

can come up to the microphone and identify


MR. BRENNAN: Michael Brennan from

Centocor, J&J. There's been a theme that's

come through a couple of the presentations

that first started with Al's talk early this

morning about do we have the right people, do

we have the right skills. I think most of

the responses, most of the discussion really

focused on the technical aspects of, you know

the staring at a computer and hitting the

right keys. But I think what we have to

recognize is the marriage between people that

have the content knowledge, the regulatory

knowledge merging with the information

processing, it's a very transformational

change that needs to happen within our


Randy, you challenged in terms of

what's the impact on our business processes

and if you're going to move to fully

electronic, we need to have that marriage

within our companies. We need to have people

to focus on the role of being an information

manager. Not an IT manager, but a true

information manager.

DR. WOODCOCK: The younger

generation provides us some hope. Couldn't

resist one editorial comment. Any questions

from the panel? Thank you. Are there any

further comments from the audience? Yes.

MR. KUBECK: Hi, I'm Bill Kubeck.

I'm from Steeple Laboratories. We are a

small pharma company, second tier. We just

migrated from third tier to second tier.

(202) 234-4433 WASHINGTON, D.C. 20005-3701
Very proud of that. We are 150 years old and

we're really good at paper.

And a number of comments that were

made in terms of the business case on going

from paper to electronic. Paper, I get the

question all the time. We've got the papers.

We can just continue to do it. Migrating to

electronic submissions is - there is a

significant investment in terms of the IT

infrastructure. The total cost of ownership

can be quite great, three to ten times the

original amount in terms of acquiring the

actual technology to do so, in terms of just

corporate personnel, in terms of training.

Steeple Laboratories are great

supporters of standards. We're involved in

the creation of standards and we want to go

fully electronic. But we need to also

understand, we file, we have branded

pharmaceuticals, we have generics, we have

OTCs, we have cosmetics, and by the way, we

register our products in a hundred different

countries with all these different standards.

The issue for us is how can we effectively

manage our submission information to all

these different regulatory agencies and all

these different standards. Now if we just

even look within the United States there are

a multitude of standards depending upon which

division we're talking to. Question to the

panel is when can we expect to have an

integrated standard for submissions?

DR. WOODCOCK: Well, under a Part

15 hearing you're not permitted to ask the

panel questions. However, I will respond to

that question. The FDA has formed a cross-

agency bioinformatics board and we are

engaged in this very moment in developing the

plans for converging our standards across all

the different FDA areas. And that does not

mean, as we've just heard from you, that the

hundred regulatory agencies around the world

are going to fall in line with us, but if we

have a strong and we work in a public-private

(202) 234-4433 WASHINGTON, D.C. 20005-3701
partnership I think towards consensus

standards we can go far that way. But your

point is well taken. We recognize that.

We've heard that actually from many other

speakers today.

MR. KUBECK: One last point in

terms of migration. We are not ECTD-

compliant. We're making the full transition.

Now comes the issue in terms of getting the

public awareness of RPS. And I would like

your thoughts in terms of that transition. I

can't go, you know again, for my senior

management to embrace going to electronic

submissions the question they're asking us,

ECTD right? And that's the question mark.

DR. WOODCOCK: I think as people -

please submit to the docket issues you have

around those particular timing and

communication issues that would be very

helpful. Did anyone on the panel have

further questions on these? Thank you very

much. Yes, from another volunteer.

MS. AZEVEDO: Good afternoon Dr.

Woodcock and the panel.

DR. WOODCOCK: Can you identify


MS. AZEVEDO: My name is

Antoinette Azevedo. I'm President of e-

Submissions Solutions, a professional

services firm assisting companies in

selecting and implementing and validating

document management and publishing. I also

provide submission publishing services.

Point one. I tried to comment to

the notice that was sent out and the web link

didn't work.

DR. WOODCOCK: That's I think been

addressed. We're sorry about that.

MS. AZEVEDO: Okay. Secondly, I

work with companies that are sometimes prior

to the IND phase. And I've worked with them

for the last six years. I've been in

business about six years. One of the factors

that prevents companies from starting INDs in

(202) 234-4433 WASHINGTON, D.C. 20005-3701
the ECTD phase is the fear that if they start

ECTD, always ECTD and that will prevent them

from licensing out products because the

business model is to be virtual license out

or sell out and they're afraid that that will

be an impediment to doing a deal with a

partner. So that's one interesting factor.

So if ECTD were mandatory throughout the

lifecycle, this would remove an impediment to

starting in the IND phase.

The other issue is a training

factor. I, because I provide submission

publishing services, I have a boutique

operation and oftentimes go into the customer

environment and work on the systems that the

customer has already purchased and validated.

And the customers can't find people to

operate their own systems so they bring in


Secondly, in my practice I have a

need for consultants to come and help me

execute my projects and it's impossible to

find people of the right skills to perform

the functions. And the skills are not just

the regulatory skills which was - is really

critical, it's the computer skills. It's

also the knowledge of the applications, how

to use Microsoft Word to go to PDF, how to

troubleshoot problem PDFs. Recently we used

Global Submit to validate a submission I was

working on, and this is a vendor's system

that has been used to produce many ECTDs

submitted to the FDA including pilots. And

we found lots of issues reported by Global

Submit that were surprises, including issues

with the standards themselves. That is,

there are issues with the ECTD, DTD and with

various lists and so on that caused false

positive issues to be reported by the

validations. This whole set of things are

big impediments, but if there were mandatory

requirements with an appropriate timeframe

and outreach by the agency to especially the

small, the tier three and tier two companies,

(202) 234-4433 WASHINGTON, D.C. 20005-3701
really critical to making this a success.

DR. WOODCOCK: Thank you. Any

questions? No. Any further comments? I

think we may have time for one last comment

here. Yes.

MS. PALLA: I'm Madeline Palla.

I'm with the Animal Health Institute. And I

just wanted to comment that the animal health

industry is often slightly CDER and CBER in

electronic requirements and we monitor these

issues closely and it's going to be very

complicated in the animal industry to know

and conceptually to give you an idea.

Listening to the speaker today talk about you

know having a notepad laptop to, you know. A

veterinarian out of the office is not going

to be able to utilize that.

And I just wanted to, I cannot

speak necessarily to the costs personally as

I am with a trade association and not a

particular sponsor company, but a lot of our

animal health companies are divisions of

larger human pharmaceutical companies and so

it may not be as great of an impact to those

companies that are already implementing these

requirements in other areas of the

corporation. But for some of our animal

health companies that are solely animal

health it's going to have a large impact.

And we do plan to answer the other questions

in the notice in our comments that we submit.

Thank you.

DR. WOODCOCK: Thank you. All

right. Is there one more person who wants to

contribute? All right. We have a few more

minutes here.

MS. VATLEIT: Thank you. Hi, I'm

Stephanie Vatleit. I'm with Animal Drug

Alliance and I would mirror in large part the

AHI position. We are cautiously optimistic

about electronic submissions, but we sense

some real hesitancy on the part of CBN to

work with us on those submissions and a

hesitancy on the part of our member companies

(202) 234-4433 WASHINGTON, D.C. 20005-3701
to work through the business case. We've

heard a lot about that today and it is really

important to the bottom line as there are

lower margins on the animal health side as

you might expect.

And we would also like to

acknowledge that it's not only a burden to

get the software and hardware necessary for

making electronic submissions on the part of

the sponsors, but also on the part of the

centers. And I think that's a real concern.

DR. WOODCOCK: Thank you.

Questions? No. All right. I would like to

reiterate first that the comment period

remains open. We hope those of you who have

benefitted today from all the comments, it

should inform your comments to the docket.

We hope for a robust set of comments that

will allow us, NIH and the FDA to move

forward in our various venues. I want to

thank the panel very much. I'd like to thank

Paula McKeever for putting this meeting

together. She did a fantastic job. I think

we learned a tremendous amount. And I'd like

to thank all of you who stayed till the end

for your attention. Thank you very much and

this hearing is closed.

(Whereupon, the foregoing matter

went off the record at 4:56 p.m.)

(202) 234-4433 WASHINGTON, D.C. 20005-3701