NICE

Scientific advice &

Diagnostics assessment
programme
How to generate evidence to
support value claims for
diagnostics
Dr Grace Jennings and Dr Sarah Byron September 23, 2014
 What is NICE?
An independent institute that identifies how to:

•prevent, diagnose and treat disease and ill health in most

effective ways

•reduce inequalities and variation

•ensure quality and value for money for the NHS

NICE’s Procedural Principles

Accountability
Accountability
for
for
reasonableness
reasonableness
A NICE Process

Independent Evidence
review submissions

Input from Decision Stakeholder

topic experts Committee Perspectives

Public Consultation

Decision
 Scarce resources

DIFFICULT CHOICES FOR

DECISION MAKERS

HTA
Health Technology Assessment (HTA)

Evidence

Product Market

Policy-making
Evidence-based way of guiding the efficient
allocation of health care resources
Benefits of engaging with NICE
How NICE makes decisions
Two key questions asked by NICE

• How well does the technology work compared to

established practice in the National Health Service (NHS)?

• How much does this course of action cost compared to

established practice in the NHS?
Decision-making at NICE
Value Proposition

Impact on Incremental
health system benefit for
resources patients

Value varies depending on your perspective

Perspective

Understand the perspective of your decision maker and know what

question they want to answer

NICE takes the perspective of the

National Health Service (NHS) and
Personal Social Services (PSS)
Defining the clinical question:
Components

• Population
• Intervention
PICO
• Comparator
• Outcomes
Putting your case together

Population Usually the patients indicated in the

marketing authorisation

Intervention Technology to be appraised

Comparators Established practice in the NHS

Outcomes Outcomes which have an impact on:

- survival
- health related quality of life (HRQoL)
Navigating NICE
Why seek scientific advice?

Product developers may be interested in ensuring their clinical

development studies and other plans generate the evidence which
is relevant to NICE

Increase the likelihood that clinical trials and other research

activities undertaken meet NICE evidence requirements
Typical issues raised for advice

Clinical Trial Programme

• Study population
• Comparators
• Acceptability of endpoints, surrogate endpoints
• Trial design
• Appropriateness of health-related quality of life and other Patient
Reported Outcomes
• Positioning in the clinical pathway

Economic Evaluation
• Plans for using specific economic models
• Sources of data , observational studies, analyses
 Additional issues raised in previous
diagnostics advice projects

• Study design
• Diagnostic cohort design, case-control study, cluster randomisation trial
• Sensitivity and specificity
• Diagnostic accuracy
• Stratification by risk factor
• Clinician blinding
• Outcomes relevant to NICE
• Role of new diagnostic in existing treatment pathway
• Assessment of cost-effectiveness
• Model structure and input assumptions
Key stages of advice processes
Advice for developers of screening tests

• Screening not part of NICE remit

• Screening and diagnostic tests often similar
• Diagnostic test may be used as screening test in treatment pathway
• National Screening Centre
• Part of Public Health England
• Areas of screening needs explored
• Models developed by external academic group
• New initiative – advice from NICE Scientific Advice in conjunction
with NSC for screening tests
Questions?
Programmes at NICE

Medical Diagnostics Technology

Technologies Assessment Appraisals
Evaluation Programme (TA)
Programme (DAP)
(MTEP)
Topic Selection
STEP 1
The Company submits a notification form to Medical Technologies
Evaluation Programme that details:
•Product description
• Patient population
• Current management and comparator(s)
• Claimed patient benefit
• Claimed healthcare system benefit
• Claimed sustainability benefit
• Costs
• Patient safety
Notifying a product to NICE
STEP 2.

NICE produces a briefing note for MTAC which decides whether

the technology is suitable for the evaluation at NICE

STEP 3.

If selected, MTAC routes the technology to the appropriate

Programme.
Process takes ~10 weeks
Questions about the process and eligibility?
Contact MTEP:
medtech@nice.org.uk
MTEP vs DAP

DAP and MTEP encourage further research into promising

technologies
Diagnostics Assessment Programme
• Specialist programme to undertake complex
assessments of diagnostic technologies
• Decision making by independent Diagnostics Advisory
Committee
• Assessment of single or multiple technologies
• No formal manufacturer submission required
• Systematic review of evidence and modelling to
estimate outcome benefits and cost effectiveness is
undertaken as part of the assessment
Overview of assessment process
Two key questions asked by NICE

• How well does the technology work compared to

established practice in the National Health Service (NHS)?

• How much does this course of action cost compared to

established practice in the NHS?
Scoping Single technology notified
and referred to DAP

DAP technical lead

Draft scope
•Diagnostic pathway
•Care pathway Scoping workshop
•Alternative Registered stakeholders
technologies
•Relevant population(s)
•Costs Revised scope
•Outcomes
•Potential equality ASG meeting
issues Specialist Committee
•Potential members
implementation barriers Standing DAC member
Final scope

Assessment of single or multiple technologies

 1. How is the
condition
managed in the
NHS? 2. Where does my
product fit in the
care pathway?

3. What does my
product deliver?
Understanding benefits: diagnostics

False False
positive? negative?
Understanding benefits: diagnostics

Patient benefits rarely arise from the diagnostic directly

– they come mainly from treatments informed by the
diagnostic

•The treatment pathway or the range of pathways must be

understood for the value of the diagnostic to be assessed
•Test side effects should be included
Assessment
What data do you need?
Regulator
• Product safety: laboratory
testing with clinical trial data
for devices with greatest risk
• Evidence of efficacy: does the
device meet its intended
purpose? (not necessarily
from comparative studies)
HTA
•Evidence on clinical effectiveness (compared
to established practice):
trials, evidence synthesis
•Evidence on cost effectiveness:
trials, modelling
•Evidence on relative safety/adverse events
Trueman P et al 2011
 Evidence considerations
• The ideal evidence would be a good quality ‘end-to-end’ study –
follows patients from testing, through treatment, to final outcomes
• Typically not available for diagnostics
• Search for data on test accuracy, direct outcomes from the test,
indirect health outcomes from the test result, and costs
• Identified evidence can then be combined through a linked
evidence approach

Impact on
Diagnostic Impact on
treatment
accuracy outcomes
decisions
Evidence hierarchy
Study design

• Outcomes: patient focussed outcomes are particularly important,

as opposed to intermediate or surrogate outcomes
• e.g. a reduction in tumour size will be given less weight than
evidence about clinical benefit such as improved survival or
quality of life
• Size: Studies with larger numbers of patients will usually be
preferred as estimates of benefits and harms will be more accurate
• Duration: Studies should have sufficient follow up to capture final
outcomes where possible
• e.g. very important for prognostic tests
Diagnostic tests: Outcomes data
Ideally comparative ‘end-to-
end’ clinical studies including Not possible
the test and subsequent
treatments should be
conducted Identify studies on the
effectiveness of those
subsequent treatments

Test side effects should be Use a systematic approach to

included identifying relevant studies
Diagnostic tests: Outcomes data

• Measurements of test accuracy are necessary:

Condition as determined by
“Gold Standard”

Condition Condition
positive negative

Test
True False
outcome PPV
positive Positive Positive
Test
outcome Test
False True
outcome NPV
negative negative Negative

Sensitivity Specificity
Diagnostic tests: Outcomes data

Cut off points

 Example:
SonoVue (sulphur hexafluoride microbubbles)
Contrast agent for contrast-enhanced ultrasound imaging
of the liver

Characterising Detection of Characterising

incidentally detected potential liver focal liver lesions
focal liver lesions metastases (cirrhosis)
No end-to-end studies available
High quality accuracy data – SonoVue vs CT and MRI
Relevant evidence on care pathway and outcomes
 Characterising
incidentally detected Cost Adoption
focal liver lesions effective recommendation

Adoption
Detection of recommendations
potential liver Slightly where CT and
metastases less cost MRI not
effective appropriate
than CT +
Characterising and MRI Research
focal liver lesions
recommendations
(cirrhosis)
to explore
potential broader
applicability
Decision making
Recommendations

• Adoption recommendations

• Research recommendations

• Not recommended
Guidance development

• Decision making in presence of uncertainty

• Public consultation can change decision making
• Clarity in recommendations on indication
o Rule-in / rule-out / diagnosis / monitoring
o Setting
 Supported by evidence, minimise risk of indication creep and
inappropriate use of tests that may lead to misdiagnosis
 Cost-effective use of NHS resources
• ‘Committee considerations’ describe uncertainties and
rationale behind decision-making.
Examples of recommendations

Diagnostic Guidance (DG5)

SonoVue (sulphur hexafluoride microbubbles) –

contrast agent for contrast-enhanced ultrasound imaging of the liver

Contrast-enhanced ultrasound with SonoVue is recommended for

characterising incidentally detected focal liver lesions in adults in whom an
unenhanced ultrasound scan is inconclusive. An unenhanced ultrasound
scan in which a focal liver lesion is detected, but not characterised, is
defined as inconclusive.
Recommendations

• Adoption recommendations
• Implementation support
• Health Technologies Adoption Programme

• Research recommendations
• NICE research commissioning

• Not recommended
NICE: Companion diagnostics
Technology Appraisal
Programme
Appraisals of new and existing
medicines and treatments
Topics from department of health
Assesses single or multiple
technologies
STA - Manufacturer submission
MTA – Manufacturer submission and
evidence gathered by academic group
Evidence assessed by independent
external group
STA – 35 weeks
MTA – 62 weeks
Diagnostics Assessment
Programme
Specialist programme to undertake
complex assessments of diagnostic
technologies
Topics from manufacturers or clinical
sponsors
Assesses single or multiple
technologies
No formal manufacturer submission
Structured information request
Evidence gathered by academic group
Evidence assessed by independent
external group
62 weeks
NICE: Companion diagnostics

• Alternative technologies
• Timing is an important factor
o Appraisal programme coordinates publication of guidance with market
authorisation of drug
o CDx may be developed simultaneously with drug
o CDx can come to market years after drug approval or may already be
available
o Rapid development of CDx and supporting technologies
Technology Appraisals: Companion diagnostics

• In January 2013, NICE published update to methods

guide
• Costs of CDx testing incorporated into evaluation of clinical and
cost effectiveness
• Sensitivity analysis to assess impact of CDx cost on cost
effectiveness of pharmaceutical
• Diagnostic accuracy can be examined and incorporated in cost
effectiveness analysis
• Potential issues of alternative CDx can be highlighted in
guidance without assessment of evidence
Example of CDx in TA programme

• TA 208 Trastuzumab for HER2-positive metastatic gastric cancer

• MA included testing with fluorescence in situ hybridisation (FISH) then revised to include silver in
situ hybridisation (SISH)
• Timing of MA meant that only FISH was included in NICE appraisal
• Trial used parallel testing strategy
• Sequential testing strategy in manufacturer’s model
• Only ICH2 positive received FISH test
• ERG scenario analyses for both sequential and parallel testing strategies
• Sequential ICER £66,982 per QALY
• Parallel ICER £71,637 per QALY due to increased incremental costs
• Committee concluded that sequential testing was most appropriate for people with metastatic
gastric cancer
Example of CDx in DAP programme
• EGFR-TK mutation testing in adults with locally advanced or
metastatic non-small-cell lung cancer
• Evidence
o Two tests used in clinical trials
o Three tests had accuracy data
o Linked to clinical trial data
o Remaining tests had no trial or accuracy data
o Included a survey of labs providing EGFR-TK testing
o test characteristics and costs
o Data from an EGFR-TK national external quality assurance
scheme study
EGFR testing - Recommendations
• 5 tests recommended but insufficient evidence to make
recommendations for others
• Key issues:
• Test validation
• Competent execution
• Participation in external quality assurance scheme
• Research recommendation
• Studies comparing different EGFR-TK mutation methods that link
to patient outcomes
• Many assumptions in assessment

Diagnostics guidance (http://www.nice.org.uk/dg9)

Summary
Ways of assessing alternative CDx:
• Only assess CDx used in clinical trial
• include general commentary on use of alternative CDx tests
• Where appropriate, undertake sensitivity analysis on
diagnostic accuracy and cost to understand the importance
of CDx in relation to patient outcome benefits and cost
effectiveness
 Technology Appraisal programme
• Assess each alternative CDx / treatment package separately
• Separate clinical and cost effectiveness analyses
 Diagnostics Assessment programme
Health Technology Adoption
Programme (HTAP)
Main objectives:

•Increase the uptake of NICE recommended technologies

•Establish an agreed approach for NICE to develop effective

partnerships with Academic Health Science Networks
(AHSN)

•To support the expansion of the Medical Technologies

Industry in the UK by providing adoption advice to suppliers
 Two approaches to adoption in the NHS:

• Site demonstrator projects for commonly used

technologies

• Adoption projects for more complex, less commonly used

technologies
Produce adoption support packs to accompany guidance produced
in MTEP, DAP, TA and Quality Standards (QS).

Please contact Jae Long (jae.long@nice.org.uk), Assistant Project

Manager for HTAP, for further details.

57
 Key contacts

• NICE Scientific Advice

• Richard Chivers (richard.chivers@nice.org.uk)
• http://www.nice.org.uk/about/What-we-do/Scientific-advice
• NICE/Medilinks seminar http://www.medilinkem.com/events/events-calendar/2014/10/21/nice-medtech-workshop
• NICE DAP
• Sarah Byron (sarah.byron@nice.org.uk)
• http://www.nice.org.uk/About/What-we-do/Our-Programmes/NICE-guidance/NICE-diagnostics-guidance
• NICE Medical Technologies Evaluation Programme
• Lee Dobson (lee.dobson@nice.org.uk)
• http://www.nice.org.uk/About/What-we-do/Our-Programmes/NICE-guidance/NICE-medical-technologies-evaluation-
programme
• Health Technologies Adoption Programme
• Jae Long (jae.long@nice.org.uk)
• http://www.nice.org.uk/About/What-we-do/Into-practice/Health-Technologies-Adoption-Programme
Any questions?