You are on page 1of 80

Unit One

Prognosis and Prognosis Research

Destaye SA

December 2023
Objectives
At the end of this unit, you will be able to:

• Define prognosis

• Describe the key motives and aims of prognosis

• Explain approaches in prognostication

• Describe the characteristics of prognosis research

• Elaborate the architecture of prognostic research

• Formulate prognostic clinical research questions


February 27, 2024 Prognosis Research 2
Contents

• Introduction
• Prognosis and prognostication
• Motives and aims of prognosis

• Approaches in prognostication

• Prognostic research

• Characteristics of prognosis research

• Architecture of prognostic research


Introduction

February 27, 2024 Prognosis Research 4


Research in Clinical Epidemiology

Research questions arise from clinical practice


 Observations
 Challenges encountered in practice
 Clinicians may notice:
• Patterns o Causes?
• Variations Lead to questions o Risk factors?
• Uncertainties in outcomes o Optimal intervention?

Practice-driven inquiry
• Often driven by the need to improve patient care and outcomes.
• Clinicians seek evidence-based answers to questions that directly impact their
ability to diagnose, treat, and prevent diseases in their patient
Provide a practically relevant answer that serves the practice
• Answers are expected to be directly applicable and relevant to clinical practice
• Findings should inform:
• Decision-making
• Clinical guidelines
• Healthcare policies Ultimately Improvements in patient care

Translational:
• Translates scientific discoveries into practical applications for patient care

• Bridges the gap between research findings and their implementation in


real-world clinical settings

February 27, 2024 Prognosis Research 6


Iterative process:
Research

Inspire to explore and


understand underlying New insight
mechanism

Clinical observation Change in clinical practice


• Patient-centered
• Focuses on understanding and addressing health issues that directly
impact individual patients
• Research aims to enhance the well-being and outcomes of patients
• Research relevant to patient care

• To carry out research relevant to patient care:


• Recognize clinical problems in daily practice
• Design and understand clinical research
• Search for the evidence to deal with these

DE PTh Model
February 27, 2024 Prognosis Research 8
What is prognosis?
• Prognosis:
• All around us:
• Weather forecasts
• Corporate finance projections
• Has medical connotation

• Simply means foreseeing, predicting, or estimating the


probability of future conditions

February 27, 2024 Prognosis Research 9


• Prognosis (in health):
• Refers to the prediction of the likely outcome or course of
a disease.

• Probability or risk of developing a particular state of


health over a specific time

• Does not address the present, but rather the future


• “Future diagnosis” - “diagnosis yet to be made”

February 27, 2024 Prognosis Research 10


• One component of the DEPTh model

• Set Diagnosis (Etiology) Prognostication


• Aims to predict future occurrence of an
outcomes based on the patient’s profile
• Death
• Recurrence
• Complication
• Pain
• QoL

February 27, 2024 Prognosis Research 11


• Prognosis may refer to all elements of future health
• Direct manifestations of disease such as mortality, pain
• Direct physical or psychological sequelae
• Adverse effects of treatment
• Treatment response or failure
• Limitations in psychosocial or societal functioning
• Disease recurrence
• Future need for invasive diagnostic procedures
• Other concerns
February 27, 2024 Prognosis Research 12
• Prognostication is crucial in the decision to initiate or refrain from
therapeutic interventions.
• Accurate prognostic knowledge is of critical importance to both patients and
clinicians.

 For clinicians: • For patients:


• Prognostication is a core activity
 Patients have a natural interest in their
future health
• Guides subsequent medical actions
 It is a reflection of a basic need for
o Monitoring the course of the disease certainty
o Planning future interventions
 Enables them to anticipate the future
o Deciding to refrain from interventions
 Supports them to make informed plans

February 27, 2024 Prognosis Research 13


Key aspects of prognostication
• Outcome prediction:
• Prognostication involves predicting outcomes such as the likelihood of:
• Recovery
• Survival
• Disease progression
• Response to treatment

• Risk assessment:
• It includes assessing the risk of specific events or complications related
to the health condition.
• This could involve estimating the risk of recurrence, complications, or
adverse effects.
February 27, 2024 Prognosis Research 14
• Time frame:
• Prognostication
• Can involve predictions for immediate outcomes or extend over
months or years.

• Individualized assessment:
• Prognostication is often individualized, taking into account the unique
characteristics of the patient, including:
o Age
o Overall health
o Comorbidities
o Response to treatment

February 27, 2024 Prognosis Research 15


• Clinical judgment:
• Prognostication:
• Relies on clinical judgment, expertise, and experience
• Involves synthesizing available evidence, clinical
observations, and relevant data to make predictions

• Communication:
• Clinicians need to convey predictions to patients to facilitate
shared decision-making

February 27, 2024 Prognosis Research 16


• Treatment planning:
• Prognostication informs treatment planning by helping healthcare
providers determine appropriate interventions

• Ethical considerations:
• Ethical considerations are often part of the prognostication process,
especially when making decisions about:
o Treatment intensity
o End-of-life care
o Respecting the patient's autonomy and preferences

February 27, 2024 Prognosis Research 17


The format of prognoses
• Prognosis is probabilistic ← future cannot be predicted with 100%
certainty
• Formulated reflecting uncertainty- risks or probabilities
• e.g.: short-term mortality in a patient with a recent diagnosis of severe HF
may be expressed as likely, uncertain, or unlikely

• Prognosis is expressed in exact quantitative terms, such as a


period-specific absolute risk.
• Clinically relevant prognoses are to be expressed as absolute risks

• E.g: the 10-year survival rate for a woman between 50 and 70 years of age
with node-negative breast cancer with a tumor diameter of less than 10
millimeters is 93%
February 27, 2024 Prognosis Research 18
Motive and aim of prognosis
• Rooted in understanding and anticipating the future course of an individual's
health condition.

February 27, 2024 Prognosis Research 19


Key motives and aims of prognosis

1. Informed decision-making:

• To provide individuals and families with information to


make informed decisions about their healthcare.

• Helps individuals and healthcare providers to be aware of


the likely trajectory of a disease to plan appropriate
interventions and make decisions about treatment
options.
2. Treatment planning

• Prognosis is crucial:

• For healthcare providers in planning and tailoring


treatments

• To determine the most appropriate interventions, whether it


• Be aggressive treatment for a potentially curable condition,
or
• Palliative care for a condition with a poor prognosis.

• To adjust treatment plans based on expected course of the


disease
3. Resource allocation:
• Prognosis assists in the allocation of healthcare resources.

• It helps healthcare systems:

• Plan for the future demand for services

• Allocate resources effectively

• Optimize the use of:


medical facilities
personnel, and
equipment

February 27, 2024 Prognosis Research 22


4. Patient counseling:
• Prognosis:
• Promoting open communication to prepare individuals and
families for the future

• Allows healthcare providers to counsel patients and their


families about what to expect in the future.
• Including:
• Discussing potential challenges
• Quality of life considerations
• Addressing emotional and psychological aspects
5. Research and clinical trials
• Aids in the design and recruitment of clinical trials and studies
aimed at:
• Improving treatments
• Developing new therapies
• Advancing medical knowledge

6. Public health planning:


• Helps public health officials and policymakers:
o Anticipate trends
o Plan for healthcare infrastructure
o Implement preventive measures at the population level

February 27, 2024 Prognosis Research 24


Ethical decision-making:
• Prognosis:
• Is essential in ethical decision-making, especially in situations
where the likelihood of benefit from medical interventions is
low.

• Guides discussions around


• End-of-life care
• Advance care planning and
• Allocation of limited resources in a manner that respects the
values and preferences of the patient

February 27, 2024 Prognosis Research 25


Approaches to prognostication
 Mechanistic and pathophysiologic insight
• Fits the educational experience

• Rarely enables to effectively differentiate patients with high


to low risk of developing a certain outcome
• Disease outcomes are determined by multiple, interrelated,
complex, and largely unknown biological factors and
processes with high variability among subjects

• Knowledge about underlying mechanisms is not always


available and if available is often difficult to measure.
• Accurate prognostic predictions are obtained by combining several
easy-to-assess clinical and non-clinical characteristics of the
patient
• Not necessarily causally linked to the course of the disease

• E.g. hip fracture can be accurately predicted from age, gender, height,
use of a walking aid, cigarette smoking, and body weight

• These predictors likely correlate with parameters involved in


the causal mechanism underlying fracture risk, i.e.:
• Low bone density
• Impaired bone quality
• Impact on the hip bone from a fall
• Postural instability
 Clinical experience
• A frequently used source of prognostic knowledge
• Paramount importance in prognostication in daily practice

• Suppose that a cardiologist observes that women diagnosed with HF return to


the hospital less frequently than men
• This observation may result from:
• True worse prognosis in men than in women
• Worse survival in women with HF  fewer readmissions
• Women with similar symptoms of worsening HF are less likely to be
referred to a hospital
• Observation may be wrong!

• Prognostic research is useful to confirm/refute and quantify prognostic associations.


 Empirical prognostic research
• Systematic investigation of factors that can predict the course,
outcome, or future events related to a specific condition or disease.

• Uses of explicit prediction models containing multiple prognostic


determinants
• Representing values of predictors and their quantitative
relationship to a certain prognostically relevant outcome.

• Guide decision-making based on more objectively estimated


probabilities as a supplement to any other relevant information,
including clinical experience and pathophysiologic knowledge
• In practice, these approaches are often used implicitly and even
simultaneously.

• It is unlikely that a physician estimates a prognosis based on a prediction


model only.

• The aim of a prediction model in any medical field is not to take over the
job of the physician.

• The intention is to guide physicians’ decision-making based on more


objectively estimated probabilities as a supplement to any other relevant
information, including clinical experience and pathophysiologic
knowledge.

February 27, 2024 Prognosis Research 30


Prognostic research
What is prognosis research?
• Prognosis research seeks to understand and improve future
outcomes in people with a given disease or health condition.

February 27, 2024 Prognosis Research 32


Why is prognosis research important?
• More people now live with diseases and conditions that impair health than
at any other time in history

• Prognosis research provides crucial evidence for translating findings from


the laboratory to humans, and from clinical research to clinical practice

• Hippocrates included prognosis as a principal concept of medicine.


Nevertheless, principles and methods of prognostic research have received
limited attention, especially compared with therapeutic and aetiological
research.

February 27, 2024 Prognosis Research 33


Characteristics of prognosis research

February 27, 2024 Prognosis Research 34


• Research questions emerge from clinical practice and the
answers must serve that practice

• Main objective is to provide quantitative knowledge about the


occurrence of an outcome in a clinically relevant domain in a
predefined period as a function of multiple predictors through:
o Assessing best combination of potential determinants
o Evaluating added value of a new prognostic determinant
o Comparing predictive accuracy of new markers

February 27, 2024 Prognosis Research 35


• Prognosis research is hierarchical
• A logical hierarchy in all available prognostic determinants exists
based on everyday practice.
• First estimate prognosis based on a combination of a limited
number of:
• Non-patient-burdening
• Easily measurable variables: history taking, comorbidity
• Physical examination

• Before using more cumbersome or costly prognostic markers, the


additional test has to show added predictive value beyond the more
easily obtained prognostic predictors.
• -----prognostic added value
February 27, 2024 Prognosis Research 36
• Aims to assist clinicians in the prediction of the future occurrence of a
certain health outcome
• To guide patient management.
• To learn about the future (for practitioner and patient)

• It is descriptive (i.e., non-causal)


• Not aimed at explaining the outcome
• Confounding is not relevant

February 27, 2024 Prognosis Research 37


• The domain usually comprises people at risk of developing an outcome of
interest, defined by the presence of a particular condition (e.g., illness,
undergoing surgery, or being pregnant) presented to a certain setting.

• Cohort study is the best design


• Enables optimal measurement of predictors and outcomes

• It inherently longitudinal
• Involve time

February 27, 2024 Prognosis Research 38


• It is a multivariable process

• A single predictor rarely gives an adequate estimate of prognosis


• Variability: patients, aetiology, presentation, Rx of diseases, etc
• Clinicians (implicitly or explicitly) use multiple predictors to estimate a patient’s
prognosis.

 Accurate prognostic predictions are obtained by combining several easy-to-assess


clinical and nonclinical characteristics

 Comprises :
• Nonclinical: age, gender
• Clinical: diagnosis, symptoms, signs, etiology, test results

 Rarely adequately estimated by a single prognostic predictor


February 27, 2024 Prognosis Research 39
• Multiple predictors used to estimate a patient’s prognosis
• A multivariable approach in design and analysis is important:
• To determine the important predictors

• To provide outcome probabilities for different combinations of predictors

• To provide tools to estimate such probabilities

• To investigate whether specific prognostic factors or markers (more


invasive or costly to measure), have worthwhile added predictive value
beyond cheap or simply obtained

February 27, 2024 Prognosis Research 40


• It should mirror real-life situations

• Performed in and mimic routine clinical practice to increase the


likelihood that results can be translated into everyday practice

• Requires practical applicability


• Predictor measurements should be feasible in everyday practice
• Practical application of the resulting prognostic model may be
hampered
• E.g: the use of an extensive questionnaire to assess personality
trait neuroticism in the prognostication of depression

February 27, 2024 Prognosis Research 41


Architecture of prognostic research

• Three components (follow logically)


Theoretical design
o Design of the occurrence relation
Design of data collection
o Design of conceptual and operational collection of data to document the
empirical occurrence relationship in a study population

Design of data analysis


o Includes a description of the data and quantitative estimates of associations

February 27, 2024 Prognosis Research 42


A. Theoretical design
• Design of the occurrence relation

• Starts with a research question


• Outcome, determinants and domain
• Formulating appropriate questions is critical as it:
• Guides the theoretical design
• Ensures the study produces an answer that fits the needs of the investigator
• Eases the design of the occurrence relation is relatively easy
• Should be expressed as a question!

February 27, 2024 Prognosis Research 43


Occurrence relation
• Central to the theoretical design of a clinical epidemiologic study

• Has a major impact on the design of data collection and the design
of data analysis.

• Relates one or multiple determinants to an outcome

• The “true” nature and strength of the occurrence relation is documented


and quantitatively estimated using empirical data

February 27, 2024 Prognosis Research 44


• Occurrence relation of prognostic research is given by:
• A mathematical function quantifying the relationship between outcome and
determinants

• Standard set of elements: outcome, determinants (D)


Outcome + Determinant + Domain + Setting
• Descriptive occurrence relations

February 27, 2024 Prognosis Research 45


Outcome
Must be a real matter to patients: Could be:
• Remission of disease Dichotomous (typically)
• Survival • Yes/no of the event or disease
• Complications course of interest
• Pain
• QoL Continuous variables:
• Specific events • Tumor growth
• Death • Pain
• Complications • Quality of life
• May also be quantities
• Disease progression
• (changes in) pain
• QoL
February 27, 2024 Prognosis Research 46
Preferably measured without knowledge of the measured predictor values

Methods for outcome ascertainment should be clearly defined


• Blinding for the studied predictors
• Duration of follow-up (if applicable)

Preferably not to be a proxy or intermediate


• Unless a clear relationship between such an intermediate outcome and outcomes more
relevant for patients has been established

o Joint space in patients with osteoarthritis of the knee (instead of pain, the
ability to walk, or quality of life)

o Use of CD4 count as a prognostic outcome (rather than the occurrence of


AIDS or even death) in HIV studies
February 27, 2024 Prognosis Research 47
Criteria defining outcome and measurement tools should be described in
detail

Must be assessed as accurately as possible


• With the best available methods to prevent misclassification

Time period during which outcome occurrence is measured should be


emphasized
• Predicting an outcome occurrence over a 3-month period typically yields
different predictors or different predictor–outcome associations than
prediction of the same outcome after 5 years.

• Prediction over a shorter period is commonly less problematic than


prediction over a longer time period.

February 27, 2024 Prognosis Research 48


Should be measured without knowledge of the value of the predictors under study
• Particularly if the outcome measurement requires observer interpretation.
• The presence of those determinants believed to be associated with the prognostic
outcome may influence the decision to consider the outcome to have occurred.
• Bias can cause under- or overestimation of the accuracy of predictors-
commonly overestimation
• Prevented by blinding assessors of outcome to values of prognostic
determinants

• Blinding is not necessary for mortality or other outcomes that can be measured without
misclassification

February 27, 2024 Prognosis Research 49


Determinants
 All potential and not necessarily causal correlates of the outcome
of interest

 Predictor choice should be based on both available literature


and a thorough understanding of clinical practice

 Can be obtained from:


• Patient history
• Physical examination
• Additional testing: imaging results, biological markers…
• Characteristics of the severity of the disease
• Interventions that the patient has received
February 27, 2024 Prognosis Research 50
Should be measured in and mimic routine clinical practice
• For translation of results to everyday practice

 Should be clearly defined and measured reproducibly


o To enhance the application of study results to daily practice
o This applies to:
Treatments that are studied as potential predictors
Predictors that require subjective interpretation: imaging test results, to
avoid studying the predictive ability of the observer rather than of the
predictors.

February 27, 2024 Prognosis Research 51


Should be measured using methods applicable to daily practice
• To enhance generalizability
• To prevent too optimistic predictive accuracy of predictors than can be
expected in real-life situations

• Specialized measurement of predictors is not necessarily a limitation of


prediction research (in its own).
• If substantially better predictions are obtained with specialized or more
elaborate measurements, this may call for such measurements in
everyday clinical practice.
• Feasibility plays an important role in choosing determinants to be included
in prognostic research.

February 27, 2024 Prognosis Research 52


All potential predictors should usually be measured and analyzed
in each subject of the study population.

• Chronological hierarchy in clinical practice


• History……physical examination….. Tests
• Measure subsequent predictors in each subject if the aim is to determine the
added predictive value.

Be aware of the inclusion of large numbers of determinants in


prognostic research

Consider proxy or surrogate predictors if the underlying predictor is


too cumbersome to measure
• E.g: the color of the newborn rather
February 27, 2024
than oxygen saturation is included in53
Prognosis Research
the Apgar score.
Domain
• Refers to the population or set of patients to whom the results can be
applied
• Individuals at risk of developing outcome of interest

• Usually defined by the presence of a particular condition


• illness
• Need for surgery
•…
• Patients with a zero or 100% probability of developing the outcome are
not part of the domain.
• Generalization of risk scores to these is invalid, irrelevant, and will not have any
bearing on patient management.

February 27, 2024 Prognosis Research 54


• Restricting domain to the setting of care
• Due to known differences in the predictive accuracy of determinants
across care

• Specifies the type of patients to whom the results can be applied.


• Guides patient selection for the research, but this selection is usually
further restricted for practical or other reasons.

February 27, 2024 Prognosis Research 55


• The study population should be representative of the domain

• Individuals with the same characteristics


• E.g: individuals with a particular symptom or sign at risk of developing a specific
health outcome.

• Recommended restricting domain definitions and thus study populations


to the setting of care (primary or secondary care) of interest
• Due to known differences in the predictive accuracy of determinants across
settings

• Often further restricted by logistical circumstances


• The necessity for patients to live near the research center or
• 27,The
February 2024
availability of their time to participate in the study
Prognosis Research 56
Which of the following are essential component of an
occurrence relation of a descriptive research
A. Outcome and determinant
B. Outcome and confounder
C. Outcome, determinant and confounder
D. Confounder and determinant

February 27, 2024 Prognosis Research 57


Example
• This research question combines
crucial elements:
What is the risk of hearing loss
from acute otitis media in children • Determinant
with severe malnutrition based on

the child’s age at first
presentation, sex, place of
residence, and general health status • Outcome
presented to a primary health •
center? • Domain:
• Setting:

February 27, 2024 Prognosis Research 58


B. Design of data collection
• Design of the conceptual and operational collection of data to document the
empirical occurrence relationship in a study population

• How data will be collected once the overall structure of the research is in
place

• Prognostic research is empirical research


 the theoretical occurrence relations are observed after analyzing empirical data
collected from individuals.

• The true nature of the occurrence relation is estimated from the


observations.

• The collection of data should capture the occurrence relation


February 27, 2024 Prognosis Research 59
• Several ways to collect data for a particular study
• Choice determined by
• The need to obtain a valid estimate of the nature of the occurrence
relation
• E.g, s the need to collect full confounder data in causal research
• Practical considerations
• E.g., restrictions in time or funding that limit the number of options for
collecting data.

• The need to find the truth, and thus the need to never compromise validity,
is an essential starting point.

• Yet, for a given level of validity there may still be several options for the
collection of data.

February 27, 2024 Prognosis Research 60


• The choices to be made for data collection include:
Time scale
• Prognostic process is inherently longitudinal  t > 0
• Time needed to observe the outcome occurrence may vary:
• Short: several hours
• e.g., early postoperative complications
• Long: days, weeks, months, or years

February 27, 2024 Prognosis Research 61


Nature of the study population
• Census:
• Everyone is studied
• All consecutive patients with a particular condition who are at risk for
developing the outcome of interest are included.

• Sample:
• Only a sample from the study base
• Sometimes a case-control design (and thus a sampling rather than a
census approach) is used in prognostic research

February 27, 2024 Prognosis Research 62


Sampling design
• Cohort study:
• Data collected prospectively allows for optimal measurement of
predictors and outcomes, as well as adequate (complete) follow-up

• Outcomes of prognostic studies are generally expressed in absolute


terms
• Most suitable design to address prognostic questions  cohort
study
• All ‘patients’ with a certain condition are followed for some time to
monitor the development of the outcome; this uses a census approach.

February 27, 2024 Prognosis Research 63


• Case-control study:
• Does not allow for an estimation of absolute risks of an outcome when cases
and controls are obtained from a source population of unknown size.

• Done for efficiency reasons,


• E.g: when the measurement of one or more of the prognostic determinants
is burdensome to patients or is expensive, or when the prognostic
outcome is rare.

• Suitable when nested case-control or case-cohort studies

• Case-cohort design a specific type of case-control study performed within a


cohort study is increasingly being used in prognostic research because of its
efficiency and because it yields absolute probabilities.
February 27, 2024 Prognosis Research 64
Observational or Experimental
Observational
• Determinants are studied without any experimental manipulation

• Almost all prognostic studies

• A well-defined group of patients with a certain condition followed for


a period to monitor outcome occurrence

• Researcher observes and measures parameters anticipated to be of


prognostic significance.
• Potential prognostic determinants are not influenced by the researcher
February 27, 2024 Prognosis Research 65
• Experimental
• When comparing the impact on a certain outcome of the use of two
prognostic risk scores by randomly allocating the two rules to
individual physicians or patients

• Subjects experimentally exposed to a particular determinant

• Prognostic studies involve experimentation


• E.g: when comparing the impact on a certain outcome of the use of
two prognostic risk scores by randomly allocating the two rules to
individual physicians or patients.

February 27, 2024 Prognosis Research 66


C. Design of data analysis
• Ideally, the design of data analysis follows naturally from the :
• Research question
• Form of the occurrence relation
• Type of data collected

• Includes a description of the data and quantitative estimates of associations


Data analysis: Aims
• To provide knowledge about which potential predictors
independently contribute to the outcome prediction, and to what
extent

• To develop and validate a multivariable prediction model or rule to


predict the outcome given the values of a combination of predictors.

• To concentrate on absolute risk estimates (incidence) of an outcome


given combinations of predictors and their values.

February 27, 2024 Prognosis Research 68


Data analysis: Description and Summarization
• Data description
o Characteristics of the study population
• Data are summarized
o Descriptive statistics
o Cross-tabulation
o Check for multicollinearity
o Univariable analysis
• Relationships between determinants and outcomes are quantified
using statistical methods.

February 27, 2024 Prognosis Research 69


Data analysis: Model building
Depends on outcome
Logistic regression model
• Dichotomous outcome
• Occurrence (yes/no) of an event within a specific
• Preferably a short period of time

• Most frequently encountered type of outcome


• E.g: occurrence of complication within 3 months
• Where ideally each included patient has been followed for at least this
period.

• Cumulative incidence of the dichotomous outcome at a certain time point is


predicted using predictors measured before t.
February 27, 2024 Prognosis Research 70
• Discriminative power: AUC
Proportional hazard model

• Time to event outcomes (survival)

• The second most common outcome in prognostic research

• The time to occurrence of the event can be predicted using the


Kaplan-Meier method or Cox proportional hazard modeling

• Possible to predict the absolute risk of a certain outcome within


multiple time frames
• e.g., 3 months, 6 months, 1 year, and 3 years

February 27, 2024 Prognosis Research 71


• Occurrence of a particular outcome event over a longer period
o Follow-up time may differ substantially between study
participants.

• Discriminative power: C- statistic


• Easily calculated and has the same interpretation as AUC

• Univariable analysis performed using the Kaplan-Meier method

• Construction of ROC curve is not straightforward as the outcomes of


the censored patients are unknown

February 27, 2024 Prognosis Research 72


Other: Less common outcomes in prognostic prediction studies

Linear regression
• Continuous outcome
• e.g.: tumor size
• Discriminatory power assessed: Explained variance (R2)

• Univariable and multivariable analyses: carried out using linear regression

Polytomous (nominal) outcomes

Ordinal outcomes
• Model performance measures:
• Discrimination (e.g., c-index),
• Calibration (plots), and
• (re)classification measures
• Internal validation:
• If the number of potential predictors in multivariable regression modeling
is much larger than the number of outcomes or subjects, any fitted model
will result in overly optimistic predictive accuracy.
• Bootstrapping and cross-validation techniques can quantify the model’s
potential for:
• Overfitting
• Optimism in estimated model performance measures
• A shrinkage factor to adjust for this optimism
Analysis: Estimating added value
• Prognostic factors, tests, and biomarkers differ in
• Predictive accuracy
• Invasiveness, and
• Cost

• Tests or markers that are burdensome and costly to collect and measure should not be
evaluated on their individual predictive abilities
• but rather on the incremental predictive value beyond established, and easier-to-obtain,
predictors.

• Measures of discrimination such as the c-statistic are not able to detect small
improvements in model performance when a new marker is added to a model that
already includes important predictors.

• Recently, new metrics that estimate the added value of predictors have been proposed.
• These quantify the extent to which an extended model (with the addition of a
subsequent predictor or marker) improves the classification of participants with
and without the outcome compared with the basic model without that predictor.

• E.g: Net reclassification improvement (NRI) does this by quantifying the


number of individuals that are correctly reclassified into clinically meaningful
higher or lower risk categories with the addition of a new predictor, using pre-
specified risk groups.

• Correct reclassifications are shifts to a higher risk category in those who develop
the prognostic outcome and shifts to a lower risk category in those who do not.

February 27, 2024 Prognosis Research 76


• Definition of these risk groups, however, is often arbitrary and differs across
studies, which may compromise comparisons of NRIs from different studies.

• To circumvent this problem, a version of the NRI that does not require
stratification of the population into risk groups may be used.

• Alternatively, the integrated discrimination improvement (IDI) may


be useful.

• In contrast to the NRI, the IDI does not require subjectively predefined risk
thresholds.

• The IDI is the estimated improvement in the average sensitivity of the basic
model with the addition of the new predictor minus the estimated decrease in
the mean specificity, summarized over all possible risk thresholds.
Analysis: Consideration
• Missing values:
• completely case analysis may lead to biased results.
• Imputation (preferably MI) of missing values often yields less biased results.

• Continuous predictors:
• Should not be turned into dichotomies and linearity should not be assumed.
• Simple predictor transformation can be implemented to detect and model
nonlinearity, increasing the predictive accuracy of the prediction model.

• Predictor selection in the multivariable modeling:


• Selection based on univariable analysis (single predictor–outcome
associations) is discouraged.
• Preferably, if needed, backward selection or a full model approach should be
used, depending on a priori knowledge.
Common pitfalls in current prognostic studies
• Studies on prognosis do not produce results that directly establish accurate
individualized prognoses in future patients in daily practice.
 Predictors measurement is not feasible in everyday practice
• practical application of the resulting prognostic model may be
hampered.
• E.g: the use of an extensive questionnaire to assess personality trait
neuroticism in the prognostication of depression
Fail to assess the added predictive value
• When the interest of prognostic research lies in the prognostic value of a
particular new marker
Fail to be validated
• internally or externally in another population reflecting the same domain

February 27, 2024 Prognosis Research 79


References

February 27, 2024 Prognosis Research 80

You might also like