You are on page 1of 6

European Journal of Obstetrics & Gynecology and

Reproductive Biology 137 (2008) 146151

Development of a nomogram to predict occurrence of preeclampsia

Stephanie Deis, Roman Rouzier *, Gilles Kayem, Christophe Masson, Bassam Haddad
Department of Obstetrics and Gynecology, Centre Hospitalier Intercommunal de Creteil and University of Paris 12, Creteil, France

Received 12 November 2006; received in revised form 11 March 2007; accepted 21 May 2007


Objective: The objective was to create a nomogram for the individual prediction of preeclampsia (PE).
Study design: In a prospective population-based study that included 4777 patients, PE occurred in 2.4%. Age, body mass index, parity,
previous preeclampsia (PPE), chronic hypertension, diastolic blood pressure (DBP), and proteinuria at first visit, and second trimester
ultrasonography and umbilical artery Doppler resistance index (UARI) data were used to develop and calibrate a nomogram based on a
multivariate logistic regression model.
Results: Based on multivariate analysis, nulliparity (P = 0.002), PPE (P = 0.004), DBP (P < 0.0001), biparietal diameter (P = 0.011), and
UARI (P = 0.08) were introduced into a nomogram. Based on these variables, the nomogram had good discrimination (area under the ROC
curve = 0.73, P < 0.01) and calibration (unreliability index = 5.2  10 4). This nomogram was validated by bootstrapping.
Conclusion: Our nomogram predicts the probability of preeclampsia. After validation in an independent population, this tool could be used to
plan a preventive trial.
# 2007 Elsevier Ireland Ltd. All rights reserved.

Keywords: Preeclampsia; Nomogram; Individual prediction; Ultrasonography

1. Introduction ultrasonography in a nomogram for predicting the individual

risk of preeclampsia.
Preeclampsia complicates 3% of pregnancies and is one
of the principal causes of maternal and perinatal morbidity
and mortality. Its prevention requires the availability of 2. Materials and methods
markers to predict or identify those women at high risk of
this complication. Several clinical tests have been proposed Between 2001 and 2003, 5011 patients were prospec-
for the prediction of preeclampsia. Most of these tests, tively included in the study at their first prenatal visit at the
however, have poor sensitivity and poor positive predictive Maternity University Creteil. Women who were preeclamp-
values [1,2]. At present, there is no single screening test that tic at the first visit were excluded. Women who did not
is considered reliable for predicting preeclampsia. As a deliver at our maternity were excluded from the analysis.
result, all studies on prevention have included women with The final cohort comprised 4777 patients. Patient char-
various risk factors for preeclampsia [3]. acteristics are reported in Table 1. Patients were followed
Assessment of the risk of preeclampsia is challenging as from their first prenatal visit in our center until delivery with
there is no available tool that can clearly separate high-risk a monthly visit. The examination included blood pressure
patients from low-risk patients. The aim of this study was to measurement and a urinary dipstick test to detect
combine first visit clinical markers and second trimester proteinuria. Second trimester ultrasonography was usually
performed at 2023 weeks gestation. Measurement of
biparietal diameter, abdominal perimeter, femur length, and
* Corresponding author at: Department of Obstetrics and Gynecology,
Centre Hospitalier Intercommunal de Creteil, 40 Avenue de Verdun, 94010 umbilical artery Doppler resistance index (UARI) was
Creteil Cedex, France. Tel.: +33 1 45 17 55 43; fax: +33 1 45 17 55 42. routinely performed. Raw values were converted to
E-mail address: (R. Rouzier). percentiles to account for gestational age at ultrasonography.

0301-2115/$ see front matter # 2007 Elsevier Ireland Ltd. All rights reserved.
S. Deis et al. / European Journal of Obstetrics & Gynecology and Reproductive Biology 137 (2008) 146151 147

Table 1 interaction terms in the model. The logistic regression model

Patient characteristics (n = 4777) was used to construct the nomogram.
Age (mean  S.D., years) 30  5 The model performance was quantified with regard to
Nulliparous (%) 55
discrimination and calibration [7,8]. Discrimination, i.e.,
Twin pregnancy (%) 4
whether the relative ranking of individual predictions is in
Ethnicity the correct order, was quantified with the concordance index,
Caucasian (%) 58
Arabic (%) 18
which is similar to the area under the receiver operating
Black (%) 17 characteristic curve. The concordance index is the prob-
Mixed/other (%) 7 ability that given two randomly selected patients, the patient
Tobacco use (%) 25 with the worse outcome will in fact have a worse prediction.
The concordance index ranges from 0 to 1, with 1 indicating
BMI (kg/m2)
<25 (%) 68
perfect concordance, 0.5 indicating no association (no better
2530 (%) 20 than flipping a coin), and 0 indicating perfect discordance.
>30 (%) 12 We used the bootstrapping technique to obtain relatively
Previous preeclampsia (%) 1 unbiased estimates (200 repetitions): it provides an estimate
Chronic hypertension (%) 2 of the average optimism of the concordance index when all
Auto-immune disease (%) 1 data are included. Calibration, i.e., agreement between
Gestational age at entry (mean  S.D., weeks) 21  1 observed outcome frequencies and predicted probabilities,
BMI: body mass index. was studied with graphical representations of the relation-
ship between the observed outcome frequencies and the
predicted probabilities (calibration curves) for groups of
Percentiles were obtained from the Colle`ge Francais patients defined by quartiles. This was done by grouping
dEchographie Ftale at patients with regard to their nomogram-predicted probabil-
communication/biometrie2000/selectframe.html. ities and then comparing the mean of the group with the
Preeclampsia was defined as the association of gesta- observed occurrence of preeclampsia. A calibration curve
tional hypertension (systolic blood pressure 140 mmHg can be approximated by a regression line with intercept a
and/or a diastolic blood pressure 90 mmHg on at least two and slope b. These parameters can be estimated in a logistic
occasions at least 6 h apart after 20 weeks gestation in regression model with the event as outcome and the linear
women known to be normotensive) plus proteinuria predictor as the only covariate. Well-calibrated models have
(300 mg/24 h). When 24-h urine collection was not a = 0 and b = 1. Therefore, a sensible measure of calibration
feasible, proteinuria was defined as a concentration is a likelihood ratio statistic testing the null hypothesis that
30 mg/dl in at least two random urine samples collected a = 0 and b = 1, which we used. The statistic has a chi-
at least 6 h apart [4]. Patients referred to our tertiary care squared distribution with two degrees of freedom (unrelia-
center for preeclampsia were not included in the study. bility [U] statistic) [9]. We used the bootstrapping technique
Multivariate logistic regression analysis was used to test to internally validate the nomogram. Internal validation does
the association between clinical variables obtained at the not demonstrate the exportability of a model, but is
first prenatal visit and second trimester ultrasonography necessary before external validation. All analyses were
characteristics and occurrence of preeclampsia. Clinical performed using the R package with the Survival, Design,
variables tested included age, ethnicity, body mass index Hmisc, Party, and Lexis libraries (
(<25; 2530; >30 kg/m2), twin pregnancy, parity, previous CRAN/) [10,11]. Local ethics committee agreement was
preeclampsia (PPE), auto-immune disease, chronic hyper- obtained for this study.
tension, tobacco use, previous intra-uterine fetal death,
history of first trimester miscarriage, arterial blood pressure
(systolic blood pressure [SBP], diastolic blood pressure 3. Results
[DBP], and mean arterial blood pressure), and urinary
protein sampling (categorized as 0, 30, 100, and Preeclampsia occurred in 110 patients (2.4%). In a
300 mg/dl). Ultrasonography variables included UARI, multivariate model, nulliparity (P = 0.0021), previous
biparietal diameter, abdominal circumference, and femur preeclampsia (P = 0.0039), DBP at the first visit
length. To determine informative covariates, we performed a (P < 0.0001), UARI (P = 0.08), biparietal diameter
backward variable selection using a method based on (P = 0.011), and twin pregnancy (P = 0.0007) were asso-
Lawless and Singhal [5]. This method uses the fitted ciated with occurrence of preeclampsia (Table 2) and were
complete model and computes approximate Wald statistics informative in the model, while chronic hypertension, body
by calculating conditional (restricted) maximum likelihood mass index, ethnicity, history of first trimester miscarriage,
estimates assuming multivariate normality of the estimates. previous intra-uterine fetal death, auto-immune disease,
Informative variables were selected according to the tobacco use, urinary proteinuria at first prenatal visit,
Akaikes information criterion [6]. We did not include abdominal perimeter, and femur length were not.
148 S. Deis et al. / European Journal of Obstetrics & Gynecology and Reproductive Biology 137 (2008) 146151

Table 2
Factors associated with occurrence of preeclampsia (multivariate analysis)
OR 95% CI P
Nulliparous 2.06 1.632.60 0.0021
Previous preeclampsia 5.08 2.898.92 0.0039
DBP 1.08 1.071.09 <0.0001
UARI 1.98 1.342.93 0.08
Biparietal diameter 0.36 0.240.54 0.011
Twin pregnancy 3.60 2.475.27 0.0007
OR: odds ratio; CI: confidence interval; DBP: diastolic blood pressure;
UARI: umbilical artery resistance index.

A nomogram was built from this model (Fig. 1). A

nomogram is used by first locating the patients position on
each predictor variable scale. Each scale position has
corresponding prognostic points at the top axis. The points
for each variable are added and the probability of
preeclampsia occurrence is estimated from the bottom line.
Fig. 2 represents the discrimination of the nomogram. The
nomogram based on the variables included in the model had Fig. 2. Discrimination of the model: the concordance indices before and
good discrimination: the concordance indices before and after bootstrapping were 0.735 (P < 0.001) and 0.729 (P < 0.001).
after bootstrapping were 0.735 (P < 0.001) and 0.729
(P < 0.001), respectively. Fig. 3 shows how the comparison
between predictions from the nomogram relates to the actual

Fig. 1. Nomogram to estimate the probability of preeclampsia. The nomogram can be used by finding predictor points indicated by the uppermost point scales
that correspond to each patient variable value; the sum of the points (six variables) is projected to the probability of preeclampsia scale at the bottom in order to
determine the probability of preeclampsia. PE: preeclampsia; DBP: diastolic blood pressure; UARI: umbilical artery resistance index.
S. Deis et al. / European Journal of Obstetrics & Gynecology and Reproductive Biology 137 (2008) 146151 149

We developed a computer program to help patients and

physicians use the nomogram. The program called
preeclampsia! (version 1) is programmed in Java text. An
Internet browser with Java enablement is required to run the
applets. The applets will be available on line at http:// An example of the web
interface is represented in Fig. 4.

4. Comment

In this study, we demonstrate that clinical variables and

second trimester ultrasonography data can be combined into
a nomogram for predicting the individual risk of pre-
eclampsia. This nomogram could be used to identify
candidates for intensive screening, prevention, or research
studies. Based on a cohort of 4777 patients, we found along
with others that nulliparity [12], previous history of
Fig. 3. Calibration of the model: the x-axis is the probability of preeclamp- preeclampsia [13], twin pregnancy [14], diastolic blood
sia calculated with the nomogram and the y-axis is the actual rate of pressure at the first prenatal visit [1417], and biparietal
preeclampsia. The dashed line represents the performance of an ideal
diameter were associated with occurrence of preeclampsia.
nomogram. The dotted line is the apparent accuracy without correction
for over-fit. The solid line is the bootstrap-corrected nomogram perfor- In contrast with previous studies that identified risk factors
mance. for preeclampsia, we were able to build and validate a tool
that might be used to predict individual risk of preeclampsia
[18,19]. Other predictors have been reported, but they are
outcomes of the 4777 patients. The x-axis is the prediction based on serum markers or ultrasonography data that are not
calculated by the nomogram and the y-axis is the actual rate routinely performed [18,19]. Moreover, some of these
of preeclampsia. The dotted line represents the performance predictors are based on case control studies and their
of an ideal nomogram in which predicted outcome perfectly utilization in a wider setting needs to be demonstrated [20].
corresponds to actual outcome. The performance of the In contrast, the predictor we developed is based on widely
nomogram is plotted in two ways. The dashed line is the used criteria. We provide a combination of well-known risk
apparent accuracy without correction for over-fit. The solid factors for preeclampsia rather than new criteria to predict
line is the bootstrap-corrected nomogram performance, a preeclampsia. This tool can be used on an individual level to
scatter estimate of future accuracy. The unreliability index decide whether to carry out close follow-up or prevention in
was 5.2  10 4. The predictions calculated with use of the high-risk patients.
nomogram correspond accurately to the actual outcomes of The most significant covariate in our nomogram was
patients with a 15% or less risk of preeclampsia. Above diastolic blood pressure at the first visit. In fact, diastolic
15%, the prediction was less accurate. Of note, 90% of blood pressure may account for chronic hypertension. We
patients have an estimated probability of preeclampsia initially tested chronic hypertension as a potential covariate
occurrence less than 4.4 and 95% of patients less than 5.9%. during the development of the nomogram. Chronic

Fig. 4. Example of the web interface available at

150 S. Deis et al. / European Journal of Obstetrics & Gynecology and Reproductive Biology 137 (2008) 146151

hypertension was associated with occurrence of preeclamp- our population and the published data (mainly by North
sia in the univariate analysis (P = 0.03, data not shown); American teams). In the Medical Birth Registry of Norway,
however, it did not remain significant in the multivariate which includes data on 547,238 women with a first and
analysis because of the interaction with diastolic blood second pregnancy, the prevalence of preeclampsia in the first
pressure. Finally, insulin-deficient diabetes was also pregnancy was 3.6% (n = 19,970) and in the second
analyzed in our study. However, the number of patients pregnancy 1.7% (n = 9535), corroborating our figures
with that condition (n = 13) was too low to draw any [26]. In fact, more than 3% of patients delivering in our
conclusion. tertiary center have preeclampsia. However, for the purposes
The nomograms provide probability estimates that might of the study, we did not include patients referred for
be useful on an individual level. For example, a primiparous preeclampsia and performed a population-based study on
woman (10 points), with a singleton pregnancy (0 point), a the cohort of patients with a first prenatal visit in our
diastolic blood pressure at first visit of 80 mmHg (42 points), Maternity University before 25 weeks gestation.
an umbilical artery resistance index >90th percentile (10 Second, variables of our nomograms were obtained
points), and a biparietal diameter at the 50th percentile (8 between 20 and 23 weeks gestation; therefore, the
points) scores 70 points, which translates into a 5% nomogram should be used within this range. Interestingly,
probability of having preeclampsia. A woman with a history even if percentiles were obtained from the Colle`ge
of preeclampsia during a previous pregnancy (23 points), a Francais dEchographie Ftale, percentiles could be
diastolic blood pressure at first visit of 85 mmHg (50 points), obtained within the study population with similar results.
an umbilical artery resistance index >90th percentile (10 The size of our population (more than 4500 women)
points), and a biparietal diameter <10th percentile (13 explains this finding.
points) scores 96 points, which translates into a 38% Third, we validated our nomogram by bootstrapping,
probability of subsequent preeclampsia, which should be which essentially allows correction for over-fitting, but the
interpreted as a more than 15% probability of preeclampsia generalizability of our nomogram has to be demonstrated by
(Fig. 3). Conversely, a woman with a previous history of applying it in an independent population [8]. The boot-
preeclampsia (23 points), but with a diastolic blood pressure strapping of the calibration curve in our study showed that
at first visit of 50 mmHg (10 points), an umbilical artery individual estimates were not well calibrated over 15%,
resistance index at the 50th percentile (5 points), and a suggesting that the nomogram was able to identify a group
biparietal diameter at the 50th percentile (8 points) scores 46 at high risk of preeclampsia (>15%), but was not able to
points, which translates into a <4% probability of provide individual predictions over 15%. We do not think
subsequent preeclampsia. These two latter examples that it is a matter of concern in daily practice for two
demonstrate that two patients with the same population- reasons. First, one has to remember that a 10% risk of
based estimate of preeclampsia occurrence (2235% based preeclampsia corresponds to a relative risk >4. Second, the
on their personal history of preeclampsia [21]) actually have identification of a group of very high-risk patients is more
different individual risk estimates. This may explain the important than a raw score. In contrast, for other patients
negative results of trials that studied the prevention of (the vast majority of patients), any preventive intervention
preeclampsia by low-dose aspirin in a high-risk population would carry a huge financial cost and will lead to over-
[22]. treatment because of the low prevalence of preeclampsia.
We plan to further improve our nomogram by adding Therefore, extremely well-calibrated predictors for low-
biological markers such as early second trimester serum and intermediate-risk patients are mandatory. Our nomo-
human chorionic gonadotropin [23] and second trimester gram is very well calibrated for these patients and could be
uterine Doppler [24]. It is likely that combining clinical data useful for calculating the cost/benefit ratio of a preventive
and serum markers will provide a more accurate predictor trial and selecting the most efficient design. The utility of
compared with the use of only one of these elements [25]. our nomogram can be illustrated by designing a preventive
When preventive medication for preeclampsia becomes trial based on it. If we consider that a preventive action will
available, we believe that our nomogram will be a useful tool reduce the occurrence of preeclampsia by 20%, for
for selecting eligible patients. example, in order to detect an effect in the general
However, there are limitations to our study. First, even population (prevalence 2.5%), the trial should include 1136
though our study population included almost 5000 patients, patients in each arm to demonstrate the benefit with a = 5%
some characteristics that have been previously reported as and b = 20%. Conversely, if only patients with more than an
risk factors for preeclampsia (body mass index, chronic 8% probability of preeclampsia are included in the trial, the
hypertension, auto-immune disease, ethnicity) did not number of patients to include will be 137 in each arm. In the
emerge in our model [14]. The low prevalence of first design, 6 occurrences of preeclampsia out of 27 would
preeclampsia in our study is likely to be the reason for have been prevented versus four out 21 in the second design.
this finding and may be explained by patients character- The number of patients over-treated and the cost of the first
istics (ethnicity, low prevalence of obesity and chronic design clearly favor the second design and indicate the
hypertensive disorders, etc.), which are different between utility of our nomogram.
S. Deis et al. / European Journal of Obstetrics & Gynecology and Reproductive Biology 137 (2008) 146151 151

5. Conclusion [13] Mostello D, Catlin TK, Roman L, Holcomb Jr WL, Leet T. Pre-
eclampsia in the parous woman: who is at risk? Am J Obstet Gynecol
Our nomogram predicts the probability of preeclampsia. [14] Duckitt K, Harrington D. Risk factors for pre-eclampsia at antenatal
We also developed a web-based interface to provide the booking: systematic review of controlled studies. BMJ 2005;330:
physician with a friendly version of our nomogram. After 5657.
validation in an independent population, this tool could be [15] Higgins JR, Walshe JJ, Halligan A, et al. Can 24-hour ambulatory
blood pressure measurement predict the development of hypertension
used to plan a preventive trial and could serve as the basis for
in primigravidae? Br J Obstet Gynaecol 1997;104:35662.
integrating future markers into a bio-clinical prediction [16] Sibai BM, Ewel M, Levine RJ, et al. Risk factors associated with
model. preeclampsia in healthy nulliparous women. Am J Obstet Gynecol
[17] Sibai BM, Gordon T, Thom E, et al. Risk factors for preeclampsia in
healthy nulliparous women: a prospective multicenter study. Am J
References Obstet Gynecol 1995;172:6428.
[18] Fayyad AM, Harrington KF. Prediction and prevention of preeclamp-
[1] Friedman SA, Lindheimer MD. Prediction and differential diagnosis. sia and IUGR. Early Hum Dev 2005;81:86576.
In: Lindheimer MD, Roberts JM, Cunningham FG, editors. Chesleys [19] Papageorghiou AT, Yu CK, Erasmus IE, Cuckle HS, Nicolaides KH.
hypertensive disorders in pregnancy. Appleton and Lange: Stamford, Assessment of risk for the development of pre-eclampsia by
Connecticut; 1999. p. 20127. maternal characteristics and uterine artery Doppler. BJOG
[2] Dekker GA. Risk factors for preeclampsia. Clin Obstet Gynecol 2005;112:7039.
1999;42:42235. [20] Spencer K, Yu CK, Savvidou M, Papageorghiou AT, Nicolaides KH.
[3] Sibai BM. Prevention of preeclampsia: a big disappointment. Am J Prediction of pre-eclampsia by uterine artery Doppler ultrasonogra-
Obstet Gynecol 1998;179:12758. phy and maternal serum pregnancy-associated plasma protein-A, free
[4] Report of the National High Blood Pressure Education Program. beta-human chorionic gonadotropin, activin A and inhibin A at
Working group report on high blood pressure in pregnancy. Am J 22 + 0 to 24 + 6 weeks gestation. Ultrasound Obstet Gynecol
Obstet Gynecol 2000;183:S122. 2006;27:65863.
[5] Lawless JF, Singhal K. Efficient screening of nonnormal regression [21] Sibai BM. Diagnosis, prevention, and management of eclampsia.
models. Biometrics 1978;34:31827. Obstet Gynecol 2005;105:40210.
[6] Akaike H. Information theory and extension of the maximum like- [22] Caritis S, Sibai B, Hauth J, et al. Low-dose aspirin to prevent
lihood principle. In: Petrov BN, Csaki F, editors. Proceedings of the preeclampsia in women at high risk. N Engl J Med 1998;338:70115.
2nd international symposium of information theory. 1973. p. 26781. [23] Ashour AMN, Lieberman ES, Wilkins Haug LE, Repke JT. The value
[7] Kattan MW, Scardino PT. Prediction of progression: nomograms of of elevated second-trimester beta-human chorionic gonadotropin in
clinical utility. Clin Prostate Cancer 2002;1:906. predicting development of preeclampsia. Am J Obstet Gynecol
[8] Harrell Jr FE. Regression modeling strategies, with applications to 1997;176:43842.
linear models. Logistic regression, and survival analysis. New York: [24] Bower S, Schuchter K, Campbell S. Doppler ultrasound screening as
Springer; 2001. part of routine antenatal scanning: prediction of preeclampsia and
[9] Cox DR. Two further applications of a model for binary regression. intrauterine growth retardation. Br J Obstet Gynaecol 1993;100:
Biometrika 1958;45:5625. 98994.
[10] Harrell Jr FE. Hmisc: a package of miscellaneous S functions. Nash- [25] Nevins JR, Huang ES, Dressman H, et al. Towards integrated clinico-
ville, Tennessee, USA: Vanderbilt University; 2005, genomic models for personalized medicine: combining gene expres- sion signatures and clinical factors in breast cancer outcomes
[11] R Development Core Team. R: a language and environment for prediction. Hum Mol Genet 2003;12(S2):1537.
statistical computing. Vienna, Austria: R Foundation for Statistical [26] Trogstad LI, Eskild A, Magnus P, Samuelsen SO, Nesheim BI.
Computing; 2005, Changing paternity and time since last pregnancy; the impact on
[12] Eskenazi B, Fenster L, Sidney S. A multivariate analysis of risk factors pre-eclampsia risk. A study of 547 238 women with and without
for preeclampsia. JAMA 1991;266:23741. previous pre-eclampsia. Int J Epidemiol 2001;30:131722.