com
SARL au capital de 150 000  RCS Toulouse 87 B 1302  SIRET 342 662 673 00025  Code NAF 722 Z
Atoll SPM
(Standard Propagation Model)
calibration guide
January 2004
 Confidential  2/21
CONTENTS
1 INTRODUCTION......................................................................................................................................... 3
2 GUIDELINES FOR CW MEASUREMENT SURVEYS.......................................................................... 4
3 CALIBRATION PROCEDURE.................................................................................................................. 6
3.1 OBJECTIVE............................................................................................................................................... 6
3.2 STANDARD PROPAGATION MODEL: DEFINITION...................................................................................... 6
3.2.1 SPM formula ................................................................................................................................... 6
3.2.2 Calculations in Atoll........................................................................................................................ 6
3.3 PREREQUISITE ACTIONS ......................................................................................................................... 13
3.3.1 Data validation.............................................................................................................................. 13
3.3.2 Signal strength filter...................................................................................................................... 13
3.3.3 Distance filter................................................................................................................................ 13
3.3.4 Points density filter........................................................................................................................ 13
3.3.5 Other filters ................................................................................................................................... 14
3.4 CALIBRATION METHOD.......................................................................................................................... 14
3.4.1 Calibration and verification stations............................................................................................. 14
3.4.2 Effective antenna height method ................................................................................................... 14
3.4.3 Clutter parameters ........................................................................................................................ 15
3.4.4 Calibration steps ........................................................................................................................... 16
 Confidential  3/21
1 Introduction
To find an accurate model for propagation losses is a leading issue when planning a
mobile radio network. Two strategies for predicting propagation losses are in use these
days; one is to derive an empirical propagation model from measurement data, and the
other is to use a deterministic propagation model. Atoll proposes a macrocell propagation
model, the Standard Propagation Model (SPM), based on empirical formulas and a set
of parameters.
When Atoll is installed, the SPM parameters are set to their default values. However, they
can be adjusted to tune the propagation model according to actual propagation
conditions. This calibration process of the Standard Propagation Model facilitates
improving the prediction reliability.
This document is a guide describing the method and the steps to calibrate SPM. The
ensuing tuned model is an additional model directly usable in Atoll.
 Confidential  4/21
2 Guidelines for CW measurement surveys
The CW measurement survey part is very important in the calibration process. Inaccurate
measurements lead to an inaccurate calibration and, hence, to an inaccurate model. The
measurement provider must abide by the following rules:
The area under test must be scanned before performing the drive test to ensure
no interference exists.
The transmit frequency must have a clearance of 400kHz from radio interference
(i.e. 1 GSM channel on either side of the transmit channel must not be in
operation). This can be verified by checking if the reception level is at zero when
the transmitter is off.
Only one frequency must be measured.
The GPS of the CW measurement equipment should be configured to that of the
mapping data.
All maps used must be have the same projection as that of the Atoll mapping
data.
A minimum of about 8 stations should be measured for each model calibrated (the
exact number is terrain dependent i.e. what route length can be traversed).
Stations should be chosen to meet the following conditions:
Good RF clearance (no nearby obstructions).
Prefer an omnidirectional antenna on stations without surrounding obstacles.
Use a sectored antenna if obstructions are present in the neighbourhood to
decrease the reflections from the backside.
Varied antenna height (20m to 50m).
Varied terrain around each station (flat, hilly)  within a 10km radius.
Varied clutter around the station (open, urban, suburban, dense urban etc.)
within a 10km radius from the station. Clutter categories should all be
represented roughly equally with a minimum of 300 measurement bins in each
category.
Sufficient roads/motorways available to perform measurements.
Panoramic photographs should be taken from each station (rooftop) starting from
north in a clockwise direction. The photographs should show the surroundings in
each direction. Orientations and station number should be written on the back of
each photograph.
A rooftop sketch must be provided. The sketch must indicate the locations of:
The transmitting antenna
Any rooftop obstacles (precise location, distance from transmitter, height)
Any nearby obstacles (other buildings) within 400m of the transmitter (precise
location, distance from transmitter, height, width)
2 measurements per mapping data pixel should be taken (defined by the mapping
data resolution e.g. 25m, 100m or 200m as appropriate).
Measurement surveys should be obtained for distances up to 10km (or until the
noise floor of the receiver has been reached).
Measurement routes must be targeted to have an equal number of samples near
as well as far from the transmitter.
Stored measurements should be averaged between samples and the mean signal
level (50th percentile) must be stored.
Antenna patterns (tilt and orientation) must be supplied in Atoll format.
CW measurement surveys should be well documented with a Station
Measurement Form (position, antenna height, transmit power, gains and losses)
and Path Measurement Form (1 form completed per measurement file).
Maps should accompany each drive test that should indicate the route surveyed.
The map should be annotated to indicate:
The location of the test transmitter
 Confidential  5/21
Locations of any spurious measurements
Where the physical clutter data does not coincide with the mapping data
 Confidential  6/21
3 Calibration procedure
3.1 Objective
The overall objective is to minimize the error between the propagation model predictions
and the CW survey data. The metrics used to quantify the error are the mean and the
standard deviation of the error.
The aim during the calibration process is to reach a null mean error and a low standard
deviation for all the measurement data taken as a whole and to minimize these two
criteria for each single measurement path. A common target value for standard deviation
is 8.
The Atoll Standard Propagation model (SPM) has a large number of parameters and
options, which may be selected or calibrated by the user in order to obtain a close
representation of measured propagation data.
3.2 Standard Propagation Model: Definition
3.2.1 SPM formula
SPM is based on the following formula:
( ) ( ) ( ) ( ) ( ) ( ) clutter f K H K H d K loss n Diffractio K H K d K K
L clutter Rxeff Txeff Txeff el
+ + + + + + =
6 5 4 3 2 1 mod
log log log log
with,
K
1
: constant offset (dB).
K
2
: multiplying factor for log(d).
d : distance between the receiver and the transmitter (m).
K
3
: multiplying factor for log(H
Txeff
).
H
Txeff
: effective height of the transmitter antenna (m).
K
4
: multiplying factor for diffraction calculation. K
4
has to be a positive number.
Diffraction loss : loss due to diffraction over an obstructed path (dB).
K
5
: multiplying factor for log(H
Txeff
)log(d).
K
6
: multiplying factor for H
Rxeff
.
H
Rxeff
: effective mobile antenna height (m).
K
clutter
: multiplying factor for f(clutter).
f(clutter): average of weighted losses due to clutter.
3.2.2 Calculations in Atoll
3.2.2.1 Visibility and distance between the transmitter and the receiver
For each calculation bin, Atoll determines:
 the distance between the transmitter and the receiver.
If the distance TxRx is lower than the maximum userdefined distance (break
distance), receiver is considered to be near the transmitter. Atoll will use the
set of values marked Near transmitter.
If the distance TxRx is greater than the maximum distance, receiver is
considered to be far from the transmitter. Atoll will use the set of values
marked Far from transmitter.
 whether the receiver is in the transmitter line of sight or not.
If the receiver is in the transmitter line of sight, Atoll will take into account the
set (K1,K2)LOS.
If the receiver is not in the transmitter line of sight, Atoll will use the set
 Confidential  7/21
(K1,K2)NLOS.
3.2.2.2 Effective transmitter antenna height
Effective transmitter antenna height (H
Txeff
) may be calculated in six different ways.
Height above ground
The transmitter antenna height is above the ground (H
Tx
in m).
H
Txeff
= H
Tx
Height above average profile
The transmitter antenna height is determined relative to an average ground height that is
calculated along the profile between a transmitter and a receiver. The profile length
depends on distance min and distance max values and is limited by the transmitter and
receiver locations. Distance min and Distance max are respectively minimum and
maximum distances from the transmitter.
( )
0 0
H H H H
Tx Tx Txeff
+ =
where,
Tx
H
0
is the ground height (ground elevation) above sea level at transmitter (m).
0
H is the average ground height above sea level along the profile (m).
Note: If the profile is not located between the transmitter and the receiver, H
Txeff
equals
H
Tx
.
Slope at receiver between 0 and distance min
The transmitter antenna height is calculated using the ground slope at receiver.
( ) ( ) d K H H H H H
Rx Rx Tx Tx Txeff
+ + + =
0 0
where,
Rx
H is the receiver antenna height above the ground (m).
Rx
H
0
is the ground height (ground elevation) above sea level at receiver (m).
K is the ground slope calculated over a userdefined distance (Distance min). In this
case, Distance min is a distance starting from the receiver.
Notes:
1. If m H
Txeff
20 < then, Atoll uses 20m in calculations.
2. If m H
Txeff
200 > then, Atoll takes 200m.
Spot H
t
If
Rx Tx
H H
0 0
> then, ( )
Rx Tx Tx Txeff
H H H H
0 0
+ =
If
Rx Tx
H H
0 0
s then,
Tx Txeff
H H =
Abs Spot H
t
Rx Tx Tx Txeff
H H H H
0 0
+ =
Enhanced slope at receiver
Atoll offers a new method, called Enhanced slope at receiver, to evaluate the effective
transmitter antenna height.
 Confidential  8/21
H
Rx
H
Txeff
H
0
+
H
0Tx
30m
R
d
LOS
line
regression
line
terrain
profile
Let xaxis and yaxis respectively represent positions and heights. We assume that xaxis
is oriented from transmitter (origin) to receiver.
This calculation is achieved in several steps:
1
st
step: Atoll determines line of sight between transmitter and receiver.
The LOS line equation is:
( ) ( )
( ) ( ) ( )
( ) i s
d
H H H H
H H i Los
Rx Rx Tx Tx
Tx Tx
Re
0 0
0
+ +
+ =
where,
i is the point index.
Res is the profile resolution (distance between two points).
2
nd
step: Atoll extracts the transmitterreceiver terrain profile.
3
rd
step: Hills and mountains are already taken into account in diffraction calculations.
Therefore, in order for them not to unduly influence the regression line calculation, Atoll
filters the terrain profile.
Atoll calculates two filtered terrain profiles, one established from transmitter and another
from receiver. It determines filtered height of every profile point. Profile points are evenly
spaced on the basis of profile resolution. To determine filtered terrain height at a point,
Atoll evaluates ground slope between two points and compares it with a threshold set to
0.05; three cases are possible.
Some notations defined hereafter are used in next part.
filt
H is the filtered height.
orig
H is the corrected original height. Original terrain height is determined from extracted
ground profile and corrected by considering Earth curvature.
 Filter starting from transmitter
Let us assume that ( ) ( ) Tx H Tx H
orig Tx filt
=
Then, for every point of profile, Atoll compares the two filtered heights and chooses the
greater one.
( ) ( ) ( ) ( ) i
H
i
H
i
H Rx filt Tx filt filt
= , max ( ) ( ) ( ) ( ) i H i H i H
Rx filt Tx filt filt
= , max
4
th
step: Atoll determines the influence area, R. It corresponds to the distance from
receiver at which the original terrain profile + 30m intersects LOS line for the first time
(when beginning from transmitter).
The influence area must satisfy additional conditions:
m R 3000 > ,
d R > 01 . 0 ,
R must contain at least three bins.
Notes:
1. When several influence areas are possible, Atoll chooses the highest one.
2. If d < 3000m, R = d.
5
th
step: Atoll performs a linear regression on the filtered profile within R in order to
determine a regression line.
The regression line equation is:
b ax y + =
( ) ( ) ( ) ( )
( ) ( )
=
i
m
i
m filt m
d i d
H i H d i d
a
2
and
m m
ad H b =
where,
 Confidential  10/21
( )
=
i
filt m
i H
n
H
1
i is the point index. Only points within R are taken into account.
2
R
d d
m
=
d(i) is the distance between i and transmitter (m).
Then, Atoll extends the regression line to the transmitter location. Its equation is:
( ) ( ) b s i a i regr + = Re
6
th
step: Atoll calculates effective transmitter antenna height,
Txeff
H (m).
a
b
H H
H
Tx Tx
Txeff
2
0
1+
+
=
If H
Txeff
is lower than 20m, Atoll recalculates it with a new influence area, which begins at
transmitter.
Notes:
1. In case m H
Txeff
1000 > , 1000m will be used in calculations.
2. If
Txeff
H is still lower than 20m, an additional correction is taken into account (7
th
step).
7
th
step: If
Txeff
H is still lower than 20m (even negative), Atoll evaluates path loss using
m H
Txeff
20 = and applies a correction factor.
Therefore, if m H
Txeff
20 < ,
( )
K
f d m
H L L lowant Txeff el el
+ = = , , 20
mod mod
where,
( )
( ) ( )

.

\

+ 
.

\

+
=
1000
93 . 6
1000
63 . 9
20 1 20
20 3 . 0
10
5
d d
H
H
d
K
Txeff
Txeff lowant
3.2.2.3 Receiver effective antenna height
( )
Tx Rx Rx Rxeff
H H H H
0 0
+ =
where,
Rx
H is the receiver antenna height above the ground (m).
Rx
H
0
is the ground height (ground elevation) above sea level at the receiver (m).
Tx
H
0
is the ground height (ground elevation) above sea level at the transmitter (m).
Note: The calculation of effective antenna heights (
Rxeff
H and
Txeff
H ) is based on extracted
DTM profiles. They are not properly performed if you have not imported heights (DTM
file) beforehand.
3.2.2.4 Correction for hilly regions in case of LOS
An optional corrective term enables Atoll to correct path loss for hilly regions when
transmitter and receiver are in LOS.
Therefore, if receiver is in transmitter line of sight and the Hilly terrain correction option
is active, we have:
( ) ( ) ( ) ( ) ( )
K
clutter f
K H K
d
H K H K
d
K K L LOS hill clutter Rx Txeff Txeff LOS LOS el , 6 5 3 , 2 , 1 mod
log log log log + + + + + + =
When transmitter and receiver are not in line of sight, the path loss formula is:
( ) ( ) ( ) ( ) ( ) clutter f
K H K
d
H K
n Diffractio
K H K
d
K K L clutter Rx Txeff Txeff NLOS NLOS el
+ + + + + + =
6 5 4 3 , 2 , 1 mod
log log log log
 Confidential  11/21
K LOS hill ,
is determined in three steps. Influence area, R, and regression line are considered
available.
1
st
step: For every profile point within influence area, Atoll calculates height deviation
between the original terrain profile (with Earth curvature correction) and regression line.
Then, it sorts points according to the deviation and draws two lines (parallel to the
regression line), one which is exceeded by 10% of the profile points and the other one by
90%.
2
nd
step: Atoll evaluates the terrain roughness, Ah; it is the distance between the two
lines.
3
rd
step: Atoll calculates
K LOS hill ,
.
We have
K K K hf h LOS hill
+ =
,
If m h 20 0 s A < , 0 =
Kh
Else ( ) ( ) 746 . 6 log 29 . 15 log 73 . 7
2
+ A A = h h
Kh
If m h 10 0 s A < , ( ) ( )
i
regr
H H K Rx Rx Rx hf
+ =
0
1924 . 0 2
Else ( ) ( ) ( )
( )
h
i
regr
H H
h h
K
Rx Rx Rx
hf
A
+
A + A =
0 2
21 . 11 log 75 . 14 log 616 . 1 2
i
Rx
is the point index at receiver.
3.2.2.5 Diffraction
Four methods are available to calculate diffraction loss over the transmitterreceiver
profile. They are not detailed here.
Deygout
EpsteinPeterson
Deygout with correction
Millington
You may take the following into account along the transmitterreceiver profile :
Either ground altitude or clutter height
In this case, Atoll takes clutter height information in clutter heights file if available
in .atl document. Otherwise, it considers average clutter height specified for each
clutter class in the clutter classes file description.
Or only ground altitude
3.2.2.6 Losses due to clutter
Atoll calculates f(clutter) over a maximum distance from receiver.
( )
=
=
n
i
i i
w L clutter f
1
where,
L: loss due to clutter userdefined in the Clutter tab (dB).
w: weight determined using the weighting function.
n: number of points taken into account over the profile. Points are evenly spaced
depending on the profile resolution.
Four weighting functions are available:
Uniform weighting function:
n
w
i
1
=
 Confidential  12/21
Triangular weighting function:
=
=
n
j
j
i
i
d
d
w
1
'
i i
d D d = , where d
i
is the distance between the receiver and the ith point, and D is the
maximum distance (userdefined).
Logarithmic weighting function:
=


.

\

+

.

\

+
=
n
j
j
i
i
D
d
D
d
w
1
1 log
1 log
Exponential weighting function:
=
n
j
D
d
D
d
i
j
i
e
e
w
1
1
1
3.2.2.7 Precautions
Be careful that the clutter influence may be taken into account in two terms, Diffraction
loss and f(clutter). To avoid this, we advise:
1. Not to take into account clutter heights to evaluate diffraction loss over
the transmitterreceiver profile if you specify losses per clutter class.
This approach is recommended if the clutter height information is statistical.
2. Not to define any loss per clutter class if you consider clutter heights in
the diffraction loss.
In this case, f(clutter)=0. Losses due to clutter are only taken into account in the
Diffraction loss term evaluated.
This approach is recommended if the clutter height information is either
semideterministic or deterministic.
In case of semideterministic clutter information, specify receiver clearance (m) per
clutter class. Both ground altitude and clutter height are considered along the whole
transmitterreceiver profile except over a specific distance around the receiver
(clearance), where Atoll proceeds as if there was only the DTM map. Atoll uses the
clearance information to model streets.
Tx
Rx
clearance
clutter
DEM
TxRx profile. Ground altitude and clutter height (here, average height specified for each clutter class in the
clutter classes map description) are taken into account along the profile.
In case of deterministic clutter height information, clearance definition is not necessary.
Clutter height information is accurate enough to be used directly without additional
 Confidential  13/21
information such as clearance. Here, losses due to clutter are taken into account in the
Diffraction loss term evaluated.
3.3 Prerequisite actions
3.3.1 Data validation
A quick means of data validation is to import the measurement files and a set of vector
files that represent roads in Atoll, in order to check that the data correspond. You can
also check that the measurement path starts and/or ends approximately at the station
location.
If some pictures from the station neighbourhood are available, you can check that no
close obstacle disturbs the propagation. If an obstacle is present in one direction, it is
possible to filter the measurement data according to the orientation by fixing a negative
and a positive angle for which the data will be taken into account.
3.3.2 Signal strength filter
The calibration process aims at producing an accurate model that will represent the
stations propagation within the validity region of the model itself. For this reason, the
models own constraints with respect to signal levels need to be taken into account.
There are limitations in the measurement equipment with dynamic range capability,
which also need to be taken into account.
Generally, signals above 40dBm are filtered out as they would be inaccurate due to
receiver overload. For the minimum signal filtering, the sensitivity of the receiver and the
tolerance have to be considered. So, signals below receiver sensitivity + target standard
deviation have to be filtered out to avoid the effect of noise saturation in the statistical
results.
3.3.3 Distance filter
Measurement data at a distance less than 200m from the base station are discarded
because these points are too close from the station to properly represent the propagation
in the whole area. A common limit for the maximum distance is 10km.
3.3.4 Points density filter
Another filtering can be done according to the project related to the clutter classes. If
only a few measurement paths contain a specific clutter class or only a few points are
located in this class, then this clutter class can be filtered out. Keeping this class can, in
fact, generate some bad statistical results or affect the calibration process incorrectly.
For example, in the following case, the water class can be filtered out. This class would
influence the prediction only in areas with large surfaces of water.
Number of points per clutter class measurement
 Confidential  14/21
3.3.5 Other filters
Some points can also be filtered out in different clutter classes because of a huge
diffraction effect which would not be representative of the propagation in the whole area.
For example, several peaks in the profile between the station and some measurement
points can introduce errors because of the nature of signal received there and may
influence the calibration in a wrong way.
3.4 Calibration method
3.4.1 Calibration and verification stations
The choice of the stations used for calibration and for verification is very important in
order to correctly calibrate the model. For that, you have to visualize the different
measurement paths on the map:
 For calibration, you have to choose the paths that cover the whole area so that all the
area characteristics be taken into account during the calibration process.
 For verification, you have to choose several paths (the number depends on the total
number of available paths) that are inside the covered area, and not at the extremity, so
that these paths have no limitations in comparison to paths used for calibration.
If enough measurement paths are not available, all of them have to be used for the
calibration process. The verification is then performed individually with the same
measurement paths.
3.4.2 Effective antenna height method
An effective antenna height method has to be chosen according to the properties of the
terrain. This model parameter is very important since the calculated antenna heights may
differ and this greatly influences the propagation. That is why it is useful to split the
entire zone into several areas according to the relief.
For hilly area, it is advised to use the Enhanced slope at receiver method and to
activate the hilly terrain correction .
 Confidential  15/21
Whereas for a plain, it is advised to use the Height above average profile method and
to deactivate the hilly terrain correction.
3.4.3 Clutter parameters
Clutter heights must not be used with a statistical clutter because data, in this case, are
not precise enough. It is advised to set all the clutter heights to 0 in the clutter
properties window and to use the losses per clutter class in the model properties. For
this, you have to set the Kclutter to 1 in the Parameters tab. Then, in the Clutter tab, you
must provide a max distance for which the loss per clutter will be applied (500m is a
common value). You have to choose a weighting function and set all the clearances per
clutter class to 0, given the fact that the clutter heights are not used.
 Confidential  16/21
3.4.4 Calibration steps
Now your model has the initial parameters necessary to start the calibration. It is advised
to duplicate this modified model as it is rare to get a good calibration in the first attempt.
 Confidential  17/21
3.4.4.1 Step 1
Set the filters in the measurement path properties window.
The filters about the field and the distance are defined and applied in the measurement
 Confidential  18/21
and in the model properties at the same time. Remember, when using filters in the
measurement properties window, that these filters will be applied during calibration even
if the corresponding filters in the model properties are modified.
In the calibration tab, select the calibration paths, set the filters and check the LOS (Line
Of Site) and NLOS (Non Line Of Site) boxes.
3.4.4.2 Step 2
Press the calibrate button and find the variable having the highest correlation with error.
You can see the regression line when you select this variable.
Press Identify, so that Atoll calculates the necessary correction to apply to its related
constant thus decreasing the correlation of this variable with error.
 Confidential  19/21
3.4.4.3 Step 3
If the most correlated variable is log(D), then you have to calibrate K1 and K2 separately
in case of LOS or NLOS. For that, press Cancel and successively calibrate these constants
in LOS and NLOS by selecting only the log(D) variable in the calibration window and
pressing Identify then press OK after having checked Step 4.
After that, check LOS and NLOS boxes again to calibrate the other variables.
3.4.4.4 Step 4
Check that the calculated correction does not make the related constant cross the
physical limits given below (recommended values).
 Confidential  20/21
Constant min typical max
K2 20 44.9 70
K3 20 5.83 20
K4 0 0.5 0.8
K5 10 6.55 0
K6 1 0 0
If the constant with the suggested correction respects the given limits, press OK. If the
constant with the suggested correction doesnt respect the given limits, manually set the
constant to the limit after pressing Cancel.
3.4.4.5 Step 5
Repeat the process from Step 2. The correction should have made the statistics better.
When the statistics are stabilizing, calibrate the Clutter variable so that the losses per
clutter class are calculated. Then you can try again to calibrate the other variables.
When the statistics are stabilizing, the calibration has finished. If the results do not
achieve the aim you can start again from the initial model by changing parameters such
as the effective antenna height method or the using of zones near the transmitter and far
from the transmitter, etc.
3.4.4.6 Step 6
When the calibration has finished, you have to check its quality with the verification
paths.
In the Calibration tab, select the corresponding paths and press Statistics.
 Confidential  21/21
You can study the statistics report to check the model accuracy.