Mco30 Boc All Tre-Book04

MCO 3.
0
Operator Course
Exercises Book
Reference: exercises_MCO_3.0.docx
Version: 12
Status: ISSUED
Release Date: 25.06.2021
Copyright © Mavenir 2021
COMMERCIAL-IN-CONFIDENCE
MCO 3.0 Operator Course Exercises Book
Copyright © Mavenir 2021. All rights reserved. This document is protected by international copyright law and may not be reprinted,
reproduced, copied or utilised in whole or in part by any means including electronic, mechanical, or other means without the prior written
consent of Mavenir.
Whilst reasonable care has been taken by Mavenir to ensure the information contained herein is reasonably accurate, Mavenir shall not,
under any circumstances be liable for any loss or damage (direct or consequential) suffered by any party as a result of the contents of this
publication or the reliance of any party thereon or any inaccuracy or omission therein. The information in this document is therefore
provided on an “as is” basis without warranty and is subject to change without further notice and cannot be construed as a commitment
by Mavenir.
The products mentioned in this document are identified by the names, trademarks, service marks and logos of their respective companies
or organisations and may not be used in any advertising or publicity or in any other way whatsoever without the prior written consent of
those companies or organisations and Mavenir.
E & OE
MCO 3.0 Operator Course Exercises Book v 12 Status: ISSUED

Copyright © Mavenir 2021 COMMERCIAL-IN-CONFIDENCE Page 2 of 89
Table of Contents
Table of Contents ............................................................................................................. 3
List of Figures 6
Document Version History ................................................................................................ 6
1. MCO Lab Overview .................................................................................................... 8
1.1. SMSC, Router, Firewall and IPSMGW - Different exercises for MCO roles ........................................... 8
1.2. SMSC, Router, Firewall - Lab layout and connections. .......................................................................... 8
1.3. IPSMGW - Lab layout and connections. ................................................................................................ 8
1.4. Web Browser and Guacamole - Training Lab Preferred Access from the Internet ............................... 9
1.5. SMSC, Router, Firewall - Emulators..................................................................................................... 12
1.6. Identifying windows ............................................................................................................................ 13
1.7. IPSMGW - Emulators ........................................................................................................................... 14
1.8. Getting familiar with VI editor ............................................................................................................ 16
2. MCO Overview Exercises ......................................................................................... 17

2.1. Getting familiar with GUI and basic OAM commands ........................................................................ 17
2.2. Getting familiar with CML and basic OAM commands ....................................................................... 17
2.3. Backup MCO ........................................................................................................................................ 19
2.4. Getting familiar with REST API ............................................................................................................ 19
3. SOL Exercises ........................................................................................................... 21

3.1. SS7oIP configuration ........................................................................................................................... 21
3.2. Verifying SS7 AF configuration ............................................................................................................ 22
3.3. Verifying SS7 linkset status ................................................................................................................. 22
3.4. Suspend a link (requires MCO V3.0-11.01 or higher) .......................................................................... 22
3.5. IPSMGW - MO Router configuration ................................................................................................... 23
3.6. SMSC - Testing SOL with Store and Forward (P2P) ............................................................................. 24
3.7. Router - Testing SOL with MO Proxy ................................................................................................... 25
4. Delivery Layer Exercises ........................................................................................... 27

4.1. IPSMGW - Primary Routing ................................................................................................................. 27
4.2. IPSMGW - Backup Routing .................................................................................................................. 27
4.3. SMSC - Provision SMPP Large Account ............................................................................................... 28
4.4. SMSC - Testing the Delivery Layer - AOMT (A2P), Store, Retry, Alerts ............................................... 28
4.5. SMSC - Primary Routing AOAT (A2A) .................................................................................................. 31
4.6. SMSC - Retry Mechanism Configuration ............................................................................................. 33
4.7. SMSC - Recipient Range Routing RARR – Challenge! .......................................................................... 34
4.8. SMSC - Message Store – Challenge! .................................................................................................... 34
4.9. SMSC - Inter SMSC Router (ISR) .......................................................................................................... 34
5. SLC Exercises ........................................................................................................... 39

5.1. SMSC, IPSMGW - SDR Function ........................................................................................................... 39
5.2. Investigate SDRs from other exercises– Challenge! ............................................................................ 41

6. Firewall - Traffic Shaping Exercises ........................................................................... 42

6.1. XMS GUI - Simple Rate Set up ............................................................................................................. 42
6.2. XMS GUI - Simple Rate Traffic generation .......................................................................................... 47
6.3. XMS GUI - Simple Rate Monitoring ..................................................................................................... 47
6.4. CML - Simple Rate Set up .................................................................................................................... 48
6.5. CML - Simple Rate Traffic generation.................................................................................................. 49
6.6. CML - Simple Rate Monitoring ............................................................................................................ 50
7. IPSMGW - SCN and PAG Exercises ............................................................................ 51

7.1. Mobile-originated messages to SIP ..................................................................................................... 51
7.2. Mobile-terminated messages to SIP ................................................................................................... 51
8. IPSMGW - Simulation Exercises ................................................................................ 52

8.1. MO SIP ................................................................................................................................................. 52
8.2. MO GSM – MT SIP (Success) ............................................................................................................... 52
8.3. MO GSM – MT SIP with fall back to GSM (Fail: SIP user not registered) ............................................ 53
8.4. MT GSM from SMSC to SIP User with fall back to GSM (Fail: SIP user not registered) ....................... 54
8.5. MT GSM from SMSC to SIP User (Success) .......................................................................................... 55
9. MCO Troubleshooting .............................................................................................. 56
9.1. Service and Process Troubleshooting ................................................................................................. 56
9.2. Collecting troubleshooting information for Mavenir Support Team .................................................. 56
9.3. Capturing network traffic .................................................................................................................... 56
9.4. SOL Health Check ................................................................................................................................ 57
9.5. SOL Tracing .......................................................................................................................................... 58
9.6. Selective debugging – Challenge! ....................................................................................................... 59
9.7. Custom log messages – Challenge! ..................................................................................................... 59
10. Extra - Bash scripting ............................................................................................... 61

10.1. counter_monitor ................................................................................................................................. 61
10.2. cml in scripts ....................................................................................................................................... 61
11. Extra - Blacklist log events in CLI .............................................................................. 62
11.1. Disable stats.ALARM_CHANGE ........................................................................................................... 62
12. XMS Access rights management ............................................................................... 63

12.1. Create monitoring account and grant view rights on XMS ................................................................. 63
12.2. Create administration account and grant modification rights on XMS – Challenge! .......................... 65
12.3. Create administration account and grant modification rights on MCO – Challenge! ......................... 65
13. XMS Dashboard Exercises ........................................................................................ 66

13.1. Create Dashboard ............................................................................................................................... 66
13.2. Create Line chart widget. .................................................................................................................... 66
13.3. Create Counter widget ........................................................................................................................ 71
13.4. Create Gauge chart widget. ................................................................................................................ 72
13.5. Create Area chart widget. ................................................................................................................... 73
13.6. Create Pie Chart widget ...................................................................................................................... 75

13.7. Create Table widget ............................................................................................................................ 76

13.8. Move widgets. ..................................................................................................................................... 77
14. Extra - XMS GUI Troubleshooting ............................................................................. 79

14.1. MCO - Monitoring Dashboard ............................................................................................................. 79
15. Extra - Single Point of Access on XMS ....................................................................... 80

15.1. cml --enable_spoa ............................................................................................................................... 80
16. Conditional - DISK_CAPACITY alarm from XMS Application /var/log ......................... 81

16.1. Showing a counter on console ............................................................................................................ 81
17. SMS Firewall ............................................................................................................ 83

17.1. Keyword filtering – Challenge! ............................................................................................................ 83
17.2. “Almost all together” – Challenge! ..................................................................................................... 83
18. End of Training clean up........................................................................................... 86

18.1. Logout of the Windows desktop ......................................................................................................... 86
Appendix A OpenVPN - Training Lab Failback Access ................................................. 88

A.1 Please skip this appendix if you use the web browser as described in chapter 1............................... 88

List of Figures
Figure 1-1 MCO SMSC, Router and Firewall Training Laboratory Architecture ...................................................... 8
Figure 1-2 MCO IPSMGW Training Laboratory Architecture .................................................................................. 9
Figure 1-3 option 2a: IPSMGW Emulator scripts running in one window. ........................................................... 15
Figure 1-4 option 2b: IPSMGW Emulator scripts running on different windows. ................................................ 16
Figure 3-1 Call flow Store and Forward ................................................................................................................ 25
Figure 3-2 Call flow MO Proxy .............................................................................................................................. 26
Document Version History

Issue Date Change Summary Updated By
1 May 2018 Initial version in Mavenir branding. Walter van den
Barselaar
2 June 2018 Added answers to chapter 7 Walter van den
Barselaar
3 June 2018 Added emu.sh script Walter van den
Barselaar
4 July 2018 Merged MCO BOC with IPSMGW Walter van den
Barselaar
5 September 2018 Added DISK_CAPACITY alarm exercise Walter van den
Barselaar
6 October 2018 Added XMS based rate exercise Walter van den
Barselaar
7 March 2019 Added Router exercises Walter van den
Barselaar
8 October 2019 Small changes to match new ipsmgw training lab Walter van den
v7 Barselaar
9 May 2020 Added section 2.4 REST API Walter van den
Barselaar
10 January 2021 Reformatted to have task first and solution later; Frantisek Rak
new baf_default.lua version
11 February 2021 Added new Custom document properties Frantisek Rak
(IP_mco, IP_xmsbaf) for IP addresses of mco
and xmsbaf to easily adjust the doc to actual lab
VMs; added “version” dco property for easy
document versioning
12 June 2021 Added Inter-SMSC Router exercise Walter van den
Barselaar
Changes since last issue

Exercise 4.9 added
Document control
Owner: Walter van den Barselaar Title: Trainer
Approved by: Title:
Approved by: Title:

Other Information: exercises_MCO_3.0.docx

Table 1: Document Version History

1. MCO Lab Overview

1.1. SMSC, Router, Firewall and IPSMGW - Different exercises for MCO roles
This book contains exercises that are specific for the various roles the MCO can perform in the network.
The title of each section or chapter is prefixed with the MCO role name that the exercise in that section
is applicable for. There are four roles: SMSC, Router, Firewall and IPSMGW.
Please only follow the roles that are covered in your MCO training course.
1.2. SMSC, Router, Firewall - Lab layout and connections.

The MCO laboratory consists out of 2 virtual machines. Each student has its own personal MCO
laboratory. The virtual machine called XMSBAF combines MSC, HLR (BAF simulators) with XMS. The
MCO laboratory is configured as follows:
XMSBAF
IP: 172.21.12.6 MCO
IP: 172.21.12.7
MSC PC: 301 M3UA PC: 701
MSC SCTP port: 7777 SCA: 420111111111
SCTP port: 17024
XMS HTTPS port: 443 SMPP port: 9000, 9001
Figure 1-1 MCO SMSC, Router and Firewall Training Laboratory Architecture
1.3. IPSMGW - Lab layout and connections.

The MCO laboratory consists out of 2 virtual machines. Each student has its own personal MCO
laboratory. The virtual machine called XMSBAF combines MSC, HLR, SMSC, S-CSCF, HSS (BAF simulators)
with XMS. The MCO laboratory is configured as follows:

XMSBAF
IP: 172.21.12.6
MSC PC: 702

MSC RC: 7000
MSC GT: 702000002 M3UA
MSC SCTP port: 17002
MCO
HLR PC 703
HLR RC 7000 IP: 172.21.12.7
HLR SCTP port: 17003 M3UA PC: 700
RC: 7000
SMSC PC: 701 SCA: 700123456
SMSC RC: 7001 SCTP port: 17000 & 17010
M3UA SIP: ipsmgw@localdomain
SMSC SCA: 701222222
SMSC SCTP port: 17001 SIP UDP port: 5060
S-CSCF (IMS) SIP: scscf01.localdomain SIP SMPP port: 9000

S-CSCF (IMS) UDP port: 5060
S-CSCF (IMS)URI: +703….@localdomain
DIAMETER
HSS TCP port: 3868
XMS HTTPS port: 443 DNS

dnsmasqd DNS port 53
Figure 1-2 MCO IPSMGW Training Laboratory Architecture
1.4. Web Browser and Guacamole - Training Lab Preferred Access from
the Internet
To access the training VMs from the internet, run Firefox or Chrome on your
computer. Internet Explorer may cause keyboard issues for some keys.
1. Go to Mavenir Training Portal https://training.mavenir.com and click the link "Click

here to access labs".
2. The trainer provides you the Username and Password per assigned student number.
3. If login does not work, ask your trainer to check if the maximum number of sessions
(10) is exceeded in the Windows virtual machine.

4. When the login is successful, you will see a windows desktop in your web browser.
5. Connect via Putty and SSH to the virtual machines:
- mco1: ssh mco@172.21.12.7

username/password: mco/mco
- baf: ssh root@172.21.12.6

username/password: root/root
6. Connect via Firefox to the XMS virtual machine:
- xms url: http://172.21.12.6

username/password: root/root

You can copy/paste between your system and the Windows Desktop via the Guacamole side bar.
1. Press Ctrl-Alt-Shift to show the Guacamole side bar on the left side of
your Windows desktop in your web browser.
2. On your laptop, copy the XMS URL from the PDF version of this
document.
3. Now go back to your browser and paste the XMS URL in the Clipboard
box of the Guacamole side bar.
4. Press Ctrl-Alt-Shift again to hide the Guacamole side bar in your web
browser.
5. In the windows desktop inside your browser start Firefox and paste the
XMS URL into the address bar.
To disconnect from the windows desktop:

1. To disconnect from the windows desktop press Ctrl-Alt-Shift to show the Guacamole side bar on the left
side of your Windows desktop in your web browser.
In the Guacamole side bar click on your username shown in the upper right corner.
2. This opens a pull-down menu with the option to logout . This allows you to
resume your windows session later.
a. Tip: The pull-down menu also presents the option to disconnect
. This allows you to Reconnect. Use this to correct the screen

scaling of the Windows desktop. This is required when you change the size of your browser
window on your laptop. Otherwise, you will see black borders in the browser window on your
laptop.
3. When the training finishes, you may choose to Logout from the Windows desktop before logging out from
Guacamole.
1.5. SMSC, Router, Firewall - Emulators

For simulation, you will use the scripts installed on the BAF server. There are scripts for the simulation
of different scenarios. Each of them simulates network element(s) and specific operation(s) (for
example, MO-FW-SM request from MSC to MCO), per the Figure 1.1. Execute each of them in a
different Putty window.
1. First, connect via ssh to the BAF server using Putty:

2. IP address: 172.21.12.6
User/Password: root/root
3. In different windows, run the different simulators scripts.
For the GSM simulator, type cd baf ; ./start_baf_fr
Type help to list the available commands.
baf> help
submit some message: submit [originator] [recipient] [text]
example: submit 1.1.111222333 1.1.444555666 test
send alert: alert [subscriber] [sc_address]

example: alert 1.1.444555666 1.1.111111111
respond 27-Absent subscriber to SRI: @set sri_err 27

respond 1-Unknown subscriber to SRI: @set sri_err 1
respond sucess to SRI: @set sri_err 0
let SRI timeout: @set sri_err timeout
respond SRI with 1000ms delay : @set sri_latency 1000
respond 9-Illegal subscriber to MT: @set mt_err 9

respond 27-Absent subscriber to MT: @set mt_err 27
respond sucess to MT: @set mt_err 0
let MT timeout: @set mt_err timeout
respond MT with 1000ms delay : @set mt_latency 1000
send sri: sri [subscriber] [sc_address]

example: sri 1.1.444555666 1.1.111111111
send deliver: deliver [originator msisdn] [recipient imsi] [text]

example: deliver 1.1.444555666 333444555666 hihihi
note: parameters are optional. you can ommit them if you don't care
scenario example:
@set sri_err 27
submit
submit
@set sri_err 0
alert
4. Repeat steps 1 and 2 to open a second putty window.

For the SMPP simulator, type cd baf ; ./start_smpp
Type help to list the available commands.
SMPP> help
to connect: @ctrl tcp connect <ip> <port>
to bind transceiver: bind

to bind transmitter: bind_tx
to bind receiver: bind_rx
to unbind: unbind
to submit message: submit [dst] [text] [src]

to submit data_sm: data [dst] [src]
to cancel message: cancel
to query message: query <message_id>
to replace message: replace <message_id>
to respond to every deliver with status 8: @set deliver_err 8

to respond to every deliver with status 0x64 (temp error): @set deliver_err
100
to respond to every deliver with 1000ms delay: @set deliver_latency 1000
1.6. Identifying windows

In the next sections for some cases, it is not explicitly indicated in the text in which window you should
type the command. The prompt that is shown as prefix to the command you need to type, will help you
to select the correct window.
Prompt Command that you ran in that window
xmsbaf# ssh to the XMSBAF server using Putty or any other terminal
application and logged in as root user
$ ssh to the MCO1 server using Putty or any other terminal

application and logged in as mco user
MCO> cml
1.6.1. SMSC, Router, Firewall

baf> start_baf_fr
smpp> start_smpp
1.6.2. IPSMGW

msc> msc.sh or emu.sh top right window
hlr> hlr.sh or emu.sh middle left window
ims> ims.sh or emu.sh middle right window
smsc> smsc.sh or emu.sh top left window
hss> hss.sh or emu.sh bottom window
1.7. IPSMGW - Emulators

For simulation, you will use the scripts installed on the XMSBAF server. There are five different scripts
under the root directory. Each of them simulates a network element, according to the Figure 1.2. Each
of them must be executed in a different Putty window:
smsc.sh simulates SMSC (MAP MT)

hlr.sh simulates HLR (MAP SRI)
msc.sh simulates MSC (MAP MO)
ims.sh simulates IMS core – CSCF (SIP SUBMIT/DELIVER)
hss.sh simulates HSS (Diameter UDR/PUR/SNR)
For convenience there is a sixth script that can start all simulators in one window.
emu.sh starts all simulators in one window (uses screen

utility)
1. First, connect via ssh to the XMSBAF server using Putty or any other Putty application:
IP address: 172.21.12.6
2. Now we have two options: 2a and 2b from which you must choose. You cannot choose both
options!
a. In one window, run the script that starts all simulator scripts.
Putty window (1): xmsbaf# ./emu.sh

Figure 1-3 option 2a: IPSMGW Emulator scripts running in one window.
• Use your left mouse button to change keyboard focus from one simulator to the other.
In some cases you may lose the ability to change keyboard focus from one simulator to
the other.
In that case terminate all simulators type Control-a \ and confirm by typing y <enter>.
Now execute step 2a and changing focus using the mouse will work again.
b. Skip this item if you chose to execute option 2a!
As alternative option, in different windows, run the different simulator scripts. Use the
image (Figure 1-4) below as an example:
Putty window (1): xmsbaf# ./smsc.sh
Putty window (2): xmsbaf# ./hlr.sh
Putty window (3): xmsbaf# ./msc.sh
Putty window (4): xmsbaf# ./ims.sh
Putty window (5): xmsbaf# ./hss.sh

Figure 1-4 option 2b: IPSMGW Emulator scripts running on different windows.
3. After you ran the scripts, you can type help for more information about script usage.
1.8. Getting familiar with VI editor

In this exercise the goal is to practice with toggling between insert and command mode, saving and
leaving the VI editor.
If you are not familiar with the VI editor, use the link below and follow the first three topics:
http://heather.cs.ucdavis.edu/~matloff/UnixAndC/Editors/ViIntro.html

2. MCO Overview Exercises

2.1. Getting familiar with GUI and basic OAM commands
See text book slides on XMS GUI. Follow the steps in the slides using your own lab.
2.2. Getting familiar with CML and basic OAM commands

• Login to MCO via ssh
IP address: 172.21.12.7
User/Password: mco/mco
• Open CML interface:

$ cml
• You will see the output of the CML welcome script:

=============================================
M C O - 3.0
-----------
Date/time: Wed Mar 6 18:24:27 CET 2019
System time spread: 0 sec
NTP servers: 172.21.12.7
L/F/S Zookeepers: 0/0/1
Software version: 0.9-77.45 (oam)
Software version: R3.0-10.09.B09.el7 (mco)
System name: MCO-SMSC-A
System ID:
Operation mode: DEMO
***CAUTION*** System is operating in the DEMO mode. Throughput
of the system is restricted to several messages per second
only !!!
License keys: 0
R/NR monitors: 1/0
R/NR/S/D/U processes: 22/0/0/6/0
Debugging processes: 0
Debugs enabled in conf: 0
SDR streams E/EF: 0/0
=============================================
• Exit CML:
MCO> .exit
• Open CML again with event logging disabled (same as .nolog).

$ cmln
• Show status:
MCO> monitor.processes.show
or
@m
• You do not have to type the complete command. Using the Tab key on your keyboard
you can have the command completed. When multiple commands share the same
characters, the screen will flash and/or a sound will play. In that case, when you press
the Tab key a second time, CML will show you all possibilities. The following sequence
will create the monitor.processes.show command. When you see [Tab] you should
press the Tab key on your keyboard (the key above the Caps Lock key).
m[Tab][Tab]o[Tab][Tab]n[Tab].[Tab][Tab]p[Tab].[Tab][Tab]sh[Tab]

• The command line interface keeps a history of the commands that you typed. You can
get to the command history by pressing <arrow-up> key on your keyboard. Pressing
<arrow-up> multiple times show you the older commands. You can ue press <arrow-
down> to go back to the more recent commands. Try it by pressing the <arrow-up> key
followed by the <enter> key. You will see that it repeats the monitor.processes.show
command from the previous exercise step:
<arrow-up><enter>
• Using CML filter table output:

MCO> monitor.processes.show class=slc
MCO> monitor.processes.show class=slc instance=slc-00
• Try with other classes, like sol, smpp, etc. Use the textbook or the Operator Manual
for reference.
• To invoke cml help feature:

MCO> .help
MCO> .help /d
MCO> .help /s
MCO> .help /c
MCO> .help /o
MCO> .help monitor.processes
• To display all attributes of a feature:

MCO> @sdr.oam
State, counters related to a particular feature.
MCO> @sdr_conf.oam
Configuration parameters related to a particular feature.
Note that attributes of type file are not shown.
• Because we started cml with the cmln command we enable logs:

MCO> .log
• Start / Stop / Restart

Using CML (process specific):
MCO> monitor.processes.start instance=restgw-00
MCO> monitor.processes.restart instance=restgw-00
MCO> monitor.processes.stop instance=restgw-00
NOTE: Stopping or restarting MCO processes deliberately may cause system

inconsistence. Do not stop or restart any process unless it is necessary. Use the
textbook or the Operator Manual for reference.
• Disable logs:
MCO> .nolog
Now try the start command again and notice that no event logs are displayed.
MCO> monitor.processes.start instance=restgw-00
MCO> .log
MCO> monitor.processes.stop instance=restgw-00
• Audit-log checking:
MCO> audit.log.show count=10
See also /var/opt/mco/audit.log
MCO> #tail -19 /var/opt/mco/audit.log

Note for your reference:

For the user mco, the right to show the audit log is by default denied.
To ease this exercise, the right was granted in the lab with the following commands:
MCO> access_conf.roles[administration].rights.add id=3 right=allow resource=audit.log.show
MCO> access.config.reload
• Attributes of type Scalar values:

Example that shows the version of the sol:
MCO> common.version.get /class=sol
• Attributes of type Tables:

Example that shows the configured sdr streams:
MCO> sdr_conf.streams.show
• Attributes of type Files:

Examples that show the content of the primary routing rules and edit them:
MCO> primary_routing_conf.rules.checkout /show
MCO> primary_routing_conf.rules.checkout /edit
Type :q! to exit the vi editor without making changes.
• Enable/Disable (default) activation hints for configuration commands:

MCO> .hint
Notice the following message that is printed that reminds you to activate the changes (if
any):
Note: For the configuration change to take effect, you need to perform the
following activation steps.
Activation steps:
- step 1:
Execute 'primary_routing.config.reload'
MCO> .nohint
Notice that now you are not reminded to perform the activation steps.
2.3. Backup MCO
2.3.1. Task
Backup MCO configuration to backup directory created in the mco user home directory.
2.3.2. Solution
As mco, create backup directory:
$ mkdir /home/mco/backup
run the configuration backup:
$ /opt/mco/cluster/scripts/backup_config.sh /home/mco/backup
Note the default storage location for backups is a separate file system mounted on /backup.
2.4. Getting familiar with REST API
2.4.1. Task
On mco system start the REST gateway and then using REST API accomplish following actions:
• check status of all MCO processes,

• check status of processes in SLC class,

• check status of slc-00 instance of SLC processes,
o try the same for SOL and SMPP processes
• start snmpag-00 instance,
• restart snmpag-00 instance,
• stop snmpag-00 instance,
• show the version of the SOL,
• shows the configured SDR streams,
• show the content of the Primary Routing rules,
• edit them,
• store the new configuration in ZOOKEEPER
2.4.2. Solution
IP address: 172.21.12.7
• Start the RESTGW

$ cml monitor.processes.start class=restgw
• Show status:
$ curl -g -X GET "http://127.0.0.1:8080/v1/monitor/processes"
• Using bash (process specific):

$ curl -g -X GET "http://127.0.0.1:8080/v1/monitor/processes?q[class]=slc"
$ curl -g -X GET "http://127.0.0.1:8080/v1/monitor/processes?q[class]=slc&q[instance]=slc-00"
• Try with other classes, like sol, smpp, etc. Use the textbook or the Operator Manual
for reference.
• Start / Stop / Restart

Using REST API (process specific):
$ curl -g -X POST "http://127.0.0.1:8080/v1/monitor/processes/start" -d '{"instance": "snmpag-00"}'
$ curl -g -X POST "http://127.0.0.1:8080/v1/monitor/processes/restart" -d '{"instance": "snmpag-00"}'
$ curl -g -X POST "http://127.0.0.1:8080/v1/monitor/processes/stop" -d '{"instance": "snmpag-00"}'
NOTE: Stopping or restarting MCO processes deliberately may cause system

inconsistence. Do not stop or restart any process unless it is necessary. Use the
textbook or the Operator Manual for reference.
•Attributes of type Scalar values:

Example that shows the version of the sol:
$ curl -g -X GET "http://127.0.0.1:8080/v1/common/version?class=sol"
• Attributes of type Tables:

Example that shows the configured sdr streams:
$ curl -g -X GET "http://127.0.0.1:8080/v1/sdr_conf/streams"
• Attributes of type Files:

Examples that show the content of the primary routing rules and edit them:
$ curl -g -X GET "http://127.0.0.1:8080/v1/primary_routing_conf/rules/data" > /tmp/primary_routing_conf_rules.cnf
$ vi /tmp/primary_routing_conf_rules.cnf
Type :q to exit the vi editor without making changes.
$ curl -g -X PUT "http://127.0.0.1:8080/v1/primary_routing_conf/rules/data" --data-binary @/tmp/primary_routing_conf_rules.cnf

3. SOL Exercises
3.1. SS7oIP configuration
3.1.1. Task
From CML verifying the SS7oIP configuration. Match the values you find in the configuration files with
the laboratory diagram in chapter 1 MCO Lab Overview.
• Check the Application Server Groups (ASG) configuration,

• Check the Signalling Gateway Groups (SGG) configuration,
• Check the Routing Keys (RK) configuration,
• Check the PointCode to Signalling Gateway Group (PC2SGG) configuration,
• Check the Global Title to Signalling Gateway Group (GT2SGG) configuration,
• Check the Global Title Translation (GTT) configuration,
• Check the Entity Sets (ES) configuration
3.1.2. Solution
From CML verifying the SS7oIP configuration. Match the values you find in the configuration files with
the laboratory diagram in chapter 1 MCO Lab Overview.
• Some files are defined per
• Login to MCO via ssh instance. For those files, the
IP address: 172.21.12.7 global file will be empty.
User/Password: mco/mco • Some files are defined only
$ cml globally, so the instance file
• Check the Application Server Groups (ASG) configuration: will be empty.
MCO> ssp_conf.ssp_asg_enabled.get
MCO> ssp_conf.ssp_asg_enabled.get /class=sol /instance=sol-00
MCO> ssp_conf.ssp_asg_file.checkout /show
MCO> ssp_conf.ssp_asg_file.checkout /show /class=sol /instance=sol-00
• Check the Signalling Gateway Groups (SGG) configuration:

MCO> ssp_conf.ssp_sgg_enabled.get
MCO> ssp_conf.ssp_sgg_file.checkout /show
MCO> ssp_conf.ssp_sgg_file.checkout /show /class=sol /instance=sol-00
• Check the Routing Keys (RK) configuration:

MCO> ssp_conf.ssp_rk_enabled.get
MCO> ssp_conf.ssp_routing_key_file.checkout /show
MCO> ssp_conf.ssp_routing_key_file.checkout /show /class=sol /instance=sol-00
• Check the PointCode to Signalling Gateway Group (PC2SGG) configuration:

MCO> ssp_conf.ssp_pc2sgg_enabled.get
MCO> ssp_conf.ssp_pc2sgg_file.checkout /show
MCO> ssp_conf.ssp_pc2sgg_file.checkout /show /class=sol /instance=sol-00
• Check the Global Title to Signalling Gateway Group (GT2SGG) configuration:

MCO> ssp_conf.ssp_gt2sgg_enabled.get
MCO> ssp_conf.ssp_gt2sgg_file.checkout /show
MCO> ssp_conf.ssp_gt2sgg_file.checkout /show /class=sol /instance=sol-00
• Check the Global Title Translation (GTT) configuration:

MCO> ssp_conf.ssp_gtt_enabled.get
MCO> ssp_conf.ssp_gtt_file.checkout /show
MCO> ssp_conf.ssp_gtt_file.checkout /show /class=sol /instance=sol-00

• Check the Entity Sets (ES) configuration:

MCO> ssp_conf.ssp_es_enabled.get
MCO> ssp_conf.ssp_ent_set_file.checkout /show
MCO> ssp_conf.ssp_ent_set_file.checkout /show /class=sol /instance=sol-00
3.2. Verifying SS7 AF configuration
3.2.1. Task
Check the SS7 AF configuration:
• default_network_domain
• network_domains
• conversion_profiles
• conversion rules of a conversion_profile
• default_conv_profile
• map_version_profiles
• map_version rules of a map_version_profile
3.2.2. Solution
• Check the SS7 AF configuration: The training lab does not
MCO> ss7_conf.default_network_domain.get
have any conversion profile
MCO> ss7_conf.network_domains.show
MCO> ss7_conf.conversion_profiles.show configured.
MCO> ss7_conf.conversion_profiles[<name>].conversion_rules.show
MOC> ss7_conf.default_conv_profile.get
MCO> ss7_conf.map_version_profiles.show
MCO> ss7_conf.map_version_profiles[mapVerProfile0].map_version_rules.show
3.3. Verifying SS7 linkset status
3.3.1. Task
Check the SS7 linkset status:
• SCTP association
• Signalling gateway processes
3.3.2. Solution
• Check the SS7 linkset status:
MCO> ssp.assoc_stat.show
MCO> ssp.sgp_stat.show
3.4. Suspend a link (requires MCO V3.0-11.01 or higher)
3.4.1. Task
• De-activate the association between MCO node and SGP-0.
• Restart SOL on MCO node and see that the association is always established after
restart.
The IPSMGW training lab has
3.4.2. Solution SGP-MSC and SGP-HLR
instead of SPG-0 configured.
• De-activate the association between MCO node and SGP-0:
MCO> ssp.assoc_link.disable name=SGP-0 /instance=sol-00

• Restart SOL on MCO node and see that the association is always established after
restart:
MCO> monitor.processes.restart instance=sol-00
3.5. IPSMGW - MO Router configuration
3.5.1. Task
• Verify MO Router configuration to route a MO-SM (MO-FW-SM MAP operation) to a
desired SMSC (profiles, destination table).
• Verify MO Router configuration to forward a MO-SM using the configured

SIP_SMSC_P profile, check the RBDL rules for the MO Router AF.
3.5.2. Solution
• Verify MO Router configuration to route a MO-SM (MO-FW-SM MAP operation) to a
desired SMSC:
MCO> mortr_conf.profiles.show
+=====================================================+
|name |primary |backup |
+-----------------+-----------------+-----------------+
|SIP_SMSC |SIP_SMSC_P | |
+=====================================================+
MCO> mortr_conf.destination_table[SIP_SMSC_P].show /wide

==============<table item>================
name: SIP_SMSC_P
destinations:
algorithm: undefined
routing_method: route_to_fixed
route_type: plmn
imsi_inclusion: yes
enforce_tcap_segm: no
cdpa_ton: preserve
cdpa_npi: preserve
cdpa_gt_digits: 701222222
cdpa_ri: route_on_pc
cdpa_tt: 0
cdpa_pc: -2
cdpa_ssn: 8
cdpa_sk: -2
sc_ton: preserve
sc_npi: preserve
sc_addr: ?
weight: 100
rate: 100
gsm_timeout: 0
ifx_domain_id: 0
rules: ???
• Verify MO Router configuration to forward a MO-SM using the configured

SIP_SMSC_P profile, check the RBDL rules for the MO Router AF:
MCO> mortr_conf.rules.checkout /show
Rule_set SIP_SMSC
{
Rule SIP_SMSC_1
Actions SELECT_ROUTE(SIP_SMSC)
}
Eval_on ROUTE_MO_MSG { SIP_SMSC }

3.6. SMSC - Testing SOL with Store and Forward (P2P)

• To simulate traffic in the SOL component, use the BAF server. Keep CML open, start a
new Putty window and ssh to the BAF server (root/root). Next type:
xmsbaf# cd baf
xmsbaf# ./start_baf_fr
Welcome to Baphomet network [ver=fcdb7df11224f762cd68a04c496a4fec68893de4]
[MNGR] registered modules: 0340 24.011 amqp camel cmd diameter dns gti gtp ifx
kafka latency ldap m2pa m3ua map map_ansi mtp2 mtp3 null pcap sccp sctp sctp2
sip smpp tcap tcap_ansi tcap_trx tcp tcp_old telesvc ucp udp x0340
type help to get help
HH:MM:SS.XXXXXX|INFO| sctp| listening on port 7777
HH:MM:SS.XXXXXX|INFO| sctp| new connection from 172.21.12.7:17024
HH:MM:SS.XXXXXX|INFO| m3ua| ASP UP
HH:MM:SS.XXXXXX|INFO| m3ua| ASP ACTIVE
HH:MM:SS.XXXXXX|INFO| m3ua| rc=7024
baf>
The BAF tool emulates the GSM MSC and HLR. It connects to the MCO with SCTP
and M3UA.
In CML you now see events that show that the association is established.
| INFO | ssp.ASSOCI_UP | MMM DD HH:MM:SS XX | sol:sol-00.XXXX | SSP:
Association (0) established for SGP-0 (0) |
| INFO | ssp.SGP_RK | MMM DD HH:MM:SS XX | sol:sol-00. XXXX |
State: Inactive, correlation: SGP_RK, info: SGP-0 (0) is waiting for activation
for RK-ITU (0) |
| INFO | ssp.SGP_ASP | MMM DD HH:MM:SS XX | sol:sol-00. XXXX |
State: Up, correlation: SGP_ASP, info: ASP is up for SGP-0 (0) |
| INFO | ssp.SGP_RK | MMM DD HH:MM:SS XX | sol:sol-00. XXXX |
State: Active, correlation: SGP_RK, info: SGP-0 (0) is active for RK-ITU (0) |
| INFO | class_sol.SS7_STATE | MMM DD HH:MM:SS XX | sol:sol-00. XXXX |
Instance state ssp.0 changed from ACTIVATED to ACTIVE |
• Now you can simulate GSM traffic on the MCO. The command submit will cause BAF
to originate a MO_FW_SM from MSC towards the MCO in Store & Forward message
mode.
baf> submit
------------------------------------------------------------------------
HH:MM:SS |S| Map.op: mofwsm ; 0340.op: submit
originator: 1.1.420111222333
recipient: 1.1.420444222333
text(ud): test text
------------------------------------------------------------------------
HH:MM:SS |R| Map.op = mo-resp
sccp.cgpa= NI:INT RI:GT GTI:4 PC:701 SSN:8 TT:0 NPI:1 ES:2 NAI:4
ADDR:3110000008
sccp.cdpa= NI:INT RI:GT GTI:4 PC:301 SSN:8 TT:0 NPI:1 ES:2 NAI:4
ADDR:3112345678
------------------------------------------------------------------------
HH:MM:SS |R| Map.op = sri
sccp.cgpa= NI:INT RI:SSN GTI:0 PC:701 SSN:8 TT:- NPI:- ES:- NAI:- ADDR:-
sccp.cdpa= NI:INT RI:GT GTI:4 PC:- SSN:6 TT:0 NPI:1 ES:2 NAI:4
ADDR:420444222333
map.msisdn=1.1.420444222333
------------------------------------------------------------------------
HH:MM:SS |S| SRI-SM response
map.op: sri-resp
map.nnn: 1.1.420999888777 map.imsi: 204691234567
------------------------------------------------------------------------
HH:MM:SS |R| Map.op = mtfwsm
ADDR:420999888777
map.imsi= 204691234567
map.sca= 1.1.420111111111
map.smrpui=040C9124101122323300001210625104004009F4F29C0EA297F174

0340.TPDU = mtfwsm
------------------------------------------------------------------------
HH:MM:SS |S| MT-FWSM response tcap.error: 0
*************************
** incoming MT details **
*************************
oa = 1.1.420111222333
pid=0 dcs=0 udl=9
ud = test text
In the BAF output you can see:

1. baf (as A-MSC) sends MO-FWSM to MCO.
2. baf (as A-MSC-A) receives acknowledgement for the MO-FWSM from MCO.
3. baf (as HLR) receives SRI-SM for the recipient address from step 1 from MCO.
4. baf (as HLR) sends SRI-SM response for the recipient address to MCO.
5. baf (as B-MSC) receives MT-FWSM for the recipient from step 1 from MCO.
6. baf (as B-MSC) sends MT-FWSM response.
Figure 3-1 Call flow Store and Forward
• Now send another message and provide arguments to submit to specify an originator
address, recipient address and message text. Use the help command to learn which
syntax to use.
baf> help
3.7. Router - Testing SOL with MO Proxy
3.7.1. Task
• Set the correct message mode for MO Proxy call flow.
• Activate the new configuration.
• Submit a message from baf and you will see that the message is forwarded as
MOFWSM.
• Change back the configuration for the next exercise.
• Activate the new configuration.
3.7.2. Solution
• To set the correct call flow the MO_PROXY message mode must be set:
MCO ifxb_conf.rules.checkout /edit
--
! operations from network Insert hash # to comment out the rule
#Rule_set MO_FWSM_IND { Rule MO_IND Actions SEND_IFX_TRIGGER(SF) }
…

Rule_set MO_FWSM_IND { Rule MO_IND Actions SEND_IFX_TRIGGER(MO_PROXY) }

…
Eval_on MO_FWSM_IND_TRIGGER { MO_FWSM_IND }
• Activate the new configuration Remove # to uncomment the rule

MCO> ifxb.config.reload
>> sol/sol-00 ifxb.config.reload :
OK
| INFO | mco.AF_RELOAD_OK | Mar DD HH:MM:SS PM | sol:sol-
00.XXXX | ifxGwBeta AF has been reloaded successfully |
• Submit a message and you will see that the message is forwarded as MOFWSM.
baf> submit
------------------------------------------------------------------------
originator: 1.1.420111222333
recipient: 1.1.420444222333
text(ud): test text
------------------------------------------------------------------------
HH:MM:SS |R| Map.op = mofwsm
sccp.cdpa= NI:INT RI:GT GTI:4 PC:- SSN:8 TT:1 NPI:1 ES:2 NAI:4 ADDR:3110000008
map.msisdn=1.1.420111222333
map.sca= 1.1.420111111111
map.smrpui=01000C91244044223233000009F4F29C0EA297F174
------------------------------------------------------------------------
HH:MM:SS |S| MO-FWSM response
da = 1.1.420444222333
pid=0 dcs=0 udl=9
ud = test text
------------------------------------------------------------------------
sccp.cgpa= NI:INT RI:GT GTI:4 PC:301 SSN:8 TT:0 NPI:1 ES:2 NAI:4
ADDR:3112345678
sccp.cdpa= NI:INT RI:GT GTI:4 PC:301 SSN:8 TT:0 NPI:1 ES:2 NAI:4
ADDR:3112345678
Figure 3-2 Call flow MO Proxy

Remove hash # to uncomment the rule
• Change back the configuration for the next exercise.
MCO ifxb_conf.rules.checkout /edit
--
Rule_set MO_FWSM_IND { Rule MO_IND Actions SEND_IFX_TRIGGER(SF) }
#Rule_set MO_FWSM_IND { Rule MO_IND Actions SEND_IFX_TRIGGER(MO_PROXY) }
Eval_on MO_FWSM_IND_TRIGGER { MO_FWSM_IND }
• Activate the new configuration Insert # to comment out the rule

MCO> ifxb.config.reload
>> sol/sol-00 ifxb.config.reload :
OK
| INFO | mco.AF_RELOAD_OK | Mar DD HH:MM:SS PM | sol:sol-
00.XXXX | ifxGwBeta AF has been reloaded successfully |

4. Delivery Layer Exercises

4.1. IPSMGW - Primary Routing
If you want to send messages from IMS to GSM or vice versa, first you must understand how MCO is
routing messages.
4.1.1. Task
• From CML verify the primary routing rules.
4.1.2. Solution
IP address: 172.21.12.7
$ cml
• Verify the routing rules:
This rule routes

Table TABLE_PRIMARY_ROUTING (B_ADDR, A_NETWORK_TYPE)
{ messages to GSM
{ "?.?.*", GSMoIP} : Actions SET_DEST_PLMN(GSM)
{ "?.?.*", GSM } : Actions SET_DEST_PLMN(GSMoIP) This rule routes
}
Eval_on PRIMARY_ROUTING { TABLE_PRIMARY_ROUTING }
messages to IMS
4.2. IPSMGW - Backup Routing

In case the delivery to IMS fails, MCO uses the backup routing to deliver the message to GSM according
to the configured rules.
4.2.1. Task
• From CML verify Backup Routing rules.
4.2.2. Solution
• From CML verify Backup Routing rules.:
MCO> backup_routing_conf.rules.checkout /show
Rule_set IPSMGW
{
Rule FALLBACK
Conditions B_NETWORK_TYPE eq GSMoIP
Actions SET_DEST_PLMN(GSM)
}
Eval_on BACKUP_ROUTING { IPSMGW }

4.3. SMSC - Provision SMPP Large Account
4.3.1. Task
• From CML check existing ESMEs.
• Activate hint mode in CML.
• Add new ESME:
o Name: SMPP_TRAIN
o Password: pooky
o Type: SALA
o Mode: normal
o LA number:1234
4.3.2. Solution
IP address: 172.21.12.7
$ cml
• Next exercise is to configure a new LA. First check the ESMEs that already exist and
then create a new one:
MCO> smpp_gw_conf.esmes.show
>> / smpp_gw_conf.esmes.show :
+================================================================================================+=====
|name |password |type |mode |remote_smsc |lasn | ...
+---------------------------------+-----------+------+-----------+-------------------------+-----+-----
|SMPP_TEST |pooky |sala |normal | |910 | ...
+================================================================================================+=====
MCO> .hint
Cparam activation hints enabled.
MCO> smpp_gw_conf.esmes.add name=SMPP_TRAIN password=pooky type=sala mode=normal lasn=1234

>> / smpp_gw_conf.esmes.add :
OK
Note: For the configuration change to take effect, you need to perform the
following activation steps.
Activation steps: Reopen the affected connection. If the connection is used in
SET_DEST_LA command, restart SLC.
Note: The SMPP_TRAIN application does not connect to MCO now. In this case you are
not required to take action to have the configuration change to take effect.
Reopen the connection can be done like this:
MCO> smpp_gw.sessions.close esme=SMPP_TRAIN
4.4. SMSC - Testing the Delivery Layer - AOMT (A2P), Store, Retry, Alerts
• Be sure that your CML (Putty window 1) is running. If not, start it:
Start a new Putty window and login to MCO via ssh
IP address: 172.21.12.7
Open CML interface:
Putty window (1): $ cml
• Be sure that your BAF GSM tool (Putty window 2) is running. If not, start it:
Start a new Putty window and login to XMSBAF via ssh
IP address: 172.21.12.6
Start BAF GSM:
Putty window (2): xmsbaf# cd baf

Putty window (2): xmsbaf# ./start_baf_fr

• Be sure that your BAF SMPP tool (Putty window 3) is running and bound. If not, start
it:
IP address: 172.21.12.6
Start BAF SMPP:
Putty window (3): xmsbaf# ./start_smpp
HH:MM:SS.XXXXXX|INFO| tcp2| Established connection to 172.21.12.7:9000
SMPP>
The BAF tool now emulates a SMPP ESME (large account). It connects to the MCO
with TCP.
To open the SMPP session we need to send an SMPP Bind towards the MCO. The
command bind will cause BAF to send a SMPP Bind for the existing SMPP_TEST
towards the MCO in transceiver mode.
Putty window (3): SMPP> bind

HH:MM:SS |S| Bind system_id:SMPP_TEST system_type: bind_type:trx
HH:MM:SS |R| Bind_resp status:0x0 [Ok]
SMPP>
• Check that the SMPP session is open on the MCO:

Putty window (1): MCO> smpp_gw.sessions.show
>> smpp/smpp-00 smpp_gw.sessions.show :
+================================================================================================+=====
|esme |session |mode |version_in |version_out |uptime |req_in |req_out |bind_type | ...
+----------+--------------+-------+-----------+------------+-------+-------+--------+----- ------+-----
|SMPP_TEST |smpp-00:XXXXX |normal |5.0 | |YY |1 |0 |TRANSCEIVER | ...
+================================================================================================+=====
• Now you can simulate SMPP traffic on the MCO. The command submit will cause BAF
to originate a SUBMIT_SM towards MCO.
Putty window (3): SMPP> submit

HH:MM:SS |S| Submit src:1.1.333444555 dst:1.1.777666555 text:nazdar bazar
HH:MM:SS |R| Submit_resp status:0x0 [Ok] message_id:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
HH:MM:SS |R| Deliver src:1.1.777666555 dst:1.1.333444555
text:id:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX sub:001 dlvrd:001 submit date:XXXXXXXXXX done
date:XXXXXXXXXX stat:DELIVRD err:000 text:nazdar bazar
HH:MM:SS |S| Deliver_resp status:0x0 [Ok]
SMPP>
(Putty window 2)
------------------------------------------------------------------------
map.msisdn=1.1.777666555
------------------------------------------------------------------------
map.op: sri-resp
map.nnn: 1.1.420999888777 map.imsi: 204691234567
------------------------------------------------------------------------
map.imsi= 204691234567
map.sca= 1.1.1234567890

map.smrpui=24099133434455F50000121062611411400CEEB09E1C9683C4617D580E
0340.TPDU = mtfwsm
------------------------------------------------------------------------
*************************
*************************
oa = 1.1.333444555
pid=0 dcs=0 udl=12
ud = nazdar bazar
In the BAF output you can see that

1. (Putty window 3) baf (as SMPP LA) submits SUBMIT_SM message to MCO
2. (Putty window 3) baf (as SMPP LA) receives response to SUBMIT_SM from MCO.
3. (Putty window 2) baf (as HLR) receives SRI-SM for the recipient address from MCO.
4. (Putty window 2) baf (as HLR) sends response to SRI-SM for the recipient address to MCO
5. (Putty window 2) baf (as B-MSC) receives MT-FWSM for the recipient address from MCO.
6. (Putty window 2) baf (as B-MSC) sends MT-FWSM response to MCO
7. (Putty window 3) baf (as SMPP LA) receives DELIVER_SM with a delivery notification with
status delivered from MCO.
8. (Putty window 3) baf (as SMPP LA) sends a DELIVER_SM response to MCO.
• If you want to have your messages stored, stop the GSM BAF by pressing <control-c>:
Putty window (2): baf> ^C
xmsbaf#
• Send a new SMPP SUBMIT_SM using the submit command in the SMPP BAF
session.
Putty window (3): SMPP> submit
HH:MM:SS |S| Submit src:1.1.333444555 dst:1.1.777666555 text:nazdar
bazar
HH:MM:SS |R| Submit_resp status:0x0 [Ok]
message_id:74EC7B917196E761831E00000000730B
• Check that the message is stored in the MCO:

Putty window (1): MCO> fms.sme.show
>> kernel/kernel-00 fms.sme.show :
+=====================================================================================================================================+=====
|id |state |msg_count |retry_time |error |head_msg |importance | ...
+--------------+------------+----------+-----------------------------+--------------+-------------------------------------+-----------+-----
|1.1.777666555 |unavailable |1 |YYYY-MM-DD HH:MM:SS.XXX+TTTT |T_SYSTEM (34) |74ec7b91-7196-e761-831e-00000000730b |0 | ...
+=====================================================================================================================================+=====
You may want to use the /wide option to see all columns.
With the command .nolog you can suppress the events regarding the association being
down.
• Now connect the GSM BAF again:

Putty window (2): xmsbaf baf# ./start_baf_fr
type help to get help
HH:MM:SS.XXXXXX|INFO| sctp| listening on port 7777
HH:MM:SS.XXXXXX|INFO| sctp| new connection from 172.21.12.7:17024
HH:MM:SS.XXXXXX|INFO| m3ua| ASP UP
HH:MM:SS.XXXXXX|INFO| m3ua| ASP ACTIVE
HH:MM:SS.XXXXXX|INFO| m3ua| rc=7024
• Send an alert to the MCO to trigger the retry of the delivery to the mobile:

Putty window (2): baf> alert 1.1.777666555

------------------------------------------------------------------------
HH:MM:SS |S| Map.op: alert
map.msisdn= 1.1.777666555
map.sca= 1.1.420111111111
------------------------------------------------------------------------
HH:MM:SS |R| Map.op = alert-resp
sccp.cgpa= NI:INT RI:GT GTI:4 PC:701 SSN:8 TT:0 NPI:1 ES:2 NAI:4 ADDR:3110000008
sccp.cdpa= NI:INT RI:GT GTI:4 PC:301 SSN:8 TT:0 NPI:1 ES:2 NAI:4 ADDR:3112345678
------------------------------------------------------------------------
map.msisdn=1.1.777666555
------------------------------------------------------------------------
map.op: sri-resp
map.nnn: 1.1.420999888777 map.imsi: 204691234567
------------------------------------------------------------------------
map.imsi= 204691234567
map.sca= 1.1.1234567890
0340.TPDU = mtfwsm
------------------------------------------------------------------------
*************************
*************************
oa = 1.1.333444555
pid=0 dcs=0 udl=12
ud = nazdar bazar
(Putty window 3)
SMPP>
text:id:E6BB8A31F05FEB61947400000000F007 sub:001 dlvrd:001 submit date:2101261704
done date:2101261704 stat:DELIVRD err:000 text:nazdar bazar
4.5. SMSC - Primary Routing AOAT (A2A)
4.5.1. Task
Route traffic from SMPP_TRAIN ESME to SMPP_TEST ESME
• Check Primary Routing rules

• Check if both ESMEs exist
• Bind both ESMEs
Hint:
Keep CML (Putty 1), the GSM BAF tool (Putty 2) and the SMPP BAF tool (Putty 3)
open, start a new Putty window (Putty 4) and ssh to the BAF server server
(root/root):
Putty window (4): SMPP> @ctrl cmd set_prompt SMPP_TRAIN
Putty window (4): SMPP_TRAIN> @set system_id SMPP_TRAIN
Putty window (4): SMPP_TRAIN> bind
• Send message from SMPP_TRAIN ESME to SMPP_TEST ESME

Hint:
Putty window (4): SMPP_TRAIN> submit 1.1.910
4.5.2. Solution
IP address: 172.21.12.7
$ cml
• If you want to send messages from a LA to another LA (A2A or Application to
Application), first we have to understand how MCO is routing messages. Verify the
routing rules:
Rule_set LASN
{
Rule LA_ROUTING
Conditions B_AIM_NAME not present,
ADDR_LENGTH(B_ADDR) ge 3,
ADDR_LENGTH(B_ADDR) le 6
Actions APPLY_LASN This rule routes
}
Rule_set RARR messages to LA
{
Rule RANGE_ROUTING
Conditions B_AIM_NAME not present,
ADDR_LENGTH(B_ADDR) ge 3,
ADDR_LENGTH(B_ADDR) le 6
Actions APPLY_RARR
}
Rule_set DEFAULT This rule routes
{
Rule DEFAULT messages to GSM
Conditions B_AIM_NAME not present
Actions SET_DEST_PLMN(GSM)
}
Eval_on PRIMARY_ROUTING { LASN, RARR, DEFAULT }
• Make sure that the two ESMEs SMPP_TEST and SMPP_TRAIN that you created in
the previous exercise exist:
MCO> smpp_gw_conf.esmes.show
>> / smpp_gw_conf.esmes.show :
+=========================================================================+=====
|name |password |type |mode |remote_smsc |lasn | ...
+-----------+-----------+------+-----------+------------------------+-----+-----
|SMPP_TEST |pooky |sala |normal | |910 | ...
|SMPP_TRAIN |pooky |sala |normal | |1234 | ...
+=========================================================================+=====
• Keep CML (Putty 1), the GSM BAF tool (Putty 2) and the SMPP BAF tool (Putty 3)
open, start a new Putty window (Putty 4) and ssh to the BAF server server (root/root):
Putty window (4): SMPP> @ctrl cmd set_prompt SMPP_TRAIN
Putty window (4): SMPP_TRAIN> @set system_id SMPP_TRAIN
Putty window (4): SMPP_TRAIN> bind
HH:MM:SS |S| Bind system_id:SMPP_TRAIN system_type: bind_type:trx

HH:MM:SS |R| Bind_resp status:0x0 [Ok]
• Submit message from SMPP_TRAIN to SMPP_TEST:

Putty window (4): SMPP_TRAIN> submit 1.1.910


message_id:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
text:id:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX sub:001 dlvrd:001 submit
date:XXXXXXXXXX done date:XXXXXXXXXX stat:DELIVRD err:000 text:nazdar bazar
Putty window (3)

SMPP>
HH:MM:SS |R| Deliver src:1.1.333444555 dst:1.1.910 text:nazdar bazar
In the BAF output you can see that the MCO

1. first acknowledges the SUBMIT_SM to the SMPP_TRAIN ESME.
2. send an DELIVER_SM with the short message to the SMPP_TEST ESME.
3. send an DELIVER_SM with a delivery notification with status delivered to the SMPP_TRAIN
ESME.
4. receives an DELIVER_SM response from BAF SMPP_TRAIN ESME.
4.6. SMSC - Retry Mechanism Configuration
4.6.1. Task
• Verify the retry schemes using CML.
• Check the levels for a retry scheme of your choice.
• Check retry profiles.
•
4.6.2. Solution
• The retry mechanism consists of retry schemes and retry profiles. Verify the retry schemes by CML:
MCO> scheduler_conf.retry_schemes.show
>> / scheduler_conf.retry_schemes.show :
+=====================================================================================+
|# |id |description |levels |priority |
+---+-------------------------+-------------------------------------+-------+---------+
| 0 |SCHEME_T_DEFAULT |Default retry scheme |... |1 |
| 1 |SCHEME_P_PERMANENT_ERROR |Default handling of permanent errors |... |0 |
| 2 |SCHEME_T_BARRED |Default handling for T_BARRED |... |1 |
....
....
|19 |SCHEME_P_APP_NW_PERM |Default handling for P_APP_NW_PERM |... |0 |
|20 |SCHEME_T_V5_DEFAULT |Temporary error - retry the queue |... |0 |
+=====================================================================================+
• The Levels sub-table describes associated retry actions. Check the levels for a specific retry scheme:
MCO> scheduler_conf.retry_schemes[SCHEME_T_SYSTEM].levels.show
>> / scheduler_conf.retry_schemes[SCHEME_T_SYSTEM].levels.show :
+==========================================+
|# |level |period |attempts |action |
+--+---------+---------+---------+---------+
|0 |1 |5m |1 |retry |
|1 |2 |15m |1 |retry |
|2 |3 |1h |72 |retry |
|3 |4 |6h |0 |retry |
+==========================================+
• A retry profile determines mapping of an error code (resulting from an outgoing attempt) to a retry
scheme:
MCO> scheduler_conf.retry_profiles.show
>> / scheduler_conf.retry_profiles.show :
+=======================================================================================================+
|# |id |description |priority |busy_sme_timeout |retry_new_msg |status_map |error_map |
+--+---+------------------------------+---------+-----------------+--------------+-----------+----------+
|0 |0 |Default retry profile 0 |0 |85 |yes |... |... |
|1 |1 |SMSC V5 similar retry profile |0 |85 |yes |... |... |
+=======================================================================================================+

MCO> scheduler_conf.retry_profiles[0].error_map.show
>> / scheduler_conf.retry_profiles[0].error_map.show :
+==============================================+
|# |error |scheme |
+----+---------------+-------------------------+
| 0 |P_UNKNOWN |BY_STATUS |
| 1 |P_GPRS_UNKNOWN |BY_STATUS |
| 2 |T_UNIDENTIFIED |SCHEME_T_ABSENT_DETACHED |
....
....
|179 |L_NO_RECEIVER |SCHEME_L_NO_RECEIVER |
|180 |L_NO_AIM |SCHEME_L_NO_AIM |
+==============================================+
4.7. SMSC - Recipient Range Routing RARR – Challenge!

In this exercise we create a range from 200 to 210 that is routed to SMPP_TEST. With baf we submit to
0.1.200 and this will arrive in the SMPP_TEST.
Please use what you have learned to complete this exercise.
4.8. SMSC - Message Store – Challenge!

In this exercise we set the baf to return absent subscriber. Next, we submit 4 short messages from
different originators. Now we query the message store to find the message id of the third message and
delete it. After changing baf to no longer return absent subscriber, we remove the head of the
subscriber queue and see the second and forth message delivered.
4.9. SMSC - Inter SMSC Router (ISR)
4.9.1. Task
• Start Remote SMSC simulator
• Configure Remote SMSC.
• Send MO traffic to Remote SMSC.
• Send AO traffic to Remote SMSC.
• Send MT traffic from Remote SMSC.
• Investigate available counters.
• Investigate SDRs
4.9.2. Solution
• Keep CML (Putty 1), the GSM BAF tool (Putty 2) and the SMPP BAF tool (Putty 3) open, start a new
Putty window (Putty 4) and use ssh to connect to the BAF server (root/root). Start the SMPP SMSC
simulator:
Putty window (4):
xmsbaf# cd baf
xmsbaf# cp -p start_smpp start_smpps
xmsbaf# vi start_smpps
LD_LIBRARY_PATH=/root/baf /root/baf/baf /root/baf/scripts/smpps.lua
:wq
xmsbaf# ./start_smpps
baf> @ctrl cmd set_prompt SMSC1
SMSC1>
Tip for Experts:

The SMPP SMSC simulator does not show received SMPP messages in detail.
Optional, you can enable verbose output:
SMSC1> @set verbose 1
Note that the exercise example output below does not contain this verbose output.
• Add the Remote SMSC"

MCO> smpp_gw_conf.remote_smscs.add name=SMSC1
>> / smpp_gw_conf.remote_smscs.add :
OK
• Add an endpoint with one connection to the Remote SMSC:

MCO> smpp_gw_conf.remote_smscs[SMSC1].endpoints.add id=1 host=xmsbaf port=9000 count=1
>> / smpp_gw_conf.remote_smscs[SMSC1].endpoints.add :
OK
• Add the ISR ESME
MCO> smpp_gw_conf.esmes.add name=ISR1 password=pooky mode=ISR type=gala remote_smsc=SMSC1 sessions_max=1
>> / smpp_gw_conf.esmes.add :
OK
• Use RARR to route traffic to the ISR LA

MCO> primary_routing_conf.rarr_table.add from=200 to=299 aim=ISR1
>> / primary_routing_conf.rarr_table.add :
OK
• Activate the RARR

MCO> primary_routing.config.reload
| INFO | mco.AF_RELOAD_OK | Jun 24 10:12:20 PM | ucp:ucp-00.14710 |
PRIMARY_ROUTING AF has been reloaded successfully |
| INFO | mco.AF_RELOAD_OK | Jun 24 10:12:20 PM | msg_T:msg_T-00.1658 |
| INFO | mco.AF_RELOAD_OK | Jun 24 10:12:20 PM | smog:smog-00.1652 |
| INFO | mco.AF_RELOAD_OK | Jun 24 10:12:20 PM | smpp:smpp-00.30796 |
MCO> primary_routing.config.
>> gsm_0340_T/gsm_0340_T-00 primary_routing.config.reload :
OK
>> msg_T/msg_T-00 primary_routing.config.reload :
OK
>> smog/smog-00 primary_routing.config.reload :
OK
>> smpp/smpp-00 primary_routing.config.reload :
OK
>> ucp/ucp-00 primary_routing.config.reload :
OK
| INFO | mco.AF_RELOAD_OK | Jun 24 10:12:20 PM | gsm_0340_T:gsm_0340_T-
00.1656 | PRIMARY_ROUTING AF has been reloaded successfully |
• Activate the ISR LA

MCO> monitor.processes.restart class=smpp instance=smpp-00
>> / monitor.processes.restart :
OK
Putty window (4)

SMSC1>
[tcp] New connection from 172.21.12.7:XXXXX
[RECEIVE] op: bind ; status: 0 ; seq: XX ; socket: Y
[SENDING] op: bind_resp ; status: 0 ; seq: XX ; socket: Y
• Send MO traffic to Remote SMSC.

Putty window (3):
baf> submit 1.1.123456789 1.1.200
------------------------------------------------------------------------
originator: 1.1.123456789
recipient: 1.1.200
text(ud): test text
------------------------------------------------------------------------
sccp.cgpa= NI:INT RI:GT GTI:4 PC:701 SSN:8 TT:0 NPI:1 ES:2 NAI:4 ADDR:3110000008
sccp.cdpa= NI:INT RI:GT GTI:4 PC:301 SSN:8 TT:0 NPI:1 ES:2 NAI:4 ADDR:3112345678

Putty window (4)

SMSC1>
[RECEIVE] op: submit ; status: 0 ; seq: XX ; socket: Y
[SENDING] op: submit_resp ; status: 0 ; seq: XX ; socket: Y
[SENDING] op: deliver ; status: nil ; seq: YY ; socket: Y
[RECEIVE] op: deliver_resp ; status: 0 ; seq: YY ; socket: Y

Putty window (3)
SMPP> submit 1.1.200
HH:MM:SS |R| Submit_resp status:0x0 [Ok] message_id:AEAAECE638D5EB619FD500000000CE13
text:id:AEAAECE638D5EB619FD500000000CE13 sub:001 dlvrd:001 submit date:2106250009 done
date:2106250009 stat:DELIVRD err:034 text:nazdar bazar
Putty window (4)

SMSC1>
[SENDING] op: deliver ; status: nil ; seq: YY ; socket: Y
[RECEIVE] op: deliver_resp ; status: 0 ; seq: YY ; socket: Y
• Send MT traffic from Remote SMSC.

Putty window (4)
SMSC1> deliver
[SENDING] op: deliver ; status: nil ; seq: XX ; socket: Y
[RECEIVE] op: deliver_resp ; status: 0 ; seq: XX ; socket: Y
Putty window (3):
baf>
------------------------------------------------------------------------
map.msisdn=1.1.777666555
------------------------------------------------------------------------
map.op: sri-resp
map.nnn: 1.1.420999888777 map.imsi: 204691234567
------------------------------------------------------------------------
map.imsi= 204691234567
map.sca= 1.1.1234567890
0340.TPDU = mtfwsm
------------------------------------------------------------------------
*************************
*************************
oa = 1.1.333444555
pid=0 dcs=0 udl=12
ud = nazdar bazar
• Create script that shows scheduler ISR related counters.

MCO> #grep isr /opt/mco/share/oam/all.oam > /opt/mco/share/oam/isr.oam
• Run the script to show the scheduler ISR counters.
MCO> @isr
>> kernel/kernel-00 scheduler.isr_dupl_id.get = 0
>> kernel/kernel-00 scheduler.isr_expired.get = 0
>> kernel/kernel-00 scheduler.isr_pending.get = 0
>> kernel/kernel-00 scheduler.isr_receipts_intermediate.get = 0
>> kernel/kernel-00 scheduler.isr_receipts_late.get = 0
>> kernel/kernel-00 scheduler.isr_receipts_total.get = 2
• To see statistics for all ISR LAs, use the following command:
MCO> smpp_gw.esmes.show mode=ISR
>> smpp/smpp-00 smpp_gw.esmes.show :
+======================================================================================+====
|esme |mode |sessions_used |req_in |req_out |bind_tx_used |bind_rx_used |bind_trx_used |...
+-----+-----+--------------+-------+--------+-------------+-------------+--------------+----

|ISR1 |isr |1 |3 |61 |0 |0 |0 |...

+======================================================================================+====
Tip: try the same command with the /wide option.
• To see active ISR sessions, use the following command:

MCO> smpp_gw.sessions.show mode=ISR
>> smpp/smpp-00 smpp_gw.sessions.show :
+======================================================================================+====
|esme |session |mode |version_in |version_out |uptime |req_in |req_out |bind_type |...
+-----+-----------+-----+-----------+------------+-------+-------+--------+------------+----
|ISR1 |smpp-00:XX |isr | |3.4 |YYY |3 |ZZ |TRANSCEIVER |...
+======================================================================================+====
Tip: try the same command with the /wide option.
• After you performed Exercise 5.1, repeat sending ISR traffic and investigate the SDRs. Look for the
infoISR flag in the MSG_FLAGS field.
4.9.3. Task
• Make(break) Remote SMSC simulator stop sending Delivery Reports back to MCO.
• Confirm that MCO is waiting for the Delivery Report from the Remote SMSC
4.9.4. Solution
• Keep CML (Putty 1), the GSM BAF tool (Putty 2), the SMPP BAF tool (Putty 3) and the SMPP SMSC
BAF tool (Putty 4) open.
• Change the SMPP SMSC simulator script by commenting out sending the delivery report.
Putty window (4):
SMSC1> ^C
xmsbaf# vi /root/baf/scripts/smpps.lua
-- if m:get_param('smpp.reg') == '1' then
-- delivery_receipt(m)
-- end
:wq
xmsbaf# ./start_smpps
baf> @ctrl cmd set_prompt SMSC1
SMSC1>
• Send AO traffic to Remote SMSC and notice that no delivery report gets delivered.
Putty window (3)
SMPP> submit 1.1.200
HH:MM:SS |R| Submit_resp status:0x0 [Ok] message_id:AEAAECE638D5EB619FD500000000CE13
Putty window (4)

SMSC1>
• Look in the message store and confirm that MCO waits for the delivery response from ISR1.
Putty window (1)
MCO> fms.msg.show recip=1.1.200 /wide
>> kernel/kernel-00 fms.msg.show :
==============<table item>================
id: xxxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxxxx
orig: 1.1.333444555
recip: 1.1.200
sme: 1.1.200
dst_aim: ISR1
submit_time: YYYY-MM-DD HH:MM:SS.000+0200
scheduled_time: never
expiry_time: YYYY-MM-DD HH:MM:SS.000+0200
last_delivery_time: YYYY-MM-DD HH:MM:SS.000+0200
final_time: never

priority: 0
importance: 0
text: ***
content: ***
udh: ***
suspend_time: never
nr_suspended: 0
await_receipt: yes
isr_id: 012345
MCO> fms.sme.show id=1.1.200 /wide

>> kernel/kernel-00 fms.sme.show :
==============<table item>================
id: 1.1.200
state: scheduled
msg_count: 1
attempts: 3
last_delivery_time: YYYY-MM-DD HH:MM:SS.000+0200
retry_time: YYYY-MM-DD HH:MM:SS.000+0200
profile: 0
scheme: SCHEME_L_NO_AIM (14)
level: 1
error: ERROR_NONE (0)
head_msg: xxxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxxxx
importance: 0
ir_profile: -1
ir_state: N/A
msg_sched: 0
msg_susp: 0
msg_dsdr: 0
msg_isr: 1

5. SLC Exercises
5.1. SMSC, IPSMGW - SDR Function
SLC applies on each message a (per-message) configurable sequence of functions. SDR function is used
to create SDR records. It is always executed as the last one in the sequence (even if the message is
blocked) and cannot be disabled.
5.1.1. Task
• Find out which current SDR streams are enabled using CML.
• Show the current state of the CSV_SM SDR stream.
• Enable the CSV_SM SDR stream.
• Modify SDR RBDL rules to create SDR files for CSV_SM stream and activate the new
configuration.
• Submit a SMS and check the results.
• Check if the created SDR files are stored on the disk.
• Inspect the content of a stored SDR file.
• Decode a stored SDR file using csv_decode.py script.
• Check the statistics of the sdr application function.
5.1.2. Solution
IP address: 172.21.12.7
$ cml
• View the current SLC SDR streams configuration to see which streams are enabled:
MCO> sdr_conf.streams#avro.show
• Show the current state of the CSV_SM SDR stream is disabled:

MCO> sdr_conf.streams[CSV_SM].show /wide
• Enable the CSV_SM SDR stream:

MCO> sdr_conf.streams.modify name=CSV_SM enabled=yes flush_per_record=yes
• Modify and save the SDR RBDL: Insert exclamation mark ! to comment out the
MCO> sdr_conf.rules.checkout /edit
action PASS_ON
Rule_set DO_SM_SDR
{
Rule SM_SDR
! Actions PASS_ON
Remove ! to uncomment the
Actions CREATE_SDR(CSV_SM) action CREATE_SDR
}
MCO> sdr.config.reload
| INFO | mco.AF_RELOAD_OK | MMM DD HH:MM:SS XX | slc:slc-
00.XXXX | SDR AF has been reloaded successfully |
MCO> sdr.config.reload>> slc/slc-00 sdr.config.reload :
OK
• Submit a short message

IPSMGW - Execute this bullet if you are SMSC, Router or Firewall - Execute
in the IPSMGW training only! this bullet if you are in the SMSC,
Router or Firewall training!
Submit a SMS and check the results: Submit a SMS via GSM BAF and check
1. Be sure that MSC and HLR scripts the results:
are running. 1. baf> submit

2. On the MSC window, send a MO-SM:

msc> mo 1.1.703012345
• On MCO node, quit CML and check the results on the recently created SDR file:
MCO> .exit
[mco@mco1:~]$ ls -la /sdr/*.curr
-rw-rw-rw-. 1 mco mco 5321 MMM DD HH:MM /sdr/CSVSM_mco1.localdomain_<file_creation_time>.csv.curr
Note: if the file size is zero more than 5 minutes elapsed.

Try find /sdr -type f -not -empty -ls to find the right file based on the file
modification date. This file will already be closed and its file name will no longer contain
the .curr postfix.
[mco@mco1:~]$ cat /sdr/CSVSM_mco1.localdomain_<file_creation_time>.csv.curr

'RecordNr','LOG_TIME()','SP_ID()','MSG_TYPE()','SEG_TYPE()','MSG_ID()','MSG_PARE
NT_ID()','MSG_MODE()','MSG_STATUS()','REJECT_REASON()','REJECT_METHOD()','SM_ERR
OR()','SUBMIT_TIME()','EXPIRY_TIME()','DELIVERY_TIME()','DONE_TIME()','PRIORITY(
)','NOTIF_REQ()','NR_DEL_ATT()','NR_DEL_SUB_ATT()','UDH_SRC_PORT()','UDH_DST_POR
T()','A_ADDR()','A_UADDR()','A_TADDR()','B_ADDR()','B_UADDR()','B_TADDR()','A_IM
SI()','B_IMSI()','MAP_A_IMSI()','MAP_B_IMSI()','A_AIM_NAME()','B_AIM_NAME()','A_
NETWORK_TYPE()','B_NETWORK_TYPE()','S_CGPA()','D_CDPA()','D_CGPA()','A_HLR_CGPA(
)','B_HLR_CGPA()','A_HLR_MAP_ERROR()','B_HLR_MAP_ERROR()','A_SC()','B_SC()','MAP
_ERROR()','MAP_CAUSE()','A_MSC()','A_SGSN()','B_MSC()','B_SGSN()','SM_TEXT(h)','
SM_TEXT_LENGTH()','SM_CONTENT()','SM_CONTENT_SIZE()','SM_CONTENT_ENCODING()','SM
_UDH()','SM_UDH_SIZE()','SM_UDH_SEG()','SM_UDH_SEG_SIZE()','UDH_MASK()','UDH_MAS
K_SEG()','SM_SEGMENT_MAX()','SM_SEGMENT_CURRENT()','SM_SEGMENT_REF()','A_COS()',
'B_COS()','S_COS()','D_COS()','A_DOMAIN()','B_DOMAIN()','S_DOMAIN()','D_DOMAIN()
','MSG_FLAGS()','DETECT_FLAGS()','DETECT_FLAGS_SEG()','RBDL_FLAGS()','RBDL_FLAGS
_SEG()','BILLING_DATA()','BILLING_DATA_SEG()','S_ADDR()','D_ADDR()','A_BILL_ID()
','S_CONS()','D_CONS()','PROTOCOL_ID()','TRACK_DATA()','TELESERVICE()','SERVICE_
TYPE()','MSG_REF()','S_LADDR()','D_LADDR()','MSG_TEMPLATE_ID()','A_LASN()','B_LA
SN()','ADT_PROFILE()','IMPORTANCE()','CBN()','A_MIN(a)','B_MIN(a)','B_ESN()','OD
B()','CALL_BARRING_FEATURES()','IR_PROFILE()','SCHEDULED_TIME()','F_IMSI()','SUS
PEND_TIME()','NR_SUSPENDED()','N_ADDR()','NETWORK_ERROR()','REMOTE_MSG_ID()','ST
AT_DATA()','A_LAI()','PARENT_MSG_STATUS()','SYSTEM_NAME()'
'0','XXXXXXXXXX','1','1','2','XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX','','1','0','',''
,'','XXXXXXXXXX','XXXXXXXXXX','','','','00','','','','','1.1.420111222333','1.1.
420111222333','1.1.420111222333','1.1.420444222333','1.1.420444222333','1.1.4204
44222333','','','','','gsm_0340','gsm_0340','1','1','0.301.8.0.1.1.3112345678','
0.701.8.0.1.1.3110000008','','','','','','1.1.420111111111','1.1.420111111111','
','','','','','','746573742074657874','9','746573742074657874','9','1','','','',
'','0000000000000000','0000000000000000','','','','0000000000000000','0000000000
000000','0000000000000000','0000000000000000','','','','','0000000000000000','00
00000000000000','0000000000000000','0000000000000000','0000000000000000','000000
0000000000','0000000000000000','','','','','','0','','','','0','','','','','','0
','','','','','','','','','','','','','','','','','','','MCO-SMSC-A'
....
....
The SDRs can be viewed more human readable.

Notice the similarities in the SDR pairs, but the difference in SEG_TYPE.
The first SDR pair differs from the second SDR pair in SP_ID, MSG_STATUS and B_IMSI.
[mco@mco1:~]$ csv_decode.py /sdr/CSVSM_mco1.localdomain_<file_creation_time>.csv.curr

================== /sdr/CSVSM_mco1.localdomain_<file_creation_time>.csv.curr -
START - ==================
------------------ Record from line: 2 ------------------
RecordNr: 0
LOG_TIME(): XXXXXXXXXX
SP_ID(): IN(1)
MSG_TYPE(): SM(1)
SEG_TYPE(): SEG(2)
MSG_ID(): XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
MSG_PARENT_ID(): N/A
MSG_MODE(): SF(1)
MSG_STATUS(): SUBMITTED(0)
....
....
REMOTE_MSG_ID(): N/A

STAT_DATA(): N/A
A_LAI(): N/A
PARENT_MSG_STATUS(): N/A
SYSTEM_NAME(): MCO-SMSC-A
================== /sdr/CSVSM_mco1.localdomain_<file_creation_time>.csv.curr -
END - ====================
Check the statistics of the sdr application function.
MCO> @sdr.oam
>> slc/slc-00 sdr.asn_records_total.get = 0
>> slc/slc-00 sdr.csv_records_total.get = 4
....
....
>> slc/slc-00 sdr.rules.show :
+======================================+=====
|stage |rule_set |rule |applied | ...
+----------+----------+-------+--------+-----
|IN_SEG |DO_SM_SDR |SM_SDR |1 | ...
|IN_MSG |DO_SM_SDR |SM_SDR |1 | ...
|IN_MSG |DO_BT_SDR |SM_SDR |1 | ...
....
....
|FINAL_SEG |DO_SM_SDR |SM_SDR |1 | ...
|FINAL_MSG |DO_SM_SDR |SM_SDR |1 | ...
|FINAL_MSG |DO_BT_SDR |SM_SDR |1 | ...
....
....
+======================================+=====
>> slc/slc-00 sdr.stats.extended_show :
+==========================================================================================================+=====
|name |records |records_buff |records_failed |records_total |files |open_time |status | ...
+-------+--------+-------------+---------------+--------------+------+-----------------------------+-------+-----
|CSV_SM |4 |0 |0 |4 |1 |YYYY-MM-DD HH:MM:SS.XXX+TTTT |UP | ...
+==========================================================================================================+=====
IPSMGW
In the IPSMGW lab you do not see 4 SDRs (records) but 6 SDRs in the statistics for
the CSV_SM stream.
5.2. Investigate SDRs from other exercises– Challenge!

In this exercise we investigate the generated SDRs from other exercises.
After you completed or repeated another exercise, please try to map the generated SDRs with the use
case of that exercise.
In case of exercise 6, please try to find the messages where the blocking profile changed the sequence
of the message (the blocking profile rejected the message).

6. Firewall - Traffic Shaping Exercises

6.1. XMS GUI - Simple Rate Set up
In the next exercise you will configure the system to analyse the traffic and apply the Simple Rate
algorithm. The objective of this exercise is to rate the incoming messages from large account named
SMPP_TEST.
You will use the XMS GUI to configure the MCO.
To switch to Edit mode, click the pencil in the top bar.
The square around the pencil will turn red when edit mode is enabled.
To switch to the MCO application menu, click on the MCO application in the navigation bar.
To switch to the Selector page, click on the Selectors page item in the navigation bar.
To create a new selector, click on the + Add selector button on the top right corner of the work area.
To name the new selector, in the right bar click in the Name field and type SMPP_TEST. To describe the
new selector, click in the Description field and type Selects SMPP_TEST originating traffic. To add the
selector's condition, press the + Append condition button.
To select traffic originating from the SMPP_TEST large account, for the condition, choose A_AIM_NAME
from the pull-down menu. For the value, type SMPP_TEST.

To store the selector, press the Save button.
To switch to the Antiflood page, click on the Antiflood page item in the navigation bar.
To enable the Antiflood feature, press the ON button in the top right corner of the main area.
To confirm enabling the Antiflood feature, press the Continue button in the pop-up window in the main
area.
To create a new antiflood profile, press the + Add profile button in the main area.
To name the new profile, type SHAPE in the Name field in the right bar. To create the new profile, press
the Add Profile button in the right bar.

To apply the Antiflood profile on traffic that originates from the SMPP_TEST large account, in the
Selectors table, press the Apply checkbox on the right side of the SMPP_TEST selector. To let the
selector ignore traffic that does not originate from the SMPP_TEST large account, press the SKIP button
right to the label Fallback action: in the right bar.
To reject traffic that fails to pass the rate profile, press the REJECT button in the Block method section.
To be able to configure profile parameters, click on the > symbol in front of Profile settings to reveal the
settings.
To allow 5 message submissions in 1 minute, type in the Limit field the value 5, msg per to 1, the unit to
minutes and Burst to 5.

To be able to fingerprint profile parameters, click on the > symbol in front of Fingerprints to reveal the
settings.
Tick the checkbox of the Originator number In the Fingerprints section.

Press the Save changes button to store the profile.
To enable the new profile, press the ON button.

6.2. XMS GUI - Simple Rate Traffic generation

Submit 5 messages from the existing SMPP_TEST ESME towards MCO.
Note: the bind command may report "Already bound"
SMPP> bind
SMPP> submit
SMPP> submit
SMPP> submit
SMPP> submit
SMPP> submit
bazar
baf>
map operation = sri
cgpa= NI:INT RI:SSN GTI:0 PC:701 SSN:8 TT:- NPI:- ES:- NAI:- ADDR:-
cdpa= NI:INT RI:GT GTI:4 PC:- SSN:6 TT:0 NPI:1 ES:1 NAI:4 ADDR:777666555
msisdn=1.1.777666555
map operation = mtfwsm
cgpa= NI:INT RI:SSN GTI:0 PC:701 SSN:8 TT:- NPI:- ES:- NAI:- ADDR:-
cdpa= NI:INT RI:GT GTI:4 PC:- SSN:8 TT:0 NPI:1 ES:2 NAI:4 ADDR:420999888777
imsi= 123456766666
sca= 1.1.1234567890
smrpui=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
TPDU = deliver
oa = 1.1.333444555
pid=0 dcs=0 udl=12
ud = nazdar bazar
• Submit 1 more message. Now the submission fails and no SRI and MTFWM is
received.
SMPP> submit
bazar
HH:MM:SS |R| Submit_resp status:0x45 [Unknown status]
6.3. XMS GUI - Simple Rate Monitoring

In the main area the Detected per profile graph shows a spike.
The other graphs in the main area show msgs/sec. When you send more messages quickly after each
other (type quickly arrow up and enter multiple times), these graphs will also show a spike.

6.4. CML - Simple Rate Set up

Next exercise you will configure MCO to analyse the traffic and apply the Simple Rate algorithm. The
objective of this exercise is to rate the incoming messages from SMPP_TEST large account.
You will use the Command Line to configure the MCO.
6.4.1. Task
• Configure MCO to block messages exceeding the maximum allowed number of
submitted messages (1 message per 60 seconds, for simulation reason).
6.4.2. Solution
IP address: 172.21.12.7
$ cml
• Configure the rules to block messages exceeding the maximum allowed number of
submitted messages (1 message per 60 seconds, for simulation reason). Enable the
Antiflood feature and add new simple rate profile as the following example:
MCO> afl_conf.enabled.set value=yes
>> / afl_conf.enabled.set :
OK
MCO> afl_conf.simple_rate_profiles.show
>> / afl_conf.simple_rate_profiles.show :
+==========================================================================+
|name |active |fingerprint |rate |time |burst |
+---------------------+-------+------------+---------+-----------+---------+
|FAIR_USE |yes |${A_ADDR} |10 |900 |5 |
+==========================================================================+
MCO> afl_conf.simple_rate_profiles.add name=SHAPE_CLI fingerprint=\${A_ADDR} rate=1 time=60 burst=1

>> / afl_conf.simple_rate_profiles.add :
OK
• Add new rules at the end of the file to apply the conditions to the profile created in the
step before:
MCO> afl_conf.rules.checkout /edit
Rule_set SHAPE_CLI_BEFORE
{
Rule SHAPE_SMPP_TEST
Conditions A_AIM_NAME eq "SMPP_TEST"
Actions APPLY_AFL(SHAPE_CLI)
}
Rule_set SHAPE_CLI_AFTER
{
Rule SHAPE_BLOCK
Conditions BLACKLISTED(SHAPE_CLI) eq TRUE

Actions BLOCK (FAIR_USE)

}
Eval_on IN_MSG_BEFORE { SHAPE_CLI_BEFORE }
Eval_on IN_MSG_AFTER { SHAPE_CLI_AFTER }
• Modify the FAIR_USE blocking profile from detect to reject messages above the
allowed limit:
MCO> rbdl_conf.block_profiles.modify name=FAIR_USE method=reject
MCO> afl.config.reload
| INFO | mco.AF_RELOAD_OK | MMM DD HH:MM:SS XX | slc:slc-
00.XXXX | AFL AF has been reloaded successfully |
MCO> afl.config.reload>> slc/slc-00 afl.config.reload :
OK
6.5. CML - Simple Rate Traffic generation

• Submit 1 message from the existing SMPP_TEST ESME towards MCO.
Note: the bind command may report "Already bound".
SMPP> bind
SMPP> submit
bazar
baf>
------------------------------------------------------------------------
ADDR:777666555
map.msisdn=1.1.777666555
------------------------------------------------------------------------
map.op: sri-resp
map.nnn: 1.1.420999888777 map.imsi: 204691234567
------------------------------------------------------------------------
ADDR:420999888777
map.imsi= 204691234567
map.sca= 1.1.1234567890
0340.TPDU = mtfwsm
------------------------------------------------------------------------
*************************
*************************
oa = 1.1.333444555
pid=0 dcs=0 udl=12
• Submit 1 more message. Now the submission fails and no SRI and MTFWM is
received.
SMPP> submit
bazar
HH:MM:SS |R| Submit_resp status:0x45 [Unknown status]

6.6. CML - Simple Rate Monitoring
6.6.1. Task
• Check the Antiflood MCO counters
6.6.2. Solution
• Check the antiflood MCO counters.
MCO> @afl.oam
>> slc/slc-00 afl.req_block.get = 2
....
....
>> slc/slc-00 afl.req_block_reject.get = 1
.....
.....
>> slc/slc-00 afl.stats.show :
+===========================================================================+
|name |applied |blacklisted |det_per_sec |susp_per_sec |bl_per_sec |
+----------+--------+-------------+-------------+-------------+-------------+
|FAIR_USE |2 |0 |0 |0 |0 |
|SHAPE_CLI |2 |1 |0 |0 |0 |
+===========================================================================+

7. IPSMGW - SCN and PAG Exercises

7.1. Mobile-originated messages to SIP
7.1.1. SIP gateway identity configuration

IP address: 172.21.12.7
$ cml
• Check SIP gateway identity parameters via the following commands:
MCO> sip_gw_conf.local_address.get
MCO> sip_gw_conf.local_uri.get
MCO> sip_gw_conf.global_uri.get
7.1.2. SIP gateway listen configuration

• Check SIP gateway identity parameters via the following commands:
MCO> sip_gw_conf.listeners.show
7.1.3. Translation table (MO scenario)

• Check configuration of the translation table which is used to extract an originator
MSISDN address from a SIP/tel URI address:
MCO> adt_conf.uri_to_msisdn.show
7.2. Mobile-terminated messages to SIP
7.2.1. Translation table (MT scenario)

• Check the configuration table for a recipient MSISDN to SIP/tel URI address translation
for a SIP delivery with the following command:
MCO> adt_conf.msisdn_to_uri.show
7.2.2. Third party queries for SIP subscribers

• To enable third party database queries for SIP subscribers in case of SIP delivery do
the following:
Check PAG triggering configuration as follows:
MCO> smsoip_conf.rules.checkout /show
Choose between third party registration database (REDIS, HSS) for PAG:
1. For internal REDIS database enable usage of third-party database as follows:

MCO> pag_conf.tdb.set value=yes
2. For HSS database check the configurations of the diameter gateway:
MCO> diam_gw_conf.orig_host.get
MCO> diam_gw_conf.orig_realm.get
MCO> diam_gw_conf.channels[HSS_CHANNEL].connections.show

8. IPSMGW - Simulation Exercises

For a full understanding of IPSMGW message flows, its requests and responses, you may want to start a
tcpdump at the xmsbaf server to capture the traces and analyse them with Wireshark. For example, as
root user:
xmsbaf# tcpdump -i any -w /tmp/mosip.cap -s0
Please see exercise 9.3 Capturing network traffic for more instructions how to analyse traces with
Wireshark.
The xmsbaf server also has tshark tool installed, for local trace, without the need to retrieve the file
from the server. It can be useful for simple analysis. For example, you can try:
xmsbaf# tshark -i any -R "sccp or sip or diameter"
Next to the network traces, you may also want to evaluate the SDRs. If you skipped the SDR Function
exercise, you need to execute it first to enable SDR generation.
8.1. MO SIP
In this exercise, you will simulate messages originated from SIP (IMS domain) to the SMSC (GSM).
3. Be sure that all emulator scripts are running

4. On the IMS window send a SIP MO-SM
ims> sms 703222444

5. Check the results on IMS and SMSC windows.
a. What were the results?
ims> sms 703222444

HH:MM:SS |S| RP-DATA MESSAGE msisdn:703222444
HH:MM:SS |R| SIP_RESP MESSAGE msisdn:703222444 202 Accepted
HH:MM:SS |R| RP-ACK MESSAGE call-id:sm-submit-report
HH:MM:SS |S| SIP_RESP MESSAGE 200 OK
smsc>
HH:MM:SS |R| MO msisdn=1.1.703222444 sca=1.1.701222222
HH:MM:SS |S| MO-resp
b. Did the SMSC accept the MO-SM?
yes
8.2. MO GSM – MT SIP (Success)

You will simulate messages originated from a GSM user (GSM domain) and successfully delivered to a
SIP user (IMS domain).
Notice that the MCO acts as SMSC and IP-SM-GW in this exercise.
Be sure that all emulator scripts are running
On the MSC window, send a MO-SM:
msc> mo 1.1.703012345
Check the results on MSC, HSS and IMS windows.

What were the results?
msc> mo 1.1.703012345
HH:MM:SS |S| MO oa=1.1.703111222 da=1.1.703012345
HH:MM:SS |R| MO-resp
hss>
HH:MM:SS |R| User-Data-Request
public identity=sip:703012345@localdomain;user=phone
ims>
HH:MM:SS |R| RP-DATA MESSAGE
HH:MM:SS |S| RP-ACK MESSAGE
HH:MM:SS |R| SIP_RESP MESSAGE 202 Accepted
What kind of request the HSS received? What was the response for that request?
HSS received a User-Data-Request for a public identity. It responded with the S-CSCF name of the IMS
emulator and user state as online.
Did the SIP user receive the SM?
Yes, the SIP user got the SM in a RP-DATA.
8.3. MO GSM – MT SIP with fall back to GSM (Fail: SIP user not registered)
You will simulate messages submit from the GSM domain to a SIP user (which is not registered to IMS).
In this case, based on the current MCO configuration, MT messages are delivered to GSM instead.
Make the HSS simulator respond with unregistered (offline) to User-Data-Requests:
hss> @set userstate 0
Submit a message from the MSC window:
msc> mo 1.1.703012345
Check the results on MSC, HSS, HLR and IMS windows.
msc> mo 1.1.703012345
HH:MM:SS |S| MO oa=1.1.703111222 da=1.1.703012345
HH:MM:SS |R| MO-resp
HH:MM:SS |R| MT imsi=7777703012345 sca=1.1.700123456
HH:MM:SS |S| MT-resp

hlr>
HH:MM:SS |R| SRI msisdn=1.1.703012345
HH:MM:SS |S| SRI-resp imsi=7777703012345 nnn=1.1.702000002
ims>

Note that if you already sent an mo earlier, the MCO may use its SRI cache and will not send an SRI to
the HLR again. If you want to see the SRI sent, try with another MSISDN like mo 1.1.7036543210.
What kind of request HSS received? What was the response for that request?
HSS received a User-Data-Request for a public identity. It responded with the user state as offline.
Which MAP operation was received by the HLR?
Send Routing Information for Short Message (SRI)
Did the SIP user receive the SM? If not, did the GSM user receive the message?
No, the SIP user did not get the SM. Instead, the GSM user received the message.
8.4. MT GSM from SMSC to SIP User with fall back to GSM (Fail: SIP user not
registered)
In the previous exercise, you have sent a MO SMS from GSM. In the next exercise, you will originate a
MT SMS from the SMSC to a SIP user that is not registered on the IMS.
Submit a message from the SMSC window:
smsc> fsg 1.1.703111222
Check the results on all script windows.
smsc> fsg 1.1.703111222

HH:MM:SS |S| SRI msisdn=1.1.703111222
HH:MM:SS |R| SRI-resp imsi=7777703111222 nnn=1.1.700123456
HH:MM:SS |S| MT imsi=7777703111222
HH:MM:SS |R| MT-resp
msc>
HH:MM:SS |R| MT imsi=7777703111222 sca=1.1.701222222
HH:MM:SS |S| MT-resp
hlr>
hss>
ims>
Note: Network Node Number (NNN, current recipient location)
What kind of request HSS received? What was the response for that request?
HSS received a User-Data-Request for a public identity. It responded with the user state as offline.
What MAP operations was requested from the SMSC?
Send Routing Information (SRI)

Mobile Terminated Forward Short Message (MT-FWSM)
Did the SIP user receive the SM? If not, did the GSM user receive the message?
No, the SIP user did not get the SM. Instead, the GSM user received the message.
8.5. MT GSM from SMSC to SIP User (Success)

Now you will simulate the previous scenario, MT SMS from SMSC, to a registered SIP user.
Make the HSS respond with registered to User-Data-Requests:
Submit a message from the SMSC window:
smsc> fsg 1.1.703111222
Check the results on all scripts windows.
smsc> fsg 1.1.703111222

HH:MM:SS |S| SRI msisdn=1.1.703111222
HH:MM:SS |R| SRI-resp imsi=7777703111222 nnn=1.1.700123456
HH:MM:SS |S| MT imsi=7777703111222
HH:MM:SS |R| MT-resp
hlr>

ims>
HH:MM:SS |R| RP-DATA MESSAGE
HH:MM:SS |S| RP-ACK MESSAGE
HH:MM:SS |R| SIP_RESP MESSAGE 202 Accepted
Did the SIP user receive the SM?
Yes
Why is the nnn 1.1.702000002 that the HLR sent not the same as 1.1.700123456 the SMSC receives?
The MCO acts as home router and replaces the nnn with its own GT to intercept the MT.

9. MCO Troubleshooting
9.1. Service and Process Troubleshooting
9.1.1. Task
• Check if processes are running using CML.
• Check MCO status from bash.
9.1.2. Solution
IP address: 172.21.12.7
$ cml
• Check if processes are running in CML:
MCO> monitor.processes.show
• Check if service mco is running in BASH as user root:

# service mco status full
9.2. Collecting troubleshooting information for Mavenir Support Team
9.2.1. Task
• Check installed MCO version.
• Generate file with all possible information about MCO.
• Investigate the generated file.
9.2.2. Solution
• Check which MCO version is installed.
# rpm -q mco
• Create file to attach to support ticket (as user MCO!) :
$ /opt/mco/cluster/scripts/collect_info.sh
• Investigate the generated file.
$ cd /tmp
$ tar xvzf /tmp/mco-collect-info*
9.3. Capturing network traffic
9.3.1. Task
• Capture one message submit and delivery using tcpdump and analyse the file with
Wireshark. Check the content of each message.
Hint: You need to be logged in as user root.
9.3.2. Solution
• If you want to check the content of each message, try to run tcpdump and analyse the
file with Wireshark. You need to be logged in as user root.
# tcpdump -i any -w /tmp/dump.cap -s0
• Now you should reproduce the problem.

In our case, submit one message.

SMSC, Router or Firewall - Follow this IPSMGW - Follow this column if you are
column if you are in the SMSC, Router in the IPSMGW training only!
or Firewall training!
To simulate traffic in the SOL component, To simulate traffic in the SOL component,
use the XMSBAF server. Be sure that your use the XMSBAF server. Be sure that your
GSM BAF is running. If not, start it like MSC and HLR are running. If not, start it
this: like this:
A. Keep CML open, start a new Putty 1. Keep CML open, start a new Putty
window and ssh to the BAF server window and ssh to the BAF server
(root/root). (root/root).
B. Next type: 2. Next type:
xmsbaf# cd baf xmsbaf# ./msc.sh
xmsbaf# ./start_baf_fr 3. In another window start the HLR
script.
xmsbaf# ./hlr.sh
Now you can simulate GSM traffic on the Now you can simulate traffic on the MCO.
MCO. The command submit will cause The command mo will cause MSC to
BAF to originate a MO_FW_SM from MSC originate a MO_FW_SM from MSC
towards the MCO in Store & Forward towards the MCO. Check the results.
message mode. 4. Be sure that MSC and HLR scripts are
C. baf> submit running.
msc> mo 1.1.703012345
• Stop tcpdump after a few seconds. To stop tcpdump press <control-c>.

• Check the size of the dump file. Note that this file can get very big if you leave
Tcpdump running for a long time with a wide filter.
# ls -lah /tmp/dump.cap
• Now start WinSCP on your Windows system (not your laptop! but in the Training Lab)
and connect to the system via SSH
• Transfer the file from /tmp/dump.cap to the Windows Desktop
• Start Wireshark and open the dump.cap
• In the filter bar you can create filters to find the packet you want to see.
For example, you can type a protocol name like sctp or smpp. IFX-sm uses port 4397
• By expanding the packet in the bottom section, you can see the individual fields that
are part of the packet. With right mouse click on a field, you can create or expand your
filter for further investigation. The possibilities of Wireshark are outside the scope of this
training.
9.4. SOL Health Check
9.4.1. Task
• Check if SOL is running on all nodes.
• Perform SOL health-check for *no_mem, *sys_error and *timeout counters.
• Check SSP AF.
• Check SS7 AF.
9.4.2. Solution
• Check if SOL is running on all nodes:
MCO> monitor.processes.show class=sol
• SOL health-check(*):
MCO> @sol_af_state.oam
MCO> @sol_no_mem.oam
MCO> @sol_sys_error.oam

MCO> @sol_timeout.oam
(*) @sol_af_state.oam , @sol_no_mem.oam , @sol_sys_error.oam and @sol_timeout.oam

are customized scripts stored under /opt/mco/share/oam that automates the task of showing the SOL
health check related counters. These customized scripts were created using the following commands:
$ grep af_state.get /opt/mco/share/oam/all.oam > /opt/mco/share/oam/sol_af_state.oam
$ grep no_mem.get /opt/mco/share/oam/all.oam > /opt/mco/share/oam/sol_no_mem.oam
$ grep timeout /opt/mco/share/oam/all.oam | grep -v _conf | grep -v _gw | grep -v smsoip | grep -v afg | grep -v \
class_slc | grep -v atocache | grep -v smrt | grep -v is41 > /opt/mco/share/oam/sol_timeout.oam
$ grep sys_error /opt/mco/share/oam/all.oam > /opt/mco/share/oam/sol_sys_error.oam
• SSP AF Status:
MCO> ssp.assoc_stat.show
MCO> ssp.addr_stat.show
MCO> ssp.asp_stat.show
• SS7 AF Status:
MCO> @ss7.oam
9.5. SOL Tracing
9.5.1. Task
Use Trace Server to trace SOL. The trace server is a process that can be used for gathering of debugging
outputs from one or more processes running on a node or cluster.
9.5.2. Solution
Use Trace Server to trace SOL. The trace server is a process that can be used for
gathering of debugging outputs from one or more processes running on a node or cluster.
• Check if trace server is enabled:

MCO> debug.trcsrv_enabled.get
MCO> debug.trcsrv_enabled.set value=yes
MCO> debug.trcsrv_enabled.set value=yes /class=<component>
MCO> debug.trcsrv_ip.get
MCO> debug.trcsrv_port.get
Note: This is a temporary debug mode. The setting is reset after a component restart! No
writing on file system.
• After configure and enable the trace server, you have to enable the debug mode and
set the debug filters for tracing a specific component:
MCO> debug.enabled.get
MCO> debug.enabled.set value=yes /class=sol
>> sol/sol-00 debug.enabled.set :
OK
| WARNING | debug.DEBUG_ON | MMM DD HH:MM:SS XX | sol:sol-
00.XXXXX | Debugging is ON. |
For example, changing SOL SS7 AF filter:
MCO> debug.filter.modify mask=SOL_SS7_AF level=3 /class=sol
• Set SOL message debug filters(*):

MCO> @debug_sol_set.oam
• Run a trace server. To run a trace server in the direct mode, just invoke the echos (as
MCO user) and check the results:
$ echos
• Optionally you can also try the below command:

$ /opt/mco/bin/echos | cut -d '|' -f 1,2,3,7,8,9
• You can also write trace to file using the following script:
$ /opt/mco/bin/echos_to_file.sh
• Now you should reproduce the problem.

In our case, submit one message.
SMSC, Router or Firewall - Follow this IPSMGW - Follow this column if you are
column if you are in the SMSC, Router in the IPSMGW training only!
or Firewall training!
To simulate traffic in the SOL component, To simulate traffic in the SOL component,
use the XMSBAF server. Be sure that your use the XMSBAF server. Be sure that your
GSM BAF is running. If not, start it like MSC and HLR are running. If not, start it
this: like this:
A. Keep CML open, start a new Putty 1. Keep CML open, start a new Putty
window and ssh to the BAF server window and ssh to the BAF server
(root/root). (root/root).
B. Next type: 2. Next type:
xmsbaf# cd baf xmsbaf# ./msc.sh
xmsbaf# ./start_baf_fr 3. In another window start the HLR
script.
xmsbaf# ./hlr.sh
Now you can simulate GSM traffic on the Now you can simulate traffic on the MCO.
MCO. The command submit will cause The command mo will cause MSC to
BAF to originate a MO_FW_SM from MSC originate a MO_FW_SM from MSC
towards the MCO in Store & Forward towards the MCO. Check the results.
message mode. 4. Be sure that MSC and HLR scripts are
C. baf> submit running.
msc> mo 1.1.703012345
• Stop echos after a few seconds. To stop echos press <control-c>.

• To set the debug filters back to zero and disable the debug mode, run:
MCO> debug.filter.modify mask=SOL_SS7_AF level=0 /class=sol
MCO> debug.enabled.set value=no
• After finish debugging, clear the log filters (*):

MCO> @debug_sol_clear.oam
(*) @debug_sol_set.oam and @debug_sol_clear.oam are customized scripts stored under

/opt/mco/share/oam that automates the task of enabling and modifying the debug filters.
• More tracing information can be acquired by tracing specific interfaces. Try tcpdump
and check ifx, sccp, smpp and sctp protocols:
# tcpdump -i any -w /tmp/dump.cap -s0
9.6. Selective debugging – Challenge!

In this exercise we enable selective debugging and with RDBL we enable debugging only for messages
that originate from one subscriber. We enable trace masks with @debug_slc_set. With baf with
submit one message and trace the message as it passes through the MCO. Also submit a message from
another subscriber and witness that no debug messages are shown.
9.7. Custom log messages – Challenge!

Use the CUSTOM_LOG_INFO RBDL action to generate custom log with INFO severity.


10. Extra - Bash scripting

10.1. counter_monitor
When you want to monitor a specific attribute in bash you can on the XMS node.
• In the bash shell on the xmsbaf node issue the following command to get every 5
seconds a timestamp and the output of the monitor.process.show command printed on
screen:
xmsbaf ~# counter_monitor monitor.processes.show 5 1
20.06.2018 14:26:10
>> monitor/monitor-00 monitor.processes.show :
+=============================================================================================================+
|id |class |instance |state |reason |starts |uptime |pid |
+-------+-------------+-----------------+-------------+-------------------------+-------+-------------+-------+
|1 |orion |orion-00 |RUNNING |AUTO_START |1 |656971 |1526 |
|2 |oamzoo |cfggw-00 |RUNNING |AUTO_START |1 |656971 |1527 |
|3 |snmpifxag |snmpifxag-00 |RUNNING |AUTO_START |1 |656971 |1528 |
|4 |pusa |pusa-00 |RUNNING |AUTO_START |1 |656971 |1529 |
|5 |restgw |restgw-00 |DISABLED |MANUAL_STOP |0 |0 |0 |
|6 |ice |ice-00 |DISABLED |MANUAL_STOP |0 |0 |0 |
|7 |xms |xms-00 |RUNNING |AUTO_START |1 |656971 |1530 |
|8 |stats |stats-00 |RUNNING |AUTO_START |2 |656954 |1707 |
|9 |pgw |pgw-00 |DISABLED |MANUAL_STOP |0 |0 |0 |
+=============================================================================================================+
20.06.2018 14:26:14
+=============================================================================================================+
+-------+-------------+-----------------+-------------+-------------------------+-------+-------------+-------+
+=============================================================================================================+
^C
10.2. cml in scripts

When you want to write your own shell scripts and use cml you can.
• In the bash shell to get an overview of the command line parameters type:
$ cml --help
• To call cml with a specific command, provide it as argument like this:
$ cml monitor.processes.show

11. Extra - Blacklist log events in CLI

11.1. Disable stats.ALARM_CHANGE
Assume there is a specific event log appearing in the Command Line Interface that you want to
suppress because for example scheduled maintenance causes these event logs.
| INFO | stats.ALARM_CHANGE | Nov DD HH:MM:SS XM |

STATS:stats-II.PPPPP | Alarm SIGTRAN_SGP 0 on MCO-SMSC-A:sol-00 has changed to
CLEARED |

IP address: 172.21.12.7
$ cml
• In cml on the xmsbaf node issue the following command to filter out these event logs:
OAM> log_conf.blacklist.add id=2 log=stats.ALARM_CHANGE apply_cml=yes
• To activate this change leave and start cml:
OAM> .exit
xmsbaf ~# cml
• Verify the change that you made:
OAM> log_conf.blacklist[2].show
>> / log_conf.blacklist.show :
+==================================================================+
|id |log |apply_cml |apply_snmp |
+---------+---------------------------------+----------+-----------+
|2 |stats.ALARM_CHANGE |yes |no |
+==================================================================+
• Notice that now the stats.ALARM_CHANGE events are not shown anymore in cml.

12. XMS Access rights management

12.1. Create monitoring account and grant view rights on XMS
• Login as user root with password root using SSH (172.21.12.6).
• Add local account with initials wb and add the account to the monitoring group.
xmsbaf ~# adduser wb --shell /opt/xms/bin/cmdline.sh --groups monitoring
• Now set the password for the wb account to wb.

xmsbaf ~# echo wb:wb | chpasswd
• Login on the web interface with the newly created user.
• Once you login, you can view the MCO monitoring page only.
Now click on the XMS Application to switch to the XMS Application page

• Notice that the buttons to stop and start and the Explorer page are missing.
• Login as local user using ssh and notice that you skip the bash prompt and go straight
into the OAM command line interface.
login as: wb
--- Unauthorised access prohibited ---
This is a closed-access system. If you do not have permission
to access this system, then log off now. If you remain logged
on
you consent to monitoring of your actions.
wb@172.21.12.6's password:
Last login: Fri Nov 3 18:01:55 2017 from 10.29.251.60
,-.
/ | `. __..-,O
: | --''_..-'.'
| . .-' `. '.
: . .`.'
| `. / ..
| `. ' .
`, `. |
,|,`. `-.|
'.|| ``-...__..-`
| |
|__|
/||V
//||\\
// || \\
__//__||__\\__
'----mavenir---' oam
type .help to get help
OAM>
• Ask for a list of running processes.

OAM> monitor.processes.show
+=============================================================================================================+
+-------+-------------+-----------------+-------------+-------------------------+-------+-------------+-------+
+=============================================================================================================+
• When you try to make a modification, like restart a process you will get the
UNAUTHORIZED error.
OAM> monitor.processes.restart class=stats
>> orion/orion monitor.processes.restart :

UNAUTHORIZED
• Now exit from the command line interface and see that your SSH connection is
terminated.
OAM> .exit
Tip: if you want to change the group after you created the account you can using the usermod
command. For example:
# usermod --groups administration wb
# cml access.config.reload
12.2. Create administration account and grant modification rights on XMS –

Challenge!
Now create an account with your own initials and make it member of the administration group.
Use section 3.1, the textbook or the XMS Operation Manual for reference.
Question: what additional rights does this account have compared to the "wb" account from section
3.1?
Answer: The difference can be found with the following commands:
OAM> access_conf.roles[monitoring].rights.show
OAM> access_conf.roles[administration].rights.show
12.3. Create administration account and grant modification rights on MCO –

Challenge!
In the previous sections of this chapter, you configured access to XMS.
For MCO the same steps need to be repeated on all MCO nodes.

13. XMS Dashboard Exercises

13.1. Create Dashboard
Click on the pencil sign in the upper right corner to switch to edit mode.
Click on the plus sign next to Dashboards in the left Navigation bar to create a new dashboard:
Type a name for the new dashboard and click on the button "Create Dashboard".
13.2. Create Line chart widget.

Finish section 5.1 before starting with this section.
Click on the button "Add widget+"
Select widget "Chart"

Type as title XMS CPU user system waitio, select as Type Line chart, type as Unit name %,
set Time scale to 5 min, type as Y-axis fixed minimum value 0, type as Y-axis fixed maximum value
100.
Type as Legend key "xms cpu user", select as Managed element "XMS", as Measurement "sysinfo.cpu",
type as Column name "user", select as Row name "cpu".

Add a second measurement by clicking on the Plus icon.
Fill in the fields for the second measurement. Type as Legend key "xms cpu waitio", select as Managed
element "XMS", as Measurement "sysinfo.cpu", type as Column name "waitio", select as Row name
"cpu".

Add a third measurement by clicking on the Plus icon.
Fill in the fields for the third measurement. Type as Legend key "xms cpu system", select as Managed
element "XMS", as Measurement "sysinfo.cpu", type as Column name "system", select as Row name
"cpu".

Click on the "Create Widget" button
After about 10 seconds the graph starts showing data:

13.3. Create Counter widget

Finish sections 5.1 and 5.2 before starting with this section.
Click on the button "Add widget+"
Choose widget type "Counter", type as title "XMS CPU user", type as Unit name "%", type as Legend key
"xms cpu user", select as Managed element "XMS", type as Measurement "sysinfo.cpu", select as Class
and instance "monitor - All instances", type as Column name "user", select as Row name "cpu" and click
on the button "Create Widget".
The result should look like this:

13.4. Create Gauge chart widget.

Finish sections 5.1, 5.2 and 5.3 before starting with this section.
Add a third widget of type Gauge chart.

13.5. Create Area chart widget.

Finish sections 5.1, 5.2, 5.3 and 5.4 before starting with this section.
Click on the three small horizontal lines to open the pull-down menu. Next click on the "Copy this
widget" menu item.
Click on the three small horizontal lines to open the pull-down menu. Next click on the "Edit this
widget" menu item.

Click on the Type and change "Line chart" into "Area chart".
Click and tag the tick box "Stacked chart".
Next click on the button "Save".

13.6. Create Pie Chart widget

Finish sections 5.1, 5.2, 5.3, 5.4 and 5.5 before starting with this section.
Copy the Area chart, change the type to Pie chart and add a fourth measurement "xms cpu idle":

13.7. Create Table widget

Finish sections 5.1, 5.2, 5.3, 5.4, 5.5 and 5.6 before starting with this section.
Add a new widget, choose widget type "Table", type as title "XMS CPU", select as Managed element
"XMS", select as Measurement "sysinfo.cpu", Click and untag the tick boxs "lwm", "cpu", "state",
"hwm", "hostname", type for Number of lines to display "1" and click the button "Create Widget".

13.8. Move widgets.

Finish sections 5.1, 5.2, 5.3, 5.4, 5.5, 5.6 and 5.7 before starting with this section.
Move all widgets next to each other on two horizontal lines.
Click on the widget header and drag each widget to its place.


14. Extra - XMS GUI Troubleshooting

14.1. MCO - Monitoring Dashboard
The disk utilization widget in the MCO Monitoring Dashboard keeps saying "Waiting for
measurements".
Task for this exercise is to find out why and make it show the graph.
Hint: The disk layout of the deployed MCO in the training lab does not follow the required structure
written in the MCO Installation manual. There are two ways to solve it. One: change the widget
(recommended for this exercise). Two: change the disk layout.

15. Extra - Single Point of Access on XMS

15.1. cml --enable_spoa
The CLI has a specific setting that allows accessing multiple clusters (and products).
• First start the CLI on the xmsbaf node with the instruction to enable this feature. From
the bash shell on the xmsbaf node type the command:
cml --enable_spoa
• Notice that the prompt now looks different
• To see which clusters with their products are available issue the cluster command:
local_cluster:OAM> .cluster
• To select a specific cluster provide its name:
local_cluster:OAM> .cluster MCO-SMSC-A
• Now issue an command and see that it goes to this cluster:
MCO-SMSC-A:MCO> monitor.processes.show
• By adding additional clusters commands also go to multiple clusters
MCO-SMSC-A:MCO> .cluster + XMS
• Now issue an command again and notice that you now get two answers.
SPOA> monitor.processes.show
• It is also possible to prefix any command with the cluster name like this:
SPOA> XMS:monitor.processes.show
Question: what has changed to the log events that are printed on the screen?

16. Conditional - DISK_CAPACITY alarm from XMS

Application /var/log
16.1. Showing a counter on console
The XMS system in the training lab is deliberately not properly configured. As result after running for
some time, the following alarm may appear:
Now it is your job to solve the problem. Note that this scenario assumes you are trained in Linux
administration. Ask your trainer for explanation if you do not understand some of steps.
• Login with SSH to the XMS node that has the problem.
• Now first validate that indeed the disk is (almost) full.
xmsbaf ~# df -h /var/log
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/lvm01-varlog.vol 997M 997M 20K 100% /var/log
• Find out with files consume most disk space.
xmsbaf ~# ls -Ssh1 /var/log | head
total 568M
251M messages
131M daemon.log
8.0M messages-20171108.gz
8.0M daemon.log-20171108.gz
7.5M messages.2.gz
7.5M daemon.log.2.gz
6.1M messages-20171109.gz
6.1M daemon.log-20171109.gz
• Because the disk is 100% full we need to make room before we can run logrotate.
Therefor we move the archived log files to the /tmp filesystem.
mkdir /tmp/var_log_full_issue
mv /var/log/*.gz /tmp/var_log_full_issue
• Now we run logrotate manually to enforce house keeping for the syslog files.
/usr/sbin/logrotate -f /etc/logrotate.d/syslog
• Verify that the disk has room again:
xmsbaf ~# df -h /var/log
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/lvm01-varlog.vol 997M 433M 564M 44% /var/log
• Verify that the alarm is cleared:


17. SMS Firewall

17.1. Keyword filtering – Challenge!
17.1.1. Task
• Setup MCO to block all messages which have word “apple” in the content.
• Test it using baf
17.2. “Almost all together” – Challenge!
17.2.1. Task
• Setup MCO to block messages which content contains the word “balloon” and its variants like
“ball00n”, etc. and is send in volumes higher than 3 messages per minute for more than 2
minutes. The sender should be blocked for 3 minutes.
• Add custom parameter A_GT to be saved with the blacklist record. Check if it is stored with the
BL.

18. SMS over Diameter

18.1. SMS over Diameter (SF) – AOC Challenge!
18.1.1. Task
Setup MCO to be able to send (for subscribers in specific address group) and receive SMS over
Diameter.
Test it using baf
o Start baf for SGd
xmsbaf# cd baf/scripts
xmsbaf# LD_LIBRARY_PATH=/root/baf /root/baf/baf /root/baf/scripts/sgd.lua
▪ Listens on port 8001
o Start baf for S6c
xmsbaf# cd baf/scripts
xmsbaf# LD_LIBRARY_PATH=/root/baf /root/baf/baf /root/baf/scripts/s6c.lua
▪ Listens on port 8002
o To send MO message from SGd baf type ofr command
18.2. SMS over Diameter (MO proxy) – AOC Challenge!
18.2.1. Task
Modify the configuration to send (for subscriber from certain address group) message to “MME” in MO
proxy mode.

19. Intelligent retry

19.1. Intelligent retry – AOC Challenge!
19.1.1. Task
Setup MCO to be able to apply IR to messages for subscriber from a specific address group. The
order of the network is GSM, 4G, SMPP_TEST.
Test it using baf
19.2. Intelligent Retry (delivery failures) – AOC Challenge!
19.2.1. Task
Modify the behaviour of baf HLS and baf HSS(S6c) to send errors back to MCO explore IR behaviour.

20. End of Training clean up

20.1. Logout of the Windows desktop
1. When the training finishes, you please Logout from the Windows.
2. Now you can log out from Guacamole.

Appendix A

Appendix A OpenVPN - Training Lab Failback Access

A.1 Please skip this appendix if you use the web browser as described in chapter 1
If access to the training lab via web browser to https://training.mavenir.com does not work, this
appendix describes an alternative method that can be used as backup plan.
To access the training VMs from the Internet, run OpenVPN on your computer:
3. Trainer will provide the vpn-xxxxxxxx.zip file containing the OpenVPN client with valid certificates.
4. Unzip de file.
5. On \OpenVPNPortable directory you will find the OpenVPNPortable.exe file.
6. Right click the executable file and chose “Run as administrator”.
7. OpenVPN will start establishing connection to training lab network.
8. When the connection is successfully established, the client window is closed, and you see a message next
to your computer’s clock: “vpn-acision-GTT is now connected”.
9. Connect via Remote Desktop Connection to the assigned Student X (see table below) virtual machine with
password the same as the username:
- windows desktop: rdp student@172.31.10.X2
10. If step 7 was successful, now continue with section 1.2 step 4 and further inside the Remote Desktop
Connection session. Do not execute the remainder of this appendix.
11. If step 7 failed, you can continue with steps 12 and 13.
12. Connect via ssh to the assigned Student X (see table below) virtual machine with password the same as
the username:
- mco1: ssh mco@172.31.12.X7
- baf: ssh root@172.31.12.X6
13. Connect via Firefox or Chrome to the assigned Student X (see table below) XMS virtual machine:
- xms: http://172.31.12.X6 username root

A.1.1 Access to Virtual Machines in Training Lab via OpenVPN directly

Because the VPN does not connect you directly to your student environment, alternative IP addresses
are configured to select the correct lab and student environment.
Lab 1 Lab 2
Student MCO Node 1 XMS and BAF Windows MCO Node 1 XMS and BAF Windows
Number Desktop Desktop
1 172.31.212.17 172.31.212.16 172.31.210.12 172.31.12.17 172.31.12.16 172.31.10.12
2 172.31.212.27 172.31.212.26 172.31.210.22 172.31.12.27 172.31.12.26 172.31.10.22
3 172.31.212.37 172.31.212.36 172.31.210.32 172.31.12.37 172.31.12.36 172.31.10.32
4 172.31.212.47 172.31.212.46 172.31.210.42 172.31.12.47 172.31.12.46 172.31.10.42
5 172.31.212.57 172.31.212.56 172.31.210.52 172.31.12.57 172.31.12.56 172.31.10.52
6 172.31.212.67 172.31.212.66 172.31.210.62 172.31.12.67 172.31.12.66 172.31.10.62
7 172.31.212.77 172.31.212.76 172.31.210.72 172.31.12.77 172.31.12.76 172.31.10.72
8 172.31.212.87 172.31.212.86 172.31.210.82 172.31.12.87 172.31.12.86 172.31.10.82
9 172.31.212.97 172.31.212.96 172.31.210.92 172.31.12.97 172.31.12.96 172.31.10.92
10 172.31.212.107 172.31.212.106 172.31.210.102 172.31.12.107 172.31.12.106 172.31.10.102
11 172.31.212.117 172.31.212.116 172.31.210.112 172.31.12.117 172.31.12.116 172.31.10.112
12 172.31.212.127 172.31.212.126 172.31.210.122 172.31.12.127 172.31.12.126 172.31.10.122
The trainer informs you of your student number and to use Lab 1 or Lab 2.


Mco30 Boc All Tre-Book04

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Mco30 Boc All Tre-Book04

Uploaded by

Copyright:

Available Formats

MCO 3.

Copyright © Mavenir 2021

MCO 3.0 Operator Course Exercises Book v 12 Status: ISSUED

2. MCO Overview Exercises ......................................................................................... 17

3. SOL Exercises ........................................................................................................... 21

4. Delivery Layer Exercises ........................................................................................... 27

5. SLC Exercises ........................................................................................................... 39

MCO 3.0 Operator Course Exercises Book v 12 Status: ISSUED

6. Firewall - Traffic Shaping Exercises ........................................................................... 42

7. IPSMGW - SCN and PAG Exercises ............................................................................ 51

8. IPSMGW - Simulation Exercises ................................................................................ 52

10. Extra - Bash scripting ............................................................................................... 61

12. XMS Access rights management ............................................................................... 63

13. XMS Dashboard Exercises ........................................................................................ 66

MCO 3.0 Operator Course Exercises Book v 12 Status: ISSUED

13.7. Create Table widget ............................................................................................................................ 76

14. Extra - XMS GUI Troubleshooting ............................................................................. 79

15. Extra - Single Point of Access on XMS ....................................................................... 80

16. Conditional - DISK_CAPACITY alarm from XMS Application /var/log ......................... 81

17. SMS Firewall ............................................................................................................ 83

18. End of Training clean up........................................................................................... 86

Appendix A OpenVPN - Training Lab Failback Access ................................................. 88

MCO 3.0 Operator Course Exercises Book v 12 Status: ISSUED

Document Version History

Changes since last issue

MCO 3.0 Operator Course Exercises Book v 12 Status: ISSUED

Other Information: exercises_MCO_3.0.docx

MCO 3.0 Operator Course Exercises Book v 12 Status: ISSUED

1. MCO Lab Overview

1.2. SMSC, Router, Firewall - Lab layout and connections.

1.3. IPSMGW - Lab layout and connections.

MCO 3.0 Operator Course Exercises Book v 12 Status: ISSUED

MSC PC: 702

S-CSCF (IMS) SIP: scscf01.localdomain SIP SMPP port: 9000

XMS HTTPS port: 443 DNS

Figure 1-2 MCO IPSMGW Training Laboratory Architecture

1. Go to Mavenir Training Portal https://training.mavenir.com and click the link "Click

MCO 3.0 Operator Course Exercises Book v 12 Status: ISSUED

- mco1: ssh mco@172.21.12.7

- baf: ssh root@172.21.12.6

6. Connect via Firefox to the XMS virtual machine:

- xms url: http://172.21.12.6

MCO 3.0 Operator Course Exercises Book v 12 Status: ISSUED

To disconnect from the windows desktop:

MCO 3.0 Operator Course Exercises Book v 12 Status: ISSUED

1.5. SMSC, Router, Firewall - Emulators

1. First, connect via ssh to the BAF server using Putty:

send alert: alert [subscriber] [sc_address]

respond 27-Absent subscriber to SRI: @set sri_err 27

respond 9-Illegal subscriber to MT: @set mt_err 9

send sri: sri [subscriber] [sc_address]

send deliver: deliver [originator msisdn] [recipient imsi] [text]

MCO 3.0 Operator Course Exercises Book v 12 Status: ISSUED

example: deliver 1.1.444555666 333444555666 hihihi

4. Repeat steps 1 and 2 to open a second putty window.

to bind transceiver: bind

to submit message: submit [dst] [text] [src]

to respond to every deliver with status 8: @set deliver_err 8

1.6. Identifying windows

Prompt Command that you ran in that window

$ ssh to the MCO1 server using Putty or any other terminal

1.6.1. SMSC, Router, Firewall

MCO 3.0 Operator Course Exercises Book v 12 Status: ISSUED