You are on page 1of 39

Single RAN System, SRAN 20C,

Issue 04

Administration and
Troubleshooting of the BTS
Conversion Service

DN254557986
Issue 02
Approval Date 2020-07-24
Nokia condential
Administration and Troubleshooting of the BTS Conversion Service

Nokia is committed to diversity and inclusion. We are continuously reviewing our customer
documentation and consulting with standards bodies to ensure that terminology is inclusive
and aligned with the industry. Our future customer documentation will be updated
accordingly.

The information in this document applies solely to the hardware/software product


(“Product”) specied herein, and only as specied herein. Reference to “Nokia” later in this
document shall mean the respective company within Nokia Group of Companies with whom
you have entered into the Agreement (as dened below).

This document is intended for use by Nokia's customers (“You”) only, and it may not be used
except for the purposes dened in the agreement between You and Nokia (“Agreement”)
under which this document is distributed. No part of this document may be used, copied,
reproduced, modied or transmitted in any form or means without the prior written
permission of Nokia. If You have not entered into an Agreement applicable to the Product,
or if that Agreement has expired or has been terminated, You may not use this document in
any manner and You are obliged to return it to Nokia and destroy or delete any copies
thereof.

The document has been prepared to be used by professional and properly trained
personnel, and You assume full responsibility when using it. Nokia welcomes your comments
as part of the process of continuous development and improvement of the documentation.

This document and its contents are provided as a convenience to You. Any information or
statements concerning the suitability, capacity, tness for purpose or performance of the
Product are given solely on an “as is” and “as available” basis in this document, and Nokia
reserves the right to change any such information and statements without notice. Nokia has
made all reasonable efforts to ensure that the content of this document is adequate and
free of material errors and omissions, and Nokia will correct errors that You identify in this
document. Nokia's total liability for any errors in the document is strictly limited to the
correction of such error(s). Nokia does not warrant that the use of the software in the
Product will be uninterrupted or error-free.

NO WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO
ANY WARRANTY OF AVAILABILITY, ACCURACY, RELIABILITY, TITLE, NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, IS MADE IN RELATION TO THE
CONTENT OF THIS DOCUMENT. IN NO EVENT WILL NOKIA BE LIABLE FOR ANY DAMAGES,
INCLUDING BUT NOT LIMITED TO SPECIAL, DIRECT, INDIRECT, INCIDENTAL OR
CONSEQUENTIAL OR ANY LOSSES, SUCH AS BUT NOT LIMITED TO LOSS OF PROFIT,
REVENUE, BUSINESS INTERRUPTION, BUSINESS OPPORTUNITY OR DATA THAT MAY ARISE
FROM THE USE OF THIS DOCUMENT OR THE INFORMATION IN IT, EVEN IN THE CASE OF

2 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

ERRORS IN OR OMISSIONS FROM THIS DOCUMENT OR ITS CONTENT.

This document is Nokia proprietary and condential information, which may not be
distributed or disclosed to any third parties without the prior written consent of Nokia.

Nokia is a registered trademark of Nokia Corporation. Other product names mentioned in


this document may be trademarks of their respective owners.

Copyright © 2021 Nokia. Nokia condential.

Important Notice on Product Safety

This product may present safety risks due to laser, electricity, heat, and other
sources of danger.

Only trained and qualied personnel may install, operate, maintain or otherwise
handle this product and only after having carefully read the safety information
applicable to this product.

The safety information is provided in the Safety Information section in the “Legal,
Safety and Environmental Information” part of this document or documentation
set.

Nokia is continually striving to reduce the adverse environmental effects of its products and
services. We would like to encourage you as our customers and users to join us in working
towards a cleaner, safer environment. Please recycle product packaging and follow the
recommendations for power use and proper disposal of our products and their components.

If you should have questions regarding our Environmental Policy or any of the environmental
services we offer, please contact us at Nokia for any additional information.

© 2021 Nokia.Nokia confidential 3


Administration and Troubleshooting of the BTS Conversion Service

4 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

Table of Contents

Summary of changes .................................................................................................................... 9

1 Accessing BTS Conversion Service ........................................................................................... 10

2 Commands used in BTS Conversion Service .......................................................................... 11


2.1 Commands for administering BTS CS and NetAct Integration ................................. 11
2.2 Commands for administering BTS CS services ........................................................... 14

3 Checking the proper time synchronization ............................................................................ 18

4 Checking and administering certificates ................................................................................. 20


4.1 Checking if certificates are valid ................................................................................... 20
4.2 Installing new certificates .............................................................................................. 20
4.3 Importing client certificate for HTTPS Server connection ........................................ 25

5 Managing accounts' passwords ................................................................................................ 26


5.1 Overview of service accounts ........................................................................................ 26
5.2 Changing service accounts' passwords ........................................................................ 27
5.3 Changing password expiration time for service accounts ........................................ 29
5.4 Updating MongoDB and Minio Docker service password .......................................... 30

6 Checking BTS Conversion Service logs .................................................................................... 33


6.1 Checking log rotation ...................................................................................................... 33
6.2 Checking BTSCS logs ....................................................................................................... 33
6.3 Checking the status of BTS Conversion Service related services ............................ 36
6.4 Assessing the log content .............................................................................................. 37

© 2021 Nokia.Nokia confidential 5


Administration and Troubleshooting of the BTS Conversion Service

List of Figures
Figure 1 Logging to BTSCS ......................................................................................................... 10
Figure 2 Sample output for btscs -i command ....................................................................... 12
Figure 3 Sample output for btscs -cp command .................................................................... 13
Figure 4 Sample output for btscs -gc command .................................................................... 13
Figure 5 Sample output for btscs -isc command .................................................................... 13
Figure 6 Sample output for btscs -icc command ................................................................... 13
Figure 7 Sample output for btscs -vc command .................................................................... 14
Figure 8 Sample output for btscs -bu command .................................................................... 14
Figure 9 Sample output for btscs -rs command ..................................................................... 14
Figure 10 Sample output for start.sh command .................................................................... 16
Figure 11 Sample output for stop.sh command ..................................................................... 16
Figure 12 Sample output for restart.sh command ................................................................. 17
Figure 13 Sample output for btscs -v command .................................................................... 17
Figure 14 Expected output after executing ntpq -p command ........................................... 18
Figure 15 Generating certificate signing request ................................................................... 21
Figure 16 Example of SAN fields ................................................................................................ 23
Figure 17 Verifying certificate in BTSCS ................................................................................... 25
Figure 18 Example of successful import of client certificate ................................................ 25
Figure 19 Changing password for service account example ................................................. 28
Figure 20 Changing root password example ........................................................................... 28
Figure 21 Checking expiration date .......................................................................................... 28
Figure 22 System notification about expired password ........................................................ 29
Figure 23 Checking exexpiration time example for toor4nsn ............................................... 29
Figure 24 Stopping BTSCS .......................................................................................................... 30
Figure 25 Starting BTSCS ........................................................................................................... 32
Figure 26 Container names and IDs .......................................................................................... 33
Figure 27 Sample conserv-api logs ........................................................................................... 35
Figure 28 Sample conserv-processor logs ............................................................................... 35
Figure 29 Sample conserv-minio logs ....................................................................................... 35
Figure 30 Sample conserv-mongo logs .................................................................................... 35
Figure 31 Sample conserv-nsqadmin-local logs ..................................................................... 36
Figure 32 Sample conserv-nsqlookupd-local logs .................................................................. 36
Figure 33 Sample conserv-nsq-local logs ................................................................................ 36
Figure 34 Sample nginx logs ...................................................................................................... 36
Figure 35 Checking docker containers statuses ..................................................................... 37
Figure 36 Example of nginx logs used for checking if the REST client requests are received
by BTS CS ........................................................................................................................ 37
Figure 37 Example of missing server certificate ..................................................................... 38
Figure 38 The conserv-api logs example .................................................................................. 38
Figure 39 The conserv-processor logs example ..................................................................... 39

6 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

Figure 40 Example of an error report in the conserv-processor logs ................................. 39

© 2021 Nokia.Nokia confidential 7


Administration and Troubleshooting of the BTS Conversion Service

List of Tables
Table 1 Scripts and commands used for administering BTS CS and NetAct Integration
........................................................................................................................................... 12
Table 2 Scripts and commands used for administering BTS CS .......................................... 14
Table 3 Services and components in BTSCS ............................................................................ 26
Table 4 Default accounts' credentials ...................................................................................... 27
Table 5 Description of the BTSCS logs ..................................................................................... 34

8 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

Summary of changes
A list of changes between document issues. You can navigate through the respective
changed topics.

Changes between issues 01 (2019-11-29, SRAN 19B) and 02 (2020-07-24,


SRAN 20B)

Commands for administering BTS CS and NetAct Integration

The btscs -bu and btscs -rs commands has been added.

© 2021 Nokia.Nokia confidential 9


Administration and Troubleshooting of the BTS Conversion Service

1. Accessing BTS Conversion Service


Instructions on accessing BTSCS.

To access BTS Conversion Service virtual machine, log in via SSH using the toor4nsn
credentials.

Figure 1: Logging to BTSCS

The banner shown in Figure: Logging to BTSCS indicates that you are already inside the BTS
CS VM. You can now have access to BTS CS:
docker services
commands/scripts
docker and system logs
configuration files
file storage

10 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

2. Commands used in BTS Conversion Service


Commands used for administration in BTS Conversion Service and NetAct

2.1 Commands for administering BTS CS and NetAct


Integration
A list of commands used for administering BTS CS and NetAct Integration. They can be
run either as toor4nsn or root user.

© 2021 Nokia.Nokia confidential 11


Administration and Troubleshooting of the BTS Conversion Service

Table 1: Scripts and commands used for administering BTS CS and NetAct Integration

Script or command Description Sample output

btscs -i Initializes BTS Conversion Service REST Figure: Sample output for btscs -i
btscs --initialise API. The default user is created. command

btscs -cp This command is used to change the Figure: Sample output for btscs -cp
btscs --change-password password of the REST API user. command

btscs -gc This command creates a certicate Figure: Sample output for btscs -gc
btscs --generate-csr request to be signed by NetAct CA when command
requesting for a server certicate.
Run this command at
/home/toor4nsn/releases/<ver
sion_name>/

btscs -isc This command is used to import the If the server certicate
btscs --import-server-cert signed server certicate to BTS server_certificate.pem is
Conversion Service. It takes the located at /home/toor4nsn then the
certicate to be imported as a sample output is as in the Figure:
parameter. Sample output for btscs -isc command.

btscs -icc This command is used to import the If the server certicate
btscs --import-client-cert client certicate for HTTPS Server to BTS client_certificate.pem is
Conversion Service VM. It takes the located at /home/toor4nsn then the
HTTPS client certicate to be imported sample output is as in the Figure:
as a parameter. Sample output for btscs -icc command.

btscs -vc This command is used to import the Figure: Sample output for btscs -vc
btscs --view-certificates client certicate for HTTPS Server to BTS command
Conversion Service VM. It takes the
HTTPS client certicate to be imported
as a parameter.

btscs -bu This command creates a copy of the Figure: Sample output for btscs -bu
btscs --backup current BTS CS conguration les and command
other important directories (storage,
certicates and so on). This command
must be run as root user, because there
are les that need permission to be
accessed, like certicates. The output is a
TAR.GZ le located in the current working
directory.

btscs -rs This command takes the backup le Figure: Sample output for btscs -rs
btscs --restore (created using btscs --bu or btscs command
--backup commands) as input and
extracts its contents to restore BTS CS
previous conguration settings. This
command must be run as root user,
because there are les that need
permission to be accessed, like
certicates.

Figure 2: Sample output for btscs -i command

12 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

Figure 3: Sample output for btscs -cp command

Figure 4: Sample output for btscs -gc command

Figure 5: Sample output for btscs -isc command

Figure 6: Sample output for btscs -icc command

© 2021 Nokia.Nokia confidential 13


Administration and Troubleshooting of the BTS Conversion Service

Figure 7: Sample output for btscs -vc command

Figure 8: Sample output for btscs -bu command

Figure 9: Sample output for btscs -rs command

2.2 Commands for administering BTS CS services


List of commands used for administering BTS CS. These commands can be run only as
toor4nsn, and must also be run in
/home/toor4nsn/releases/<version_name>/directory, unless noted otherwise.

Table 2: Scripts and commands used for administering BTS CS

14 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

Script or command Description Sample output

start.sh This command is used to start Figure: Sample output for


all of BTS Conversion Service start.sh command
docker containers

stop.sh This command is used to stop Figure: Sample output for


running all of BTS Conversion stop.sh command
Service docker containers.

Note:

Any ongoing operation in


the REST API are stopped
after executing this
command.

restart.sh This command is used to Figure: Sample output for


restart all of BTS Conversion restart.sh command
Service docker containers. It
stops then starts the docker
containers. Data in database
and storage are not removed.

Note:

Any ongoing operation in


the REST API are stopped
while this command is
running.

btscs -v This command is used to Figure: Sample output for


btscs --version check the version of BTS btscs -v command
Conversion Service. It displays
the following information:
• Docker build version (BTS
Conversion Service)
• OVA version (Virtual Machine)
• CLI version (BTSCS CLI)

Note:

This command can be run


either as toor4nsn or
root user. It's also
possible to run this
command from any
current working directory.

© 2021 Nokia.Nokia confidential 15


Administration and Troubleshooting of the BTS Conversion Service

Figure 10: Sample output for start.sh command

Figure 11: Sample output for stop.sh command

16 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

Figure 12: Sample output for restart.sh command

Figure 13: Sample output for btscs -v command

© 2021 Nokia.Nokia confidential 17


Administration and Troubleshooting of the BTS Conversion Service

3. Checking the proper time synchronization


Instructions on how to check if the clock is synchronized or not.

To check which NTP servers are configured in BTS CS, login as root user using su command
and view the configuration file located at /etc.

[toor4nsn@btscs ~]$ su

[root@btscs toor4nsn]# vi /etc/ntp.conf

Look for the server directive in the configuration file. Servers marked as server
<ntp_server_address> iburst are configured NTP servers.

Note:

There must be at least two external NTP servers.

Checking if the clock is synchronized


To check if NTP is synchronized, run ntpq -p command.

If the synchronization is successful then the console returns output similar as in Figure:
Expected output after executing ntpq -p command.
Figure 14: Expected output after executing ntpq -p command

If there is no activity shown in the output of this command, NTP is unsynchronised.

In order to synchronize the server run the following commands:

# su

# service stop ntpd

# ntpdate <server_ip> <server_ip_1> <server_ip_2> <server_ip_3>

# service start ntpd

where the <server_ip_X> is the IP of the server described in the NTP configuration
(/etc/ntp.conf)

18 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

After a few minutes, run ntpq -p command again to check if NTP is in sync.

© 2021 Nokia.Nokia confidential 19


Administration and Troubleshooting of the BTS Conversion Service

4. Checking and administering certificates


Basic instructions on managing BTS CS certificates

4.1 Checking if certificates are valid


Instructions on checking if certificates are up-to-date.

When rsyslog is configured, the service will send messages or alerts to the rsyslog server
when the certificate is expired. To check server certificate expiration date manually, log in to
BTS CS VM through SSH using toor4nsn account, then run the following command as root:

# su

# openssl x509 -enddate -noout -in <server_certificate>

Usually <server_certificate> is located at /etc/ssl/certs/server.pem. BTS CS


returns a message:

notAfter=Jul 26 05:51:12 2069 GMT

where Jul 26 05:51:12 2069 GMT is the expiration date.

Compare the result to the current date to check if the certificate is up-to-date.

If it is expired, generate another certificate request to the NetAct Certification Authority.


For more information, see Installing new certificates.

4.2 Installing new certificates


Instructions on how to install new certificates.

Procedure
1 Login to BTSCS VM through SSH using toor4nsn account.

2 Generate certificate signing request (CSR).

2.1 Go to /home/toor4nsn/releases/<version>/.

2.2 Modify a san.cnf file using $ vi san.cnf command.

20 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

2.3 Change the DNS.1 value to the fully qualified domain name for BTS Conversion
Service.

Step example
DNS.1 = <fully qualified domain name of BTSCS>

2.4 In the same directory execute one of the commands below and fill in required
information.

btscs -gc
btscs --generate-csr

Figure 15: Generating certificate signing request

For more information on filling required values, see NetAct Operating


Documentation > Security Management > Security Management Operating
Procedures > Administering NetAct System Security > Managing certificates.

Step result
A server.csr file is created in the /home/toor4nsn/releases/<version>/
directory.

3 Sign certificate using NetAct CA.

The end entity (EE) certificate is issued by the CMP server and will require support from
CMP server maintenance. In this process, NetAct CA is used.
3.1 Log in to the VM, where the Netact dmgr service is running and switch to root
user.

3.2 Copy the generated BTSCS CSR to NetAct CA by executing the command:

© 2021 Nokia.Nokia confidential 21


Administration and Troubleshooting of the BTS Conversion Service

cd /opt/oss/NSN-sm_conf_cert/generated/server/

3.3 Copy the CSR content from BTSCS to the destination file with the filename
<systemName>_<certId>.csr, where <systemName> is the name used for
NetAct CA and <certId> is the BTSCS of CSR source.

3.4 Copy the configuration file (CNF) template by executing the command:

$ cp /opt/oss/NSN-sm_conf_cert/templates/server.cnf
/opt/oss/NSNsm_conf_cert/generated/server/<systemName>_<cert
Id>.cnf

4 Verify the contents of the subject alternative name (SAN) field in the CSR file
generated from BTSCS.

4.1 Execute the command:

$ openssl req -text -noout -in


/opt/oss/NSNsm_conf_cert/generated/server/<systemName>_<cert
Id>.csr

4.2 Find the fields for SAN or execute the command:

$ openssl req -noout -text -in


/opt/oss/NSNsm_conf_cert/generated/server/<systemName>_<cert
Id>.csr | grep DNS

22 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

Figure 16: Example of SAN fields

Note:

If DNS parameter doesn't exist, the SAN field is missing.

4.3 If SAN field is missing, add the following parameters to the CNF:

subjectAltName - @alt_names
[alt_names] - the host name
<FQDNsAndIPs> - the fully qualified hostname and IPs.

4.4 If the IP addresses or DNS are listed in the SAN field, copy the IP address and
DNS details under [alt_names].

Step example

© 2021 Nokia.Nokia confidential 23


Administration and Troubleshooting of the BTS Conversion Service

DNS.1 = Fully qualified hostname

IP.1 = IP address

5 Finish signing the BTSCS certificate with NetAct CA by executing the following
command:

$ /opt/oss/NSN-sm_conf_cert/bin
/smcert_sign_server_certificate.sh --systemName <systemName> --
caLevel 0 --certId <certId> -a <hashingAlgorithm>

Step example
$ /opt/oss/NSNsm_conf_cert/bin/smcert_sign_server_certificate.sh
--systemName NetAct --caLevel 0 --certId BTSCS -a SHA1

Step result
Once the signing is completed, the generated certificate (privacy enhanced mail (PEM)
file) for BTSCS is located at /opt/oss/NSN-sm_conf_cert/generated/server.

6 Transfer the certificates to BTSCS.

7 Copy the TA (NetAct_L0_CACert.pem) and EE certificates (NetAct_BTSCSCert.pem)


into one PEM file, by combining them manually in a text editor or executing
command:

cat NetAct_BTSCSCert.pem NetAct_L0_CACert.pem > <filename>.pem


This becomes the chain file to install in BTS Conversion Service.

8 Add the certificate chain in BTS Conversion Service. Use one of the commands:

$ btscs --import-server-cert </path/to/file.pem>


$ btscs -isc </path/to/file.pem>

9 Check the installed certificate in BTSCS. Execute btscs --view-certificates


or btscs -vc command.

24 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

Figure 17: Verifying certificate in BTSCS

4.3 Importing client certificate for HTTPS Server


connection
To download ADMIN CLI software packages a secure connection with the HTTPS server
must be established. In order to do that, import a client certificate.

Procedure
1 Copy the certificate to BTSCS.

2 Import the client certificate to BTSCS, by using one of the commands:

$ btscs -icc <file-path-to-file.pem>


$ btscs --import-client-cert <file-path-to-file.pem>

Result
Figure 18: Example of successful import of client certificate

© 2021 Nokia.Nokia confidential 25


Administration and Troubleshooting of the BTS Conversion Service

5. Managing accounts' passwords


Instructions on managing accounts passwords

5.1 Overview of service accounts


Table 3: Services and components in BTSCS

Service or component Default account Account description How to change a


credentials password for the
account?

BTS Conversion Service See Table: Default It is used to enable ssh See Changing service
VM accounts' credentials connection to BTS CS. accounts' passwords

BTS CS REST API See Table: Default It is used to enable


accounts' credentials connection from REST
client to BTS CS.

MINIO Can be viewed in le It is used to access the See Updating MongoDB
/etc/conserv/conf le storage of BTS CS. and Minio Docker service
ig.ini password
([local_s3/global_s3])

MongoDB Can be viewed in It is used to access the See Updating MongoDB


le/etc/conserv/co le storage of BTS CS. and Minio Docker service
nfig.ini ([db]) password

26 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

Table 4: Default accounts' credentials

Username Default password Account description

toor4nsn bts.conserv123 Credentials for service


management. Default
password should be changed
immediately.

root bts.conserv123 Credentials for service


management with root
privileges. Default password
should be changed
immediately.

csuser jR#g'Kw2b/4 Credentials for service


management. Default
password should be changed
immediately.

Password complexity requirements for toor4nsn and root users:


Minimum password length is 8 characters
At least one lowercase letter
At least two numbers
At least five unique characters
Does not contain control characters, delete and space.
Should not be the same as previous password

Password complexity requirements for csuser:


Minimum length - 8 characters
Maximum length - 128 characters
At least one non-alphanumeric character
At least two numbers
Must contain upper and lower case letters
Two of same characters cannot be used consecutively
Cannot be the same as the username
Cannot contain:control characters, space, delete, <,>,",{,},|,\,^,',:

5.2 Changing service accounts' passwords


Instructions on changing service accounts' password and checking their expiration date.

© 2021 Nokia.Nokia confidential 27


Administration and Troubleshooting of the BTS Conversion Service

Changing password
To change service account (toor4nsn) password, log in to BTSCS through SSH using the
service account credentials and use passwd command.
Figure 19: Changing password for service account example

To change root password, log in as root first, then use passwd command.

Figure 20: Changing root password example

Expired service account password


Password expires every 60 days. To check the when the password of the user account will
expire, type chage -l <username>. To check root account password details, log in first as
root using su command. The console displays information shown in Figure: Checking
expiration date.

Figure 21: Checking expiration date

When the password expires, the system prompts you to change the password at first log-in
attempt.

28 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

Figure 22: System notification about expired password

5.3 Changing password expiration time for service


accounts
By default, toor4nsn and root account password will expire after every 60 days and will
need setting up of a new password. The password expiration can be adjusted using chage
command as root user.

Procedure
1 Login to BTSCS VM through SSH on toor4nsn account.

2 Access root account by using the su command

[toor4nsn@btscs ~]$ su

Password:

[root@btscs toor4nsn]#

3 Check password expiration of user toor4nsn or root using chage -l <username>


command.

Figure 23: Checking exexpiration time example for toor4nsn

4 Update password expiration time using the # chage -M <number of days>


<username> command.

5 Verify the expiration date using # chage -l <username> or command.

© 2021 Nokia.Nokia confidential 29


Administration and Troubleshooting of the BTS Conversion Service

5.4 Updating MongoDB and Minio Docker service


password
It is possible to customize the credentials of Minio and MongoDB services by modifying
configuration files of BTS Conversion Service. None of this services can be accessed
outside BTS Conversion Service Virtual Machine. Upgrading BTS Conversion service will
restore the passwords for MongoDB and Minio to the default values.

Procedure
1 Login to BTSCS VM through SSH using toor4nsn account.

2 Stop BTS Conversion Service.

Go to /home/toor4nsn/releases/<version>/ and stop running docker services.

$ cd /home/toor4nsn/releases/<version>

$ chmod +x stop.sh

$ ./stop.sh

Step example
Figure 24: Stopping BTSCS

3 Modify /etc/conserv/config.ini.

Use the command:

30 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

$ vi /etc/conserv/config.ini
For MongoDB update fields username and password under the section db.

[db]

username = <username>

password = <password>

For Minio update fields access_key and secret_access_key under the sections
global_s3 and local_s3.

Note:

The values set in the global_s3 must be the same as in the local_s3.

[global_s3]

access_key = <access key>

secret_access_key = <secret key>

[local_s3]

access_key = <access key>

secret_access_key = <secret key>

4 Modify /home/toor4nsn/releases/<version>/docker-compose.yml.

$ vi /home/toor4nsn/releases/<version>/docker-compose.yml
For MongoDB:
update
services ► mongo ► environment ► MONGO_INITDB_ROOT_USERNAME
based on the value set in config.ini for db ► username
update
services ► mongo ► environment ► MONGO_INITDB_ROOT_PASSWORD
based on the value set in config.ini for db ► password

For Minio:

update services ► minio ► environment ► MINIO_ACCESS_KEY based on


the value set in config.ini for global_s3 ► access_key
update services ► minio ► environment ► MINIO_SECRET_KEY based on
the value set in config.ini for global_s3 ► secret_access_key

© 2021 Nokia.Nokia confidential 31


Administration and Troubleshooting of the BTS Conversion Service

5 Start docker services.

Go to /home/toor4nsn/releases/<version>/ and start docker services.


Use the command:

$ cd /home/toor4nsn/releases/<version>/

$ chmod +x start.sh

$ ./start.sh

Step example
Figure 25: Starting BTSCS

32 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

6. Checking BTS Conversion Service logs


Instructions on checking BTSCS logs

6.1 Checking log rotation

Docker Logs
Docker container log rotation for each docker services can be checked under
/home/toor4nsn/releases/latest/docker-compose.yml on
services ► <service> ► logging ► options ► max-size.
The default maintaining log size is 50MB.

System logs
System log rotation configuration can be checked under /etc/logrotate.conf.

Logging configuration that has a log rotation can be checked under /etc/logrotate.d/*

6.2 Checking BTSCS logs


The description of log containers and how to access them.

To access the logs of individual docker containers in BTS Conversion Service run the docker
logs <container name/ID> command. The logs for the specified container will be
displayed on screen.

The logs of a specific docker container can be identified using its container name or its
container ID. To get container ID or container name, execute command docker
container ls.

Figure 26: Container names and IDs

© 2021 Nokia.Nokia confidential 33


Administration and Troubleshooting of the BTS Conversion Service

The description of the logs for BTSCS related services is in Table: Description of the BTSCS
logs.

Table 5: Description of the BTSCS logs

Docker container Log description Initial log sample

conserv-api Shows project-related information (for Figure: Sample conserv-api logs


example Project ID, input le IDs,
output le IDs, project-related errors)

latest_conserv- Shows process-related information Figure: Sample conserv-processor logs


processor_1, (for example Process ID, input le IDs,
latest_conserv- output le IDs, process-related errors)
processor_2,
latest_conserv-
processor_3,
latest_conserv-
processor_4

conserv-minio Shows information related to storage Figure: Sample conserv-minio logs


(S3) access

conserv-mongo Shows information related to database Figure: Sample conserv-mongo logs


(MongoDB) access

conserv-nsqadmin-local, Shows information related to Figure: Sample conserv-nsqadmin-


conserv-nsqlookupd-local, communication between the API local logs
conserv-nsq-local (conserv-api) and processor (conserv- Figure: Sample conserv-nsqlookupd-
processor) local logs
Figure: Sample conserv-nsq-local logs

nginx Shows information related to API Figure: Sample nginx logs


access (for example remote address Nginx log shows the following details:
which accessed the service, SSL • Remote address which is accessing
protocol used, SSL cipher used, the BTS Conversion Service (ex.
request status and so on). 10.x.x.x)
• REST API user (ex. bts_user)
• Time and date of HTTP request (ex.
27/Nov/2019:08:38:21 +0000)
• TLS Protocol used and cipher used
(ex. TLSv1.2/ECDHE-RSA-AES256-GCM-
SHA384)
• HTTP request details (ex. GET
/api/projects HTTP/1.1)
• HTTP response status (ex. 404)
• Bytes of HTTP request body
• HTTP referer
• HTTP User Agent
• Request ID

To identify a request you have to use the timestamp, which is also available in the log.

34 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

Note:

Nginx logs are the most important and useful logs to view in Conversion Service. They
show if:
the request status to BTSCS was successfully executed (for example: Response
status code in 200 range - 200, 201, and so on).
the request status was received by BTSCS (by checking the request details: time
and date of the request, and also remote address and user).

Figure 27: Sample conserv-api logs

Figure 28: Sample conserv-processor logs

Figure 29: Sample conserv-minio logs

Figure 30: Sample conserv-mongo logs

© 2021 Nokia.Nokia confidential 35


Administration and Troubleshooting of the BTS Conversion Service

Figure 31: Sample conserv-nsqadmin-local logs

Figure 32: Sample conserv-nsqlookupd-local logs

Figure 33: Sample conserv-nsq-local logs

Figure 34: Sample nginx logs

6.3 Checking the status of BTS Conversion Service related


services
Instructions on checking if the docker containers are running properly

To check if BTS Conversion Service is running properly, verify if the docker containers are
running. Use the docker container ls command.

The following containers must be active:


conserv-api
conserv-processor
conserv-minio
conserv-nsqadmin-local
conserv-nsqlookupd-local
conserv-nsq-local
conserv-mongo
nginx

36 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

Figure 35: Checking docker containers statuses

If the STATUS is different then UP <x


years/months/days/hours/minutes/seconds>, view the logs of an individual
container for troubleshooting.

Use the docker logs <container ID/name> command.

If there are no docker containers running, check if the docker images were loaded with the
command: docker image ls.

If the docker images are loaded, go to ~/releases/<version> and then run start.sh.

If there are no docker images loaded, go to ~/releases/<version>, then run


install.sh.

6.4 Assessing the log content


BTS CS sends expected responses and handled error messages to the REST Client (for
exampleInternal Server Error). If any error occurs, check the following docker
container logs:
conserv-api
conserv-processor
nginx

The nginx logs


The nginx logs record the requests sent to BTS CS REST API. They can be used to check if
the REST client requests are received by BTS CS.

Figure 36: Example of nginx logs used for checking if the REST client requests are received by BTS
CS

Nginx logs can also be used for checking, if the server certificate is working correctly. They

© 2021 Nokia.Nokia confidential 37


Administration and Troubleshooting of the BTS Conversion Service

return the warnings about the following certificates' errors:


invalid certificate
no server certificate present
mismatch between server key and server certificate

Figure 37: Example of missing server certificate

The conserv-api logs


The conserv-api logs record the communication between the API and the conserv-
processor (where the core functionality of the BTS Conversion service is). They also record
traces of unexpected errors returned to the REST client (for example unhandled internal
server errors). They can be used to check if any operation is queued.
Figure 38: The conserv-api logs example

The first log entry indicates that the project is currently queued and is waiting for all its
operations to be done.

The second log entry indicates that the project has been completed.

The conserv-processor logs


The conserv-processor logs record the communication with conserv-api, and record traces
of unexpected failure during migrate, merge or convert operations. They can be used to
check if an operation is started or ongoing.

38 © 2021 Nokia.Nokia confidential


Administration and Troubleshooting of the BTS Conversion Service

Figure 39: The conserv-processor logs example

The first log entry indicates that an individual process is currently queued.

The second log entry indicates that the process has been completed.

When the processing of an operation fails, conserv-processor sends error to the API as a
response to the REST client.
Figure 40: Example of an error report in the conserv-processor logs

The conserv-api picks up the Renaming from MRBTS_123 to MRBTS-321.xml failed


error and sets the project status to FAILED. The same error is reported on the REST client
side.

Sometimes a project seems to be ongoing for a long time (more then 1.5 minutes). It means
that the conserv-processor container is not running. To solve this, restart the docker
services using the command restart.sh, then check if all docker containers are running
using docker container ls command.

© 2021 Nokia.Nokia confidential 39

You might also like