Emergency Restoring a Configuration

Version Backup

OPERATING INSTRUCTIONS

150/1543-CRX 901 49/1 Uen G

Copyright

© Ericsson AB 2011–2014. All rights reserved. No part of this document may be

reproduced in any form without the written permission of the copyright owner.

Disclaimer

The contents of this document are subject to revision without notice due to
continued progress in methodology, design and manufacturing. Ericsson shall
have no liability for any error or damage of any kind resulting from the use
of this document.

Trademark List

All trademarks mentioned herein are the property of their respective owners.
These are shown in the document Trademark Information.

150/1543-CRX 901 49/1 Uen G | 2014-10-06

 Contents

Contents

1 Overview 1
1.1 Description 1
1.2 Prerequisites 1

2 Procedure 5
2.1 Backing Up Data for Fault Investigation 5
2.2 Preparing the CV and the UPs on a PC 5
2.3 Configuring the IP Address on the Node 6
2.4 Transferring and Activating the Backup CV and the UPs 6
2.5 Restoring Licenses and Security Configuration 7
2.6 Concluding Activities 8
2.7 Health Check 8

3 Further Information 11
3.1 Description 11
3.2 Error Cases 12

150/1543-CRX 901 49/1 Uen G | 2014-10-06

Emergency Restoring a Configuration Version Backup

150/1543-CRX 901 49/1 Uen G | 2014-10-06

 Overview

1 Overview

1.1 Description
This instruction describes how to restore a Configuration Version (CV) in an
RBS 6000 node, from a PC that has serial and Ethernet connections to the
node. This instruction is valid when the node is NOT accessible and is NOT
operational (for example, in cyclic restart). At least one Core MP in the main
subrack must be free from hardware faults. A DU with a Core MP role is one of
up to two DUs in a node that controls the loading of the node and that houses
the /c disk.

Note: 1. Use this instruction if the node must be recovered by restoring a

CV. See Recovering a Node on Site to find out how to recover a node
in other situations.

2. A CV does not contain licensing and security configuration files.

These files are normally present on an external file server. If these
files are lost on the node, install them again using the appropriate
procedures after node recovery. See Licensing and Security for O&M
Node Access for more information.

1.2 Prerequisites
This section provides information on the documents, tools and conditions that
apply to the procedure.

1.2.1 Documents

Before starting this procedure, ensure that you have read the documents:

• System Safety Information

• Handling Faulty Equipment

• Personal Health and Safety Information

1.2.2 Tools

Before starting this procedure, ensure that the following tools are available:

• A PC with Windows® OS that can be connected to the serial and Ethernet

ports on a DU.

• Cables for serial and Ethernet connections to the node.

150/1543-CRX 901 49/1 Uen G | 2014-10-06 1

 Emergency Restoring a Configuration Version Backup

1.2.3 Preparations on the PC

Software to be installed and configured on the PC:

• Element Manager with the relevant Java Version, HyperTerminal, Telnet,

SSH and (S)FTP client.

• A program for communication using the serial port. The recommended

program is WinFiol.

• Relevant copies of remotely stored backup CVs.

Note: Each backup CV must be created as described in Saving a

Configuration Version Backup. The CV must origin from this node.

• The Upgrade Package (UP) zip file(s) corresponding to the backup CV(s).

• The correct Active Library Explorer library, available on site.

• The serial connection is configured in the PC, with the following parameter
settings:

0 baud rate = 9600 bits per second

0 parity = none

0 stop bits = 1 bit

0 data bits = 8 bits

0 flow control = none

• Any firewall on the PC has been turned off.

1.2.4 Conditions

Before starting this procedure, ensure that the following conditions are met:

• You have the task profile, FM Advanced or higher, if security level 3 is set.

• The IP configuration for the node is known.

• You know the IP address to an Operations Support System for Radio and
Core (OSS-RC) at the Operation and Maintenance Center (OMC).

• You have information about any board that needs to be replaced, or:

• You have information about any data that must be saved that might help
finding out the reason for the stoppage.

• You have read Section 2 on page 5 and Section 3.1 on page 11, and you
know how to perform the procedure. This is an emergency procedure and it
must normally be performed quickly.

2 150/1543-CRX 901 49/1 Uen G | 2014-10-06

 Overview

• If the attribute signatureValidationService for the PlugInUnit MO

has the value ENABLED, all the software in the Upgrade Package (UP)
which is pointed to by the backup CV has to be of type Signed Software
(SSW).

150/1543-CRX 901 49/1 Uen G | 2014-10-06 3

 Emergency Restoring a Configuration Version Backup

4 150/1543-CRX 901 49/1 Uen G | 2014-10-06

 Procedure

2 Procedure

Do the following at the node:

Note: This instruction is intended for use when handling a Complete

Exchange Failure (CEF).

2.1 Backing Up Data for Fault Investigation

If possible, do the following steps. If it is not possible, continue with Section
2.2 on page 5.

1. Collect traces from the node by using the following commands:

• bs -g -c "te log read" >> /d/node_data.txt

• bs -g -c "llog -l" >> /d/node_data.txt

• bs -g -c "vii" >> /d/node_data.txt

• bs -g -c "dumpelg" >> /d/node_data.txt

2. Save information regarding the CV usage by using the command:

cv cu >> /d/node_data.txt.

3. Back up the following files and directories to the PC:

• /d/node_data.txt

• /c/logfiles

• /c/systemfiles

• /c/pmd

Note: Use binary mode when transferring files to and from the node.

4. Continue with Section 2.2 on page 5.

2.2 Preparing the CV and the UPs on a PC

1. Unzip the backup CV on the PC, if not already done.

2. Unzip the UPs on the PC, if not already done.

3. Open a command prompt window on the PC, and go to the \misc directory
of the CV to be restored, using command: cd <pathToCV>\misc.

150/1543-CRX 901 49/1 Uen G | 2014-10-06 5

 Emergency Restoring a Configuration Version Backup

4. If the hardware configuration has been changed since the backup CV

was created, edit the file that controls the autoconfiguration in the node
(autoconfig.xml). This is explained with comments within the file.

5. Continue with Section 2.3 on page 6.

2.3 Configuring the IP Address on the Node

1. If the node has two Core MPs, and both are in cyclic restart, stop the Core
MPs one at a time by connecting the PC to the serial port on the board, and
execute the command reload --.

2. Verify that the /c2 volume is mounted, using the command: vols /c2. If
the /c2 volume is not mounted, mount the volume, using the command:
mount_c2.

3. Format both volumes using the command formathd /c2 and formathd
/d respectively, on both MPs.

4. Set the IP configuration on one of the boards, with the command

ifconfig le0 <ipv4address> netmask <netmask> broadcast
<broadcastaddress>. <ipv4address> is the IPv4 address of the board.

5. Set the default gateway address on the board with the command route
add default <gateway>.

6. Make the new IP configuration active in the node, by restarting the board
again with the command reload --.

7. Continue with Section 2.4 on page 6.

2.4 Transferring and Activating the Backup CV and the

UPs
1. Open a command prompt window on the PC, and go to the \misc directory
of the CV to be restored, using the command cd <pathToCV>\misc,
if not already done.

2. Transfer the CV to the node with the command:

ftp -v -s:cvftpput.sh <ipv4address>. <ipv4address> is the IPv4

address of the board.

When executing the command, stay in the \misc directory of the CV to

be restored.

3. Go to each UP directory that has to be transferred to the node, and transfer

the UP files with the command:

ftp -v -s:<pathToCV>\misc\upftpput_<no_rev>.sh
<ipv4address>. <ipv4address> is the IPv4 address of the board.

6 150/1543-CRX 901 49/1 Uen G | 2014-10-06

 Procedure

When executing the command, stay on the same level in the structure as
where the Upgrade Control File (UCF) <ucf>.xml file is located.

4. Before starting the active Core MP, verify the software consistency of the
node. Ensure that all UP and CV files have been successfully transferred
and are present on the node, as follows:

• Verify the existence of the /c2 and /d volumes, using the vols
command.

• Verify that the correct UP has been transferred to the node, using the
command ls -l /c/up/.

• Use the command cat /d/cv.ptr and verify that the corresponding
CV directory exists under /d/configuration/cv.

5. Start the node with the command reload.

6. Wait until the node is operational. If the node does not recover, contact the
next level of maintenance support.

Note: Licensed functions are not operational until the License and
Security configurations are restored, see Section 2.5 on page 7.

7. Verify that the correct time is set in the node by using the command
readclock. If the time is incorrect, set the time by using the command
setclock YYYY-MM-DD HH:MM:SS.

8. If the node has two Core MPs, connect the PC to the serial port on the
other board, and execute the command reload. If the node has just one
Core MP, continue with Section 2.6 on page 8.

9. Wait until the board enters the state PassiveReady. This is indicated by
printout from the command getAttrObj me.

Note: It takes between several minutes and a maximum of approximately

40 minutes for the passive MP to reach state PassiveReady. If
the node does not recover, contact the next level of maintenance
support.

10. Continue with Section 2.5 on page 7.

2.5 Restoring Licenses and Security Configuration

1. If any alarm has been issued for the Security MO, act on the alarm,
using the corresponding instruction.

2. Verify the security level setting by reading the attribute secmode in the
Security MO. If the node is on security level LEVEL_UNDEFINED, use
the action adaptSecurityLevel, or the secmode command to set the
attribute requestedSecurityLevel to level 1.

150/1543-CRX 901 49/1 Uen G | 2014-10-06 7

 Emergency Restoring a Configuration Version Backup

3. Verify that the correct security level is set by reading the attribute secmode
in the Security MO. If necessary, set the desired security level, using the
instruction Setting Security Level.

4. If necessary, restore the original security level of the node, using the
instructions in the documents Security for O&M Node Access and Setting
Security Level. Ensure that all conditions in Setting Security Level are
fulfilled before changing the security level to level 2 or level 3.

5. If any alarm has been issued by the LicensingMO, act on the

alarm, using the corresponding instruction. As the License Key File is
missing after a stoppage, it must be installed again by using the action
updateLicenseKeyFile on the MO Licensing.

6. Make other reconfigurations that are necessary. As the CV might be old,

both traffic and O&M-related reconfigurations might be needed.

7. Continue with Section 2.6 on page 8.

2.6 Concluding Activities

Do the following:

1. If the cabling to node was changed to connect to the laptop, restore the
normal cabling.

2. Verify that the O&M connection is working. If the O&M connection is not
working, contact the next level of maintenance support.

3. The node creates a new CV in Step 5, containing the possibly new IP

parameters. Consider deleting the downloaded CV at a later stage. The
CV cannot be deleted at this time, as it is loaded.

4. If necessary, create a new CV, that contains the reconfigurations made in

Section 2.5 on page 7, and set it to startable. See Upgrade and Backup.

5. Complete the procedure with a Health Check as described in Section 2.7

on page 8.

2.7 Health Check

Do the following:

1. Run the disk health check to initiate a consistency check of the file
systems, configuration versions and upgrade packages, using the action
startHealthCheck on the ManagedElement MO. Verify that the none of
the following alarms is issued:

• Configuration Version Corrupt

• File System Diagnostic Error

• Upgrade Package Corrupt

8 150/1543-CRX 901 49/1 Uen G | 2014-10-06

 Procedure

2. If any of these alarms is issued, act on the alarm using the corresponding
instruction.

3. Exit this procedure.

150/1543-CRX 901 49/1 Uen G | 2014-10-06 9

 Emergency Restoring a Configuration Version Backup

10 150/1543-CRX 901 49/1 Uen G | 2014-10-06

 Further Information

3 Further Information

3.1 Description
This instruction describes how to restore a CV in a node, from a PC, even if
some hardware is faulty. The backup of the CV must have been saved correctly
from this node, for example by using the instruction Saving a Configuration
Version Backup. However, the CV might have been saved correctly using
another procedure that also uses the save function in the node.

The backup CV is stored in the form of a zip archive on an FTP server, and is
fetched and unzipped on the PC. The CV itself contains FTP scripts that will
transfer the CV and the UP files to the node.

The CV and UPs will be transferred to (one of) the Core MP(s) in the node. If
there are two Core MPs, the other Core MP is stopped. A Core MP is one of
the two DUs in a node that controls the loading of the node, and that houses
the /c disk. If the first board is faulty, try to start the other board instead. You
can also try to format the disk first, to eliminate any disturbing data.

Note: When the disk is formatted, all data are lost. Configuration files that
are not included in a CV must then be installed again on the node.
These files are normally present on an external file server. Examples
of configuration files not included in a CV are the License Key File
and the files for Security.

The commands to the node are sent on the serial port, and the files are
transferred from the PC on the Ethernet port.

If other boards are faulty, they might be stopped automatically, but they can
also cause the restoration of the CV to fail, due to cyclic restarts.

The autoconfiguration setting is handled by the file autoconfig.xml, making

it possible to handle a different hardware configuration than in the database
backup. Autoconfiguration should be enabled, but if you are certain that the
node configuration matches the backup CV, autoconfiguration can be disabled.

To start the CV, the CV and the necessary load module files must be stored on
the disk in the Core MP. The FTP scripts transfers the CV and the load module
files that are part of the necessary UPs to the Core MP. The necessary UPs are
listed in the file uplist.xml in the backup CV. The load module files will be
distributed to the other DUs after the restart.

The scripts that transfers all files to the node are executed when the Core MPs
are in backup mode. This eliminates all possible interference from the node
itself. You might need the task profile FM Advanced to execute the reload
-- command, but if the board is already in backup mode the command is
always allowed.

150/1543-CRX 901 49/1 Uen G | 2014-10-06 11

 Emergency Restoring a Configuration Version Backup

All necessary files, including the files that control the CV function, are
transferred by scripts, and it does not matter if the disk is empty, or already
contains the files.

3.2 Error Cases

If the restoration fails, typical error cases are:

• The scripts fail to transfer files because:

0 Files are missing on the PC.

0 The Core MP is not in backup mode.

0 The IP configuration is not correct.

0 There is write protection on the CV control files, or the CV already

exists.

0 The UCF file in the Backup CV does not have the same name as the
UCF file on the FTP server. This could happen if the UpgradePackage
MO that is associated to the CV is of type ‘‘virtual’’, that is when the
UpgradePackage MO has state ONLY_DELETEABLE. Then the name
of the UCF in the Backup CV will have a file name based on the product
data specified in the UCF on the FTP Server instead.

• Restart of Core MP fails because:

0 The other Core MP is not in backup mode.

0 There is a hardware fault on the board.

0 Mismatch in the SSW activation between the CV and the node. If

the SSW feature is active on the node, the loaded CV has to be of
type SSW.

0 Signature validation failure of the LMs against the trusted anchor

located on their respective boards.

• Restart of the node fails because:

0 The other boards create problems, leading to cyclic node restart.

0 The other boards have hardware faults and are stopped.

0 The other boards are not loaded due to a mismatch between the CV
and the node.

• Loss of communication after the node is started because:

0 Vital software is not loaded due to a mismatch between the CV and

the node.

12 150/1543-CRX 901 49/1 Uen G | 2014-10-06

 Further Information

0 The IP configuration is incorrect.

150/1543-CRX 901 49/1 Uen G | 2014-10-06 13