ASR9000/XR Understanding Turboboot and

initial System bring up

• Introduction on page 1
• Core Issue on page 1
• File System overview on page 2
Steps to Turboboot on page 6
Different Procedures that may be required to start a Turboboot on page 7
• Clear the ROM Monitor environmental variables on all RSPs: on page 7
• Clear BOOT_DEV_SEQ_OPER and MIRROR_ENABLE ROM Monitor
environment variables to disable disk mirroring. on page 8
• Change the default behavior of RP in ROMMON to not reset in 30 minutes on
page 9
• Define the local ip address to be used on the MGMT interface on the front face of
the RSP. on page 9
• On the RSP, set the TFTP environment variables: on page 9
• Set the Turboboot variables on the RSP: on page 10
• On the RSP, boot the vm image located on the TFTP server on page 11
• Restore disk mirroring if it was disabled on page 12
• How to boot from the external USB port (RSP440 only) on page 13
• How to update the FPD's on page 16
• Related Information on page 17

Introduction

This document provides an understanding of what Turboboot is and how to bring up a

system running IOS-XR from scratch

Core Issue

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
1
ASR9000/XR Understanding Turboboot and initial System bring up

Turboboot is a frequently misunderstood concept. What it really means is that you will do a
fresh install of the IOS-XR software.

XR devices have multiple medias for storage and they all have their individual purpose.

There are 2 ways to upgrade the system.

One is to install add and activate the new XR release mini-p(x).pie

or using the executable with the extension .vm.

The executable vm file needs to be transferred via TFTP (on the RSP2) or can be loaded
from the external USB port or TFTP (on the RSP440).

No other media are possible to be used for a turboboot other then the ones specified above.

File System overview

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
2
ASR9000/XR Understanding Turboboot and initial System bring up

Volume RSP2 RSP440 Trident Typhoon

disk0: Embedded USB SSD (SATA)

disk0a: Embedded USB SSD (SATA)

disk1: Embedded USB SSD (SATA)

disk1a: Embedded USB SSD (SATA)

harddisk: Harddisk (SAS) Embedded USB

harddiska: Harddisk (SAS) Embedded USB

harddiskb: Harddisk (SAS) Embedded USB

compactflash: Compactflash1 External USB1

lcdisk0: Embedded USB Embedded USB

lcdisk0a: Embedded USB Embedded USB

bootflash: NOR Flash NOR Flash NOR Flash

configflash: NOR Flash

nvram: NVSRAM NVSRAM

Kernel dump Harddisk (SAS) SSD (SATA) TFTP bootflash:

1. Removable

Access (Mount) Points (in /dev)

Volume RSP2 RSP440 Trident Typhoon

disk0: disk00t77 hd0t77 qsm to active rsp qsm to active rsp

disk0a: disk00t78 hd0t78 qsm to active rsp qsm to active rsp

disk1: disk10t77 hd1t77 qsm to active rsp qsm to active rsp

disk1a: disk10t78 hd1t78 qsm to active rsp qsm to active rsp

harddisk: hd0t79 usb00t77 qsm to active rsp qsm to active rsp

harddiska: hd0t77 usb00t78 qsm to active rsp qsm to active rsp

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
3
ASR9000/XR Understanding Turboboot and initial System bring up

harddiskb: hd0t78 usb00t11

compactflash: disk20t6,11,121 usb10t6,11,121

lcdisk0: lcdisk00t77 lcdisk00t77
lcdisk0a: lcdisk00t78 lcdisk00t78

bootflash: fs0p1 fs0p1 fs0p1

configflash: fs1p1

nvram: nvram nvram

Kernel dump hd0t80 hd0t80 or hd1t802 fs0p2

1. Any one 2. Either one

Usage

Volume RSP2 RSP440 Trident Typhoon

disk0: IOS-XR IOS-XR
Packages, Packages,
Configs Configs
disk0a: sysmgr_debug sysmgr_debug

disk1: IOS-XR Packages IOS-XR Packages

(if Mirrored) (if Mirrored)
disk1a: wdsysmon_debug wdsysmon_debug

harddisk: Crash files, logs Crash files, logs

harddiska: NP logs, crash NP logs, crash

files files
harddiskb:

compactflash: File Copy File Copy

lcdisk0: Kernel dump files Kernel dump files

lcdisk0a:

bootflash: MBI Images

configflash: OBFL

nvram: Configs Configs

Kernel dump Raw kernel dumps Raw kernel dumps Raw kernel dumps

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
4
ASR9000/XR Understanding Turboboot and initial System bring up

Filesystems
Volume RSP2 RSP440 Trident Typhoon
disk0: QNX4 QNX4
disk0a: QNX4 QNX4

disk1: QNX4 QNX4

disk1a: QNX4 QNX4

harddisk: QNX4 QNX4

harddiska: QNX4 QNX4

harddiskb: QNX4 FAT

compactflash: FAT1 FAT1,2

lcdisk0: QNX4 QNX4

lcdisk0a: QNX4 QNX4

bootflash: FFSv3 FFSv3 FFSv3

configflash: FFSv3

nvram: Cisco Cisco

Kernel dump Raw Raw File Raw

1. FAT F/S only 2. Flash Media only

Approximate Parition Size (minimum)

Volume RSP2 RSP440 Trident Typhoon

disk0: 1.6GB 11.0GB

disk0a: 0.4GB 2.2GB

disk1: 1.6GB 11.0GB

disk1a: 0.4GB 2.2GB

harddisk: 35GB 3.1/6.2GB

harddiska: 8GB 0.4/0.8GB

harddiskb: 8GB 0.4/0.8GB

compactflash: 1GB 1-32GB

lcdisk0: 1.6GB 1.6GB

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
5
ASR9000/XR Understanding Turboboot and initial System bring up

lcdisk0a: 0.4GB 0.4GB

bootflash: 44MB 56MB 56MB

configflash: 28MB
nvram: 220K 500K

Kernel dump 21GB 500MB x 2 24MB

Note that unlike many IOS devices, nvram is NOT used for the configuration storage.

Only rommon variables and license info is stored there primarily.

Steps to Turboboot

As mentioned Turbobooting means that you load the "VM" (virtual machine" XR base image
which is effectively an executable.

Turboboot is started from Rommon and is effectively putting a disk with the desired OS in
your laptop and reboot the machine to boot from CDROM.

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
6
ASR9000/XR Understanding Turboboot and initial System bring up

Before the Turboboot process starts, you can instruct the system to wipe all files from the
system and start clean or install the image to be turbobooted along side with any existing
releases currently found on the disk.

Turbobooting may be required if you want to sweep clean your system, or we also had some
issues in XR4.2.0 with the RSP2 (due to the smu sizes) whereby the upgrade pie could not
be loaded. A turboboot was required in that case also.

Different Procedures that may be required to start a Turboboot

Some or all of these procedures below are needed.

Clear the ROM Monitor environmental variables on all RSPs:

The command "set" gives you an overview of all the rommon environment variables
currently set to their values.

rommon>

unset BOOT

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
7
ASR9000/XR Understanding Turboboot and initial System bring up

unset TFTP_FILE
sync

the command *unset* clears the variable value from rommon.

the command *sync *saves or writes the newly set and unset variables to persistent memory
so they are saved cross reloads and power cycles.

Clear BOOT_DEV_SEQ_OPER and MIRROR_ENABLE ROM Monitor

environment variables to disable disk mirroring.

unset BOOT_DEV_SEQ_OPER
unset MIRROR_ENABLE
sync

By default, the two internal USB partitions (disk0 and disk1) are mirrored to each other, if
you break the mirror, turboboot will only affect the disk

that you are turbobooting target to and not the other one (nice if you want to fall back).

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
8
ASR9000/XR Understanding Turboboot and initial System bring up

Change the default behavior of RP in ROMMON to not reset in 30 minutes

Rommon> priv
Rommon> diswd <- Disable the CPU watchdog

If you omit this step and the TFTP download for the turboboot mini-vm image takes longer
due to network delays etc, then the RSP might reset and you'll have to do it again. Very
painful. So disabling this watchdog makes sure the system is not going to reload during the
transfer of the image in rommon.

Define the local ip address to be used on the MGMT interface on the front
face of the RSP.

IP_ADDRESS=ip_address
IP_SUBNET_MASK=mask
DEFAULT_GATEWAY=ip_address

On the RSP, set the TFTP environment variables:

TFTP_RETRY_COUNT=4

sets the number of retries to contact the tftp server

TFTP_TIMEOUT=6000

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
9
ASR9000/XR Understanding Turboboot and initial System bring up

sets the TFTP timeout for the transfer, you may need to set this larger to prevent abort
during xfer if there are network delays

TFTP_CHECKSUM=1

whether checksum on teh transfer is needed, this is adviceable in case the image gets
corrupted during transfer.

TFTP_SERVER=server_ip_addr

the server address can also be specified in the boot statement, or fixed in the rommon
variable.

TFTP_MGMT_INTF=0

which of the 2 mgmt interfaces you want to use, either 0 or 1.

Set a larger tftp block size in rommon:

Rommon> TFTP_BLKSIZE=1400

this is recommended to pack larger packets and be done with this TFTP xfer quicker.

Set the Turboboot variables on the RSP:

TURBOBOOT=on,disk0:,format
sync

This will instruct the system to do a turboboot to disk0 as target and it will format the
complete filesystem to start from base.

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
10
ASR9000/XR Understanding Turboboot and initial System bring up

the format key is optional.

Currently today we only support targeted install to disk0 but this will change likely in XR4.3.1
whereby you can use disk1 as install target.

NOTE: a recent tac case showed that the command for turboboot failed on the ASR9001.

Supposedly this was made to work by omitting the colon after disk0:

Suggesting to try the disk0 (without colon) if the command with colon fails.

On the RSP, boot the vm image located on the TFTP server

(Works only with the VM image, not the TAR file)

boot tftp://server/directory/filename

During the boot process the image is copied first on to the memory(RAM) and is installed
from memory(RAM). Once it is insalled from memory, it will copy the image back on to
disk0: and reload the device. Wait till you get the message "SYSTEM CONFIGURATION
COMPLETED"

Output of show install active when in memory,

RP/0/RSP0/CPU0:ios#sh instal active

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
11
ASR9000/XR Understanding Turboboot and initial System bring up

<SNIP>
Active Packages:
mem:asr9k-mini-p-4.2.0

Output of show install active after image copied on to disk0:,

RP/0/RSP0/CPU0:ios#sh instal active

<SNIP>
Active Packages:
disk0:asr9k-mini-p-4.2.0

The system will also self unset the TURBOBOOT rommon variable.

Restore disk mirroring if it was disabled

To restore disk mirroring, use the mirror command in the global configuration mode.
For more information on the mirror command, see the "Boot Commands on Cisco IOS
XR Software" module in Cisco ASR 9000 Series Aggregation Services Router System
Management Command Reference.

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
12
ASR9000/XR Understanding Turboboot and initial System bring up

How to boot from the external USB port (RSP440 only)

The RSP-440 can boot from the USB front panel port, insteafd of using "boot tftp:// or boot
disk0:/" you need to use a different command, mediaboot.

The command is *mediaboot *usb:\yourfilename

Summary steps for using turboboot

Considering that turboboot can also erase the config, it is smart to have a back from that
obviously.

same thing with SSH keys and other things like licenses. This short list tries to provide a
couple of checks to take into consideration when doing upgrades, RSP2->RSP440 swaps or
turboboots.

1. Run a cfs check in admin & non-admin mode

2. Copy active licenses and SNMP files to tftp server

3. Copy running config to a tftp-server or laptop

4. Capture "show ipv4 int brief" output to a text file

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
13
ASR9000/XR Understanding Turboboot and initial System bring up

5. Capture "show ipv6 int brief | i Up/Up" output to a text file

6. Offline. Edit the saved RSP config - add "no shutdown" for all physical interfaces that are
up/up from the above IPv4 & IPv6 interface

captures and save cfg changes. Note that it is not necessary to “no shut” sub-
interfaces, only the main physical interface.

7. Connect a laptop console cable to the RSP in RSP0 slot and enable a log file to monitor
and capture the RSP bootup logs.

8 . Turn the power supplies on to power up the asr9k system. (approx. 7-12 minutes)

9. After the LED's indicate IOS-XR on the LC's, and ACTV or STBY on the RSP’s,
log in via the console of the RSP that is ACTV and run some preliminary checks to
check system stability.

NOTE: The default root-system username and password on the RSP440 are root/root

(if root/root does not work also try cisco/cisco, or admin/admin or viking/viking)

10. Verify the ASR9K IOS XR version

11. un a cfs check in admin & non-admin mode

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
14
ASR9000/XR Understanding Turboboot and initial System bring up

12. Optional. Install add & commit any missing SW packages (pies) or required
SMU’s

13. Upgrade FPD in admin mode

14. Reload any nodes that had FPD upgrades

15. Configure the Mgmt ethernet interface with an IP address to reach tftp

server & load and commit the saved RSP config from tftp server or laptop

a) or log into the console and cut & paste a saved cfg from laptop

b) or copy saved cfg from laptop to usb, then insert usb into RSP440 and copy

and commit cfg

c) copy licenses and snmp files back to the RSP’s

16. Optional. create and generate new crypto keys if required.

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
15
ASR9000/XR Understanding Turboboot and initial System bring up

How to update the FPD's

FPD upgrade for all ASR9K devices using FPD.

a) Enter admin mode via the admin command, and capture the output of the current
firmware versions using CLI show hw-module fpd location all. save this output to a
text file. Notice any LC that has a “yes” in the Upg/Dng?  column. This indicates
the FPD should be upgraded or downgraded to match the current FPD version.

b) From admin mode upgrade FPD using the CLI: upgrade hw-module fpd location r/s/m

or if all locations require FPD upgrade (suggested) use CLI:* upgrade hw-module fpd
location all *

Disk Space occupied for each image

Simplest way is to use the ksh df utility.

Install a release and packages and run df:

# df /disk0:

/dev/disk00t77 3813344 733477 3079867 20% /dev/disk0:/

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
16
ASR9000/XR Understanding Turboboot and initial System bring up

Divide the highlighted number by 2000. That gives the approximate size in MB. 366MB in
this case.

Repeat for any other releases we should be interested in.

If you do an upgrade, gather the df output before and after upgrade and compute the
difference in df output.

Related Information

Xander Thuijs, CCIE #6775

Principal Engineer ASR9000

Postings may contain unverified user-created content and change frequently. The content is provided as-is and
is not warrantied by Cisco.
17