<CourseUpgrades

System Title>

LY
N
O
SE
U
AL
N
R
TE
IN
 System Upgrades

LY
N
O
SE
U
AL

Downloading Software Images

N

To upgrade a QFabric system or component, you must obtain the appropriate software image from
Juniper Networks. The downloads are available at http://www.juniper.net/support/downloads. You
R

must have an active service contract with Juniper Networks, including an access account. To obtain
an account, complete the product registration form at https://www.juniper.net/registration/
Register.jsp.
TE

Once you have accessed the software page, choose the link for either the QFX3000-M or QFX3000-G
system. Then click the Software tab.

Complete Install Package

IN

The software download page lists a number of software files available for download. At the top of the
page is the complete install package. This package is an RPM containing the software for the various
components of the QFabric system and is used for standard upgrades as well as nonstop software
upgrades (NSSU). In a NSSU upgrade, the same package is used in multiple steps of the process.
Continued on the next page.

www.juniper.net 2
 System Upgrades
Install Media Images
There are also install media images available for the QFabric system or for individual QFabric system
components. Install media images are primarily used for recovery purposes. Note that loading an
install media image completely overwrites the entire contents of the Director device. It is important
to backup the configuration file to an external device or location. To use the install media image for
the Director device, you must download the .tgz file and unpack the file using the following command
on a UNIX workstation:
%tar -xvzf install-media-qfabric-12.2X50-D10.3.img.tgz
The resulting .img file can then be copied to a USB drive using the following command:
dd if=install-media-qfabric-12.2X50-D10.3.img of=/dev/sdb bs=16k
To recover the QFabric system, insert the USB drive into the Director device and reboot the system.

LY
When prompted to “reinstall the QFabric software on this Director device, type: install”, type the
install command and press Enter. Once the installation completes, the device reboots. You will
be prompted to re-run the initial setup script. At this point you can choose to bypass the initial setup
script and proceed with reloading your configuration from the stored external device or location.

N
O
SE
U
AL
N
R
TE
IN

www.juniper.net 3
 System Upgrades

LY
N
O
SE
U
AL

Component Software Images

N

The software download page also lists install packages for Node and Interconnect devices as shown
on the slide. These files are in the .tgz format and useful for recovery of an individual QFabric
R

component. As mentioned previously, the components also have install media images available for
download to a USB device.
TE

Control Plane Switch Software

The software download page provides quick access to the recommended Junos OS version for
EX Series switches used in the control plane of a QFabric system. The EX Series switches run a
standard EX Series version of the Junos OS. Also note, the recommended virtual chassis
IN

configuration files are also available for download.

www.juniper.net 4
 System Upgrades

LY
N
O
SE
U
AL

Standard Upgrade Overview

N

Performing a standard upgrade with the provided RPM image provides the fastest method for
upgrading a QFabric system and should be used when time is a factor and forwarding resiliency of
R

the data plane is not a factor. All components are upgraded simultaneously. The standard upgrade
procedure consists of four primary steps:
TE

1. Backup current configuration files either locally or externally.

2. Download the appropriate RPM image.
3. On the Director device, retrieve the software image from your local workstation or file
server.
IN

4. Install the software package.

The slide shows a typical RPM image name and its parts.

www.juniper.net 5
 System Upgrades

LY
N
O
SE
U
AL

Back Up Configuration Files

N

Before upgrading the QFabric system, backup the configuration file and initial installation settings by
using the command shown on the slide. The result will be a file with both the Junos OS configuration
R

as well as the initial configuration script settings.

Download Install Package

TE

Next download the install package RPM file to either your workstation or a file server supporting the
FTP or SCP protocols.
IN

www.juniper.net 6
 System Upgrades

LY
N
O
SE
U
AL

Retrieve Software File

N

As shown on the slide, use the request system software download command to transfer
the file from your workstation to the Director device. In this example, SCP was used to transfer the
R

file. However, you can also use FTP. By default the file is placed in the /pbdata/packages directory.
user@qfabric> file list
TE

/pbdata/packages/:
jinstall-qfabric-12.2X50-D10.3.rpm
ais/
IN

Install Software Package

To install the software package, use the request system software add command as shown
on the slide. Note the inclusion of the component all reboot keywords. The system will
commence upgrading the Director, Interconnect, and Node devices. The upgrade can take up to an
hour.

www.juniper.net 7
 System Upgrades

LY
N
O
SE
U
AL

Software Version Verification

N

To verify the software upgrade completed successfully, use the show version component all
command shown on the slide. The output on the slide is trimmed for brevity, but the output displays
R

the Junos OS version on all the QFabric components, which should all match. You should also verify
all the components are once again connected to the system and operational using the show
fabric administration inventory command.
TE
IN

www.juniper.net 8
 System Upgrades

LY
N
O
SE
U
AL

NSSU Overview
N

A nonstop software upgrade enables some QFabric system components to continue operating while
similar components in the system are being upgraded. In general, the QFabric system upgrades
R

redundant components in stages so that some components remain operational and continue
forwarding traffic while their relevant counterparts upgrade to a new version of software.
TE

Nonstop upgrades are useful in situations where the service impact must be minimized. However,
the NSSU process can be time consuming--requiring several hours.
IN

www.juniper.net 9
 System Upgrades

LY
N
O
SE
U
AL

NSSU Prerequisites
N

To qualify for a nonstop upgrade, QFabric systems must be running Junos OS 12.2 or later. Before
beginning the process, verify that all system components are connected and configured using the
R

show fabric administration inventory command. Download the complete install

package and place the package on the Director group device as discussed in the previous section
about standard upgrades.
TE

To minimize traffic impact, sensitive traffic should transit LAGs in redundant server Node groups or
the network Node group. This design allows traffic to continue to flow through one LAG member
interface connected to an operational Node device while the counterpart Node device is rebooting as
part of the upgrade process.
IN

To minimize routing protocol churn, enable graceful restart for supported protocols such as BGP and
OSPF on the network Node group.

www.juniper.net 10
 System Upgrades

LY
N
O
SE
U
AL

Three Primary Steps

N

The slide illustrates the three primary steps in the NSSU process:
1. Upgrade the Director devices.
R

2. Upgrade the fabric components. This includes the Interconnect devices and the Fabric
Control Routing Engines (REs).
TE

3. Upgrade the Node groups, including all server Node groups, redundant server Node
groups and the network Node group.
The steps must be completed in the order shown and all steps must be completed for a successful
IN

nonstop upgrade. Between each step, and each Node group upgrade, it is important to ensure all
components are operational before moving on to the next step. You can verify successful operation
with the show fabric administration inventory command.

www.juniper.net 11
 System Upgrades

LY
N
O
SE
U
AL

Upgrading the Director Devices

N

Use the request system software nonstop-upgrade director-group command to

perform the first step of the NSSU process. By default, the Director device that is not hosting the CLI
R

session is upgraded first. Therefore, we recommend that you execute the upgrade with a CLI session
from the device which hosts the master Fabric Manager and network Node group RE, by issuing the
cli command using a console connection or a direct SSH session. Although mastership switching is
TE

automated, this eliminates the necessity of switching mastership to begin the upgrade.
Once the upgrade is initiated from the master Director group device, the QFabric system installs the
software on the backup Director device and reboots the device. The master device then begins a
15-minute sequence that includes a temporary suspension of services and a database transfer. You
IN

cannot issue operational mode commands in the CLI during this time period.
Next, the QFabric system installs the new software for the fabric manager and diagnostic REs on the
master Director device. The QFabric system switches mastership of all processes to the backup
Director device and reboots the master Director device. The previous master Director devices
resumes operation as a backup device and all associated processes such as the Fabric Manager
and network Node group REs become backup as well.
To verify the Director group devices’ software version, use the show version component
director-group command. Additionally, you can monitor or view the upgrade process with the
/tmp/rolling_upgrade_director.log file. This file resides on the Director device from
which the upgrade command was issued.

www.juniper.net 12
 System Upgrades

LY
N
O
SE
U
AL

Upgrading the Fabric Components

N

Use the request system software nonstop-upgrade fabric command to initiate the
next step of the NSSU process. With this command, the QFabric system downloads, validates, and
R

installs the new software in all Interconnect devices and Fabric Control REs. First, one Fabric Control
RE reboots and comes back online. Then the other Fabric Control RE reboots and comes back
online. Next, the first Interconnect device reboots and comes back online, resuming the forwarding
TE

of traffic. Subsequent Interconnect devices reboot and return to service.

To verify the software version of fabric components, issue the show version component
fabric command. In addition to monitoring the upgrade using the CLI, logging is stored in the
/tmp/perl_DCF.Utils.log file, which is accessible in the shell on the Director device from
IN

which the upgrade was issued. This file contains information pertaining to Node group upgrades
as well.

www.juniper.net 13
 System Upgrades

LY
N
O
SE
U
AL

Upgrading the Node Groups

N

Use the request system software nonstop-upgrade node-group command to

upgrade Node groups. All Node groups must be upgraded to complete the NSSU process. Verify the
R

Node group version with the show version component node-group-name command.
When upgrading a network Node group, the QFabric system copies the new software to each Node
TE

device, one at a time. The QFabric system validates and installs the new software in all Node devices
simultaneously. The QFabric system copies the new software to the network Node group REs. The
system then installs the software in the network Node group REs one at a time. The software is
installed on the backup RE first and then the master network Node group RE. Subsequently, the
backup network Node group RE and its supporting Node devices are rebooted and come back
IN

online, one at a time. Next, the master network Node group RE relinquishes mastership, reboots, and
comes back online.
For redundant server Node group, the QFabric system copies the software to the backup Node
device, then the master Node device. The system validates and installs the software on the backup
Node device, then the master Node device. The backup Node device reboots and becomes the
master Node device. The previous master Node device reboots and comes back online as the
backup Node device. Note that both devices in a redundant server Node group must be online before
issuing the upgrade command.
For server Node groups containing only one Node device, the QFabric system downloads the
software, validates the software, installs the software, and reboots the device. Traffic loss will occur
in a server Node group containing only one Node device.

www.juniper.net 14
 System Upgrades

LY
N
O
SE
U
AL

NSSU Upgrade Groups

N

You can upgrade network Node group devices one at a time or in groups, known as upgrade groups.
Upgrade groups can shorten the time required to perform the NSSU process by upgrading two or
R

more Node devices in a network Node group, or an entire network Node group simultaneously. The
slide illustrates the configuration of NSSU upgrade groups. Note that if you add Node devices that
have links to the same LAG to the same upgrade group, traffic loss can occur.
TE
IN

www.juniper.net 15
 System Upgrades

LY
N
O
SE
U
AL

NSSU Caveats and Considerations

N

The slide lists some caveats and considerations to keep in mind when performing or preparing to
perform an NSSU upgrade process.
R
TE
IN

www.juniper.net 16