ECS EXSeries DellSwitch OS10.4.3.6 CertificateRenewal Rev1.0

ECS: Gen3 Dell Switch OS 10.4.3.
6 Certificate
Renewal Procedure
Affected Product: ECS Appliance Gen 3
Abstract
This document provides the details regarding the July 27, 2021 expiration of
OS10 x.509v3 certificate on Dell Switches, the impact to ECS Appliances and
details on resolving this issue.
June 2021
Revisions
Date Description
June 2021 Initial release
Acknowledgments
Author: Dell Technologies
Acknowledgments
The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in this
publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.
Use, copying, and distribution of any software described in this publication requires an applicable software license.
This document may contain certain words that are not consistent with Dell's current language guidelines. Dell plans to update the document over
subsequent future releases to revise these words accordingly.
This document may contain language from third party content that is not under Dell's control and is not consistent with Dell's current guidelines for Dell's
own content. When such third-party content is updated by the relevant third parties, this document will be revised accordingly.
Copyright © 2021 Dell Inc. or its subsidiaries. All Rights Reserved. Dell Technologies, Dell, EMC, Dell EMC and other trademarks are trademarks of Dell
Inc. or its subsidiaries. Other trademarks may be trademarks of their respective owners. [6/3/2021] [Certificate Renewal]
Certificate Renewal
Table of contents
Table of contents
ECSDOC-946
Revisions.............................................................................................................................................................................1
Acknowledgments ...............................................................................................................................................................1
Table of contents ................................................................................................................................................................3
Summary .............................................................................................................................................................................4
Behavior post expiry ...........................................................................................................................................................4
Resolution Summary...........................................................................................................................................................4
Technical Notes ..................................................................................................................................................................4
Download Service Console .................................................................................................................................................5
Upgrade Service Console to the latest version...................................................................................................................6
Common SC commands used for this procedure ...............................................................................................................7
1.1 Checks for vulnerability in switch certificate .......................................................................................................7
1.2 Procedure for environments with default switch configuration ...........................................................................8
1.3 Procedure for environments with custom Front End switches .........................................................................10
Full output of Service Console Commands ......................................................................................................................12
A Technical support and resources ...............................................................................................................................16
Certificate Renewal
Summary
Summary
This article provides details regarding the July 27, 2021 expiration of OS10 x.509v3 certificate on Dell
Switches, the impact to ECS Appliances, and the details on resolving this issue.
The issue impacts ECS Gen3 Appliances running ECS version 3.4.x or higher. Dell switches in ECS
Appliance models EX300, EX500, and EX3000 use X.509 certificates to establish secure VLT connections
with their switch pair in the same rack. The issue impacts both Front-End (if they are not custom switches)
and Back-End switches X.509v3 certificates expires on July 27, 2021.
Behavior post expiry

• If VLT connection is not interrupted, there is no immediate impact to ECS Appliance.
• If VLT connectivity is interrupted, traffic reachability issues may occur.
- VLT link cannot be reestablished until new certificates are installed.

- VLT link may be lost due to a switch reboot, link flap, operator-triggered configuration change, or
other network issues.
Resolution Summary
• Renew_Switch_Certificates procedure helps you renew OS10 x.509v3 security certificates on ECS
Gen3 Appliances. This procedure is part of ECS Service Console utility.
• ECS Service Console is used by ECS Remote Support and Field personnel to perform service
activities on ECS Appliance.
• To ensure timely resolution of OS10 x.509v3 certificate expiration issue on Dell Switches ECS
Service Console is made available for customer download from https://support.dell.com.
• Renew_Switch_Certificates is the only procedure that can be executed by an ECS Appliance user. All
other procedures available in the utility are limited to execution by Dell personnel only.
• The procedure executes all necessary checks to ensure that certificate renewal is successful. If you
get any error, then stop the procedure and contact ECS Remote Support.
Technical Notes
• Renew_Switch_Certificates procedure is VDC based. It detects if VDC has multiple racks and
updates OS10 x.509v3 switch certificates on all racks.
• Procedure supports multirack environments with mixed platforms and updates switch certificates only
on impacted hardware.
• Procedure takes approximately an hour on single rack system, with an additional 20 minutes per rack
in a multi-rack configuration.
• Maintenance of custom Front End switches is customer responsibility. Procedure does not update
certificates on custom switches. If customer provided switches are Dell switches, then follow Dell
Networking OS10 Certificate Expiration and Solution.
• There is no need to re-run the procedure later. Certificate expiration date is extended by 1000 years.
• Procedure is designed to be non-disruptive. No I/O interruption was detected during tests of the
procedure performed in Dell labs.
Certificate Renewal
Download Service Console
Download Service Console

Follow these steps to download the Service Console.
1. Sign in https://www.dell.com/support/
Note: You must Sign in to display the service-console file.
2. Search for ECS Service Console.

3. Click on Downloads and Drivers on the upper left to narrow your search results.
4. Download the latest service-console (This procedure works with SC6.6 and above).
Service Console download link
Certificate Renewal
Upgrade Service Console to the latest version
Upgrade Service Console to the latest version

Download and install Service Console 6.6 into /tmp/service_console on the installer node (the first node of the
first rack).
Upload service-console-<service_console_version><service_console_build_number>.tgz to
/tmp/service_console directory that is on Node1 and Rack1 of the cluster. This node is also referred to as
Installer node.
If you are not sure which node is the Installer Node, connect to any node in the cluster and then follow the
steps below:
# ssh ECS_Node
# ssh master
# hostname
# provo-gen3-cyan.svt.lab.emc.com
For example, provo-gen3-cyan.svt.lab.emc.com is the name of the Installer node. Upload service
console code to /tmp/service_console directory on provo-gen3-cyan.svt.lab.emc.com host using any SSH
client. Now you are ready to upgrade to the latest service console.
# cd /tmp/service_console
# tar -xf service-console-
<service_console_version><service_console_build_number>.tgz
# ./service-console upgrade
# service-console run Cluster_Config
# admin@provo-gen3-cyan:~> service-console -v
6.X.0.0-XXXXX.XXXXXXXXXX
Certificate Renewal
Common SC commands used for this procedure
1.1 Checks for vulnerability in switch certificate

This command checks the switch vulnerability. It does not perform any disruptive operation.
Note: The status of the procedure displays FAIL if the system is vulnerable.
#service-console run Check_Switch_Certificates
Example:
#service-console run Check_Switch_Certificates

Service Console is running on node 169.254.19.1 (suite
20210503_101239_Check_Switch_Certificates)
Service console version: 6.5.0.0-21423.191e17c371
Debug log: /opt/emc/caspian/service-
console/log/20210503_101237_run_Check_Switch_Certificates/dbg_robot.log
================================================================================
Check Switch Certificates
20210503 10:12:53.696: Validate Switch certificates
Checking switches on Rack Master node 169.254.19.1 (Rack name: auburn)
PSNT: XXXXXXX
[WARN] The switch fox.rack is vulnerable
[WARN] The switch hound.rack is vulnerable
[WARN] The switch rabbit.rack is vulnerable
[WARN] The switch hare.rack is vulnerable
20210503 10:13:52.825: | FAIL (59 sec)
================================================================================
Status: FAIL
Time Elapsed: 1 min 20 sec
console/log/20210503_101237_run_Check_Switch_Certificates/dbg_robot.log HTML log:
/opt/emc/caspian/service-
console/log/20210503_101237_run_Check_Switch_Certificates/log.html
================================================================================
Messages: Problematic rack(s): auburn.
================================================================================
The following error displays in an environment that has custom FE switches:
“Not able to connect to FE switches. These could be customer provided switches. See details in KB 185695”.
• If you are not sure of the error, contact ECS Remote Support.
• If the message is accurate and your environment contains custom Front End switches, then proceed
to Section1. 3 Procedure for environments with custom Front End switches
Certificate Renewal
1.2 Procedure for environments with default switch configuration

This procedure performs switch certificate renewal on all 4 switches in the rack. This Procedure flaps VLT
links after passing all health checks. It may take several minutes for the VLT links to come back on-line. The
status is checked every 15 seconds.
#service-console run Renew_Switch_Certificates
For multirack VDCs: The procedure updates all the impacted racks in a VDC. It is possible that only a subset
of the racks pass Renew_Switch_Certificates prechecks. Contact ECS Remote Support if a subset of the
racks does not pass the prechecks.
Example:
#service-console run Renew_Switch_Certificates

Service Console is running on node 169.254.19.1 (suite
20210503_101500_Renew_Switch_Certificates)
Service console version: 6.5.0.0-21423.191e17c371
console/log/20210503_101458_run_Renew_Switch_Certificates/dbg_robot.log
================================================================================
Renew Switch Certificates
<truncated, see Additional Info section for full output>
Checking public switches nodes links…
Checking Rack: auburn...
Checking switch: rabbit.rack...
Checking switch: hare.rack...
20210503 10:26:02.250: | | PASS (2 min 2 sec)
20210503 10:26:02.252: | Check BE switches links
Checking private switches nodes links...
Checking switch: fox.rack...
Checking switch: hound.rack...
20210503 10:26:46.364: | | PASS (44 sec)
20210503 10:26:46.366: | PASS (11 min 31 sec)
Flapping VLT links on the primary switch fox.rack.
VLT links on the primary switch fox.rack are flapped.
Checking switches links after flap
Links aren't up, sleeping 15 sec...
Links are up
Certificate Renewal
Running Health Checks in the middle of certificates refreshment...

20210519 18:28:49.415: | Check DT status
================================================================================
Status: PASS
HTML log: /opt/emc/caspian/service-
console/log/20210503_101458_run_Renew_Switch_Certificates/log.html
Certificate Renewal
1.3 Procedure for environments with custom Front End switches

This procedure performs switch certificate renewal only on BE switches and skips FE switches.
This Procedure flaps VLT links after passing all the health checks. It may take several minutes for the VLT
links to come back on-line. The status is checked every 15 seconds.
Note: This procedure should be followed only when you use the custom Front End Switches and not the
default Dell Switches.
#service-console run Renew_Switch_Certificates --skip-fe-switch true
Example:
admin@provo-auburn:~> service-console run Renew_Switch_Certificates --skip-fe-switch

true
2021-05-26 11:39:00 main#1405 INFO: Redirecting console output to
console/log//20210526_113859_run_Renew_Switch_Certificates/output.log
Procedure is running inside a 'screen' session. If you are disconnected, the
procedure will continue to run. To reconnect to your running procedure session in
the future, run 'screen -r service-console'
-
provo-auburn 05/26 11:39
Service console version: 6.6.0.0-21464.adf8539ff2
================================================================================
Running Health Checks before certificates refreshment ...
20210526 11:39:18.928: | Check that the system is not in TSO state
20210526 11:39:26.637: | | PASS (7 sec)
20210526 11:39:26.639: | Check DT status
Checking DT status (with timeout 10 min).
20210526 11:39:39.391: | | PASS (12 sec)
20210526 11:39:39.392: | Validate that all nodes are available - OS
20210526 11:39:39.807: | | PASS
20210526 11:39:39.809: | Check network interfaces
20210526 11:39:40.759: | | PASS
20210526 11:40:25.078: | | PASS (44 sec)
20210526 11:40:25.081: | Check BE switches ntp
Certificate Renewal
Checking ntp status on Rack Master node 169.254.19.1 ( Rack name: auburn ) PSNT:
psnt1
Public switches are skipped
Running Health Checks after all switches certificates refreshment ...
20210526 11:45:09.431: | | PASS
20210526 11:45:09.432: | Check DT status
20210526 11:45:20.048: | | PASS (10 sec)
20210526 11:45:20.051: | | PASS
20210526 11:45:20.970: | | PASS
20210526 11:46:18.136: | | PASS (57 sec)
psnt1
NTP is configured on the switch fox.rack
NTP is configured on the switch hound.rack
20210526 11:46:46.325: | | PASS (28 sec)
20210526 11:46:46.326: | PASS (7 min 30 sec)
================================================================================
Status: PASS
Certificate Renewal
Full output of Service Console Commands

service-console run Renew_Switch_Certificates
2021-05-26 10:51:34 main#1405 INFO: Redirecting console output to
console/log//20210526_105134_run_Renew_Switch_Certificates/output.log
Procedure is running inside a 'screen' session. If you are disconnected, the
procedure will continue to run. To reconnect to your running procedure session in
the future, run 'screen -r service-console'
-
provo-auburn 05/26 10:51
Service console version: 6.6.0.0-21464.adf8539ff2
================================================================================
Running Health Checks before certificates refreshment ...
20210526 10:52:02.062: | | PASS (8 sec)
20210526 10:52:02.064: | Check DT status
20210526 10:52:15.405: | | PASS (13 sec)
20210526 10:52:15.809: | | PASS
20210526 10:52:16.745: | | PASS
20210526 10:52:16.747: | Check FE switches links
Checking public switches uplinks...
WARNING: Switch rabbit.rack has VLT port 120 configured.
WARNING: Switch hare.rack has VLT port 120 configured.
Checking public switches nodes links...
20210526 10:54:21.261: | | PASS (2 min 4 sec)
20210526 10:54:21.263: | Check FE switches ntp
psnt1
Private switches are skipped
NTP is configured on the switch rabbit.rack
NTP is configured on the switch hare.rack
20210526 10:54:49.325: | | PASS (28 sec)
Certificate Renewal
20210526 10:54:49.328: | Check if FE switches vlt-mac is configured correctly

Checking MAC addresses on Rack Master node 169.254.19.1 ( Rack name: auburn ) PSNT:
psnt1
Configured vlt-mac address of the switch rabbit.rack is d8:9e:f3:c0:7e:00
Configured vlt-mac address of the switch hare.rack is d8:9e:f3:c0:7e:00
20210526 10:55:22.425: | | PASS (33 sec)
20210526 10:56:08.493: | | PASS (46 sec)
psnt1
20210526 10:56:37.183: | | PASS (28 sec)
20210526 10:56:37.185: | Check if BE switches vlt-mac is configured correctly
Checking MAC addresses on Rack Master node 169.254.19.1 ( Rack name: auburn ) PSNT:
psnt1
Configured vlt-mac address of the switch fox.rack is d8:9e:f3:c0:81:00
Configured vlt-mac address of the switch hound.rack is d8:9e:f3:c0:81:00
20210526 10:57:10.845: | | PASS (33 sec)
Checking switches on Rack Master node 169.254.19.1 ( Rack name: auburn ) PSNT: psnt1
The switch fox.rack is primary
The switch fox.rack is vulnerable - trying to fix...
The switch fox.rack is not vulnerable now, certs are updated.
The switch hound.rack is vulnerable - trying to fix...
The switch hound.rack is not vulnerable now, certs are updated.
Flapping VLT links on the primary switch fox.rack.
VLT links on the primary switch fox.rack are flapped.
Links are up
Certificate Renewal
The switch rabbit.rack is vulnerable - trying to fix...

The switch rabbit.rack is not vulnerable now, certs are updated.
The switch hare.rack is primary
The switch hare.rack is vulnerable - trying to fix...
The switch hare.rack is not vulnerable now, certs are updated.
Flapping VLT links on the primary switch hare.rack.
VLT links on the primary switch hare.rack are flapped.
Links are up
Running Health Checks after all switches certificates refreshment ...
20210526 11:05:01.340: | | PASS
20210526 11:05:01.341: | Check DT status
20210526 11:05:12.118: | | PASS (10 sec)
20210526 11:05:12.120: | | PASS
20210526 11:05:13.127: | | PASS (1 sec)
20210526 11:05:13.129: | Check FE switches links
Checking public switches uplinks...
WARNING: Switch rabbit.rack has VLT port 120 configured.
WARNING: Switch hare.rack has VLT port 120 configured.
Checking public switches nodes links...
20210526 11:07:14.915: | | PASS (2 min 1 sec)
20210526 11:07:14.917: | Check FE switches ntp
psnt1
NTP is configured on the switch rabbit.rack
NTP is configured on the switch hare.rack
Certificate Renewal
20210526 11:07:42.714: | | PASS (27 sec)

20210526 11:08:26.976: | | PASS (44 sec)
psnt1
20210526 11:08:54.737: | | PASS (27 sec)
20210526 11:08:54.738: | PASS (17 min 3 sec)
================================================================================
Status: PASS
================================================================================
Certificate Renewal
A Technical support and resources

Dell.com/support is focused on meeting customer needs with proven services and support.
Certificate Renewal

ECS EXSeries DellSwitch OS10.4.3.6 CertificateRenewal Rev1.0

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ECS EXSeries DellSwitch OS10.4.3.6 CertificateRenewal Rev1.0

Uploaded by

Copyright:

Available Formats

ECS: Gen3 Dell Switch OS 10.4.3.

Behavior post expiry

- VLT link cannot be reestablished until new certificates are installed.

Download Service Console

Note: You must Sign in to display the service-console file.

2. Search for ECS Service Console.

Service Console download link

Upgrade Service Console to the latest version

Common SC commands used for this procedure

1.1 Checks for vulnerability in switch certificate

#service-console run Check_Switch_Certificates

#service-console run Check_Switch_Certificates

The following error displays in an environment that has custom FE switches:

1.2 Procedure for environments with default switch configuration

#service-console run Renew_Switch_Certificates

#service-console run Renew_Switch_Certificates

Running Health Checks in the middle of certificates refreshment...

1.3 Procedure for environments with custom Front End switches

#service-console run Renew_Switch_Certificates --skip-fe-switch true

admin@provo-auburn:~> service-console run Renew_Switch_Certificates --skip-fe-switch

Full output of Service Console Commands

20210526 10:54:49.328: | Check if FE switches vlt-mac is configured correctly

The switch rabbit.rack is vulnerable - trying to fix...

20210526 11:07:42.714: | | PASS (27 sec)

A Technical support and resources

You might also like