Isilon - ESRS Cluster

EMC ® Isilon ® SolVe Generator
Solution for Validating your engagement
Topic
Isilon Miscellaneous
Selections
Select Miscellaneous Activity: How to Enable EMC Secure Remote Support on Isilon Clusters
Generated: 4:01 PM >

November 14, 2016
SolVe Generator Updated:
REPORT PROBLEMS
If you find any errors in this procedure or have comments regarding this application, send email to
SolVeFeedback@emc.com
Copyright© 2012 – 2021 EMC Corporation. All rights reserved.

Publication Date: November, 2016
EMC believes the information in this publication is accurate as of its publication date. The information is subject to
change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION MAKES NO
REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS
PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
FOR A PARTICULAR PURPOSE.
Use, copying, and distribution of any EMC software described in this publication requires an applicable software
license.
For the most up-to-date regulatory document for your product line, search for “regulatory” on the applicable product
page at https://support.EMC.com
For the most up-to-date listing of EMC trademarks, see the list of EMC Corporation Trademarks on EMC.com.
All other trademarks used herein are the property of their respective owners.
EMC CONFIDENTIAL version: 4.4.4.0

1 of 12
Contents
How to Enable EMC Secure Remote Services on Isilon Clusters..........................................3
EMC ISILON INTERNAL ONLY..............................................................................................3
Abstract........................................................................................................................................................ 3
Publication History....................................................................................................................................... 3
About EMC Secure Remote Services.....................................................................................3

Expectations................................................................................................................................................ 4
Prerequisites................................................................................................................................................ 4
Procedures.................................................................................................................................................. 5
Check the current status of SupportIQ.................................................................................................... 5
Use the OneFS web administration interface......................................................................................... 5
Use the OneFS command-line interface................................................................................................. 5
Enable ESRS ConnectHome on the cluster................................................................................................. 5
Verification................................................................................................................................................... 6
Verify that Isilon receives a test event.......................................................................................................... 6
Verify the cluster can send log files.............................................................................................................. 6
Verify the cluster details in SYR................................................................................................................... 7
Verify the node is visible in ServiceLink....................................................................................................... 8
Obtain the remotesuppport user password.................................................................................................. 8
Verify remote access using CLIviaSSH....................................................................................................... 8
Log Files...............................................................................................................................11
Known Issues.......................................................................................................................11
ESRS Resources..................................................................................................................11

2 of 12
How to Enable EMC Secure Remote Services on Isilon

Clusters
EMC ISILON INTERNAL ONLY
Abstract
This document is for Isilon Technical Support to use when helping customers configure EMC Secure
Remote Services on Isilon clusters. It also contains instructions on how Isilon Technical Support uses
ServiceLink to remotely connect to a customer’s cluster.
July 2021
Publication History
Table 1
Date Description
October 29, 2013 Initial publication.
November 17, 2014 Replaced apostrophe with tick marks in password generation command.
March 3, 2015 Removed details about the deprecated clusterPasswordGenerator.
Added note recommending the use of a management subnet with static IPs.
May 5, 2015 Renamed EMC Secure Remote Support to EMC Secure Remote Services.
Updated references to ESRS Gateway
About EMC Secure Remote Services

EMC Secure Remote Services (ESRS) is a two-way, secure, IP-based customer service support system
that allows 24x7 remote monitoring of EMC products. ESRS is supported on Isilon clusters that run
OneFS 7.1 and later.
The ESRS application architecture consists of a secure, asynchronous messaging system designed to
support the functions of secure encrypted file transfer, monitoring of device status, and remote execution
of diagnostic activities. This distributed solution is designed to provide a scalable, fault-tolerant, and
minimally intrusive extension to the customer’s system support environment.
By implementing the optional ESRS Policy Manager, customers can enable monitoring on a node-by
node basis, allow or deny remote support sessions, and review remote customer service activities.
Access to customer clusters is provided by the ServiceLink application. ServiceLink is the graphical user
interface component of the ESRS-IP Solution. ServiceLink can be used to search for devices, check for
connectivity status, and create remote access sessions to specific devices.
ESRS and SupportIQ perform many of the same functions, including:
 Sending alerts regarding the health of the cluster.

3 of 12
 Enabling support personnel to run the same scripts used by SupportIQ to gather node and cluster
information.
 Allowing support personnel to establish remote access to troubleshoot Isilon clusters.
Only one remote support system should be enabled on an Isilon cluster. The EMC products in use in the
customer’s environment determine which system is most appropriate:
 If the customer’s environment includes one or more EMC products that can be monitored, use ESRS.
 If ESRS is currently implemented in the environment, use ESRS.
 If the customer's implementation of ESRS requires the ESRS Client, use SupportIQ. Isilon nodes do
not currently support ESRS Client connectivity.
 If the customer's cluster is in a high-security environment, use ESRS.
 If the only EMC products in the customer’s environment are Isilon nodes, use SupportIQ.
For additional information, see the Isilon OneFS 7.2 CLI Administration Guide on the EMC Online Support
site.
Expectations
When set up correctly, ESRS will:
 Allow remote access to the remote command-line interface using the remotesupport account.
Command line interaction with the remotesupport user account is limited to running pre-configured
scripts installed on the cluster. Customers may choose to allow root access to the Isilon web
administration interface or command-line interface by providing Isilon Support with the appropriate
credentials.
 Create a System Report (SYR) and Services Request (SR) is created in the Customer Replaceable
Unit (CRU) Queue when a device failure occurs. The SYR will contain the cluster name, GUID and
identify all nodes within the cluster.
 Automatically provision new nodes into the ESRS gateways and automatically approve the nodes in
ServiceLink.
 List decommissioned nodes as “Missing” in ServiceLink and the ESRS gateway within 4 hours. Within
14 days, nodes will be removed automatically from ServiceLink and the ESRS gateway.
ote: Any node that is automatically removed from the ESRS gateway will need to be manually re-added
to the ESRS gateway if the node is re-added to the cluster.
Prerequisites
The following prerequisites need to be met before you can remotely connect to a customer’s cluster using
ServiceLink.
 ESRS 2.24 or later must be installed in the customer’s environment, on dedicated, physical hardware,
or a virtual machine. Refer to the ESRS documentation on the EMC Online Support site
 All nodes visible in the Install Base with a Status of Shipped, Install, or Install/T&M.
 PuTTY or similar terminal application installed on your workstation.
 RSA SecureID log in credentials to access ServiceLink.
 Internet Explorer 8 with Java enabled.

4 of 12
 SupportIQ disabled on the cluster.
Procedures
Check the current status of SupportIQ
Do one of the following to check the current status of SupportIQ and disable, if required:
Use the OneFS web administration interface

1. [ ] Log in to the OneFS web administration interface.
2. [ ] Go to Cluster Management > General Settings > SupportIQ
3. [ ] Verify that the box next to Enable SupportIQ is not checked. If the box is checked, click
the box to uncheck it.
4. [ ] Click Submit.
Use the OneFS command-line interface

1. [ ] Open an SSH connection on any node in the cluster and log in using the root account.
2. [ ] Run the following command to verify the status of SupportIQ:
isi_promptsupport –s | grep enabled
If the output that appears is:
SupportIQ enabled: : Disabled
SupportIQ is disabled and no further action is required.
If the output that appears is:
SupportIQ enabled: : Enabled
disable SupportIQ with the following command:
isi_promptsupport
Type no at the prompt and then press ENTER.
SupportIQ is now disabled.
Enable ESRS ConnectHome on the cluster

2. [ ] Run the following command, where <GW1> is the IP address or host name of the primary
ESRS gateway (required), and <GW2> is the IP address or host name of the secondary ESRS
gateway (optional) and <subnet> is the subnet on the Isilon cluster to be used for remote support
connections, for example subnet0:
isi remotesupport connectemc modify --enabled=yes --primary-esrs-gateway=<IP1> --secondary-

esrs-gateway=<IP2> --remote-support-subnet=<subnet>
ote: It is recommended that you use a separate management subnet with a static IP address pool in the
System access zone. If a separate subnet is not available then the subnet chosen should have static IPs
and must be in the System access zone.
3. [ ] Verify the settings by running the following command:

isi remotesupport connectemc view
Output similar to following appears:
Enabled: Yes

5 of 12
Primary Esrs Gateway: eng-sea-esrs1

Secondary Esrs Gateway:
Use Smtp Failover: No
Email Customer On Failure: No
Remote Support Subnet: subnet0
After ESRS is enabled on the cluster, any additional nodes added to the cluster will be automatically
provisioned into the ESRS gateways.
ote: To disable ESRS on the cluster, run the following command:
isi remotesupport connectemc modify --enabled=no
Verification
After ESRS has been enabled on the cluster, perform the following verification steps to ensure that the
cluster can successfully communicate with the ESRS gateway, that log files are successfully uploaded to
Isilon and that the remotesupport user can log in to the command-line interface.
Verify that Isilon receives a test event

2. [ ] Run the following command to send a test event:
isi events sendtest
3. [ ] Log on to SYR and verify that the test event is logged. For more information, see article
196620 on the EMC Online Support site.
Verify the cluster can send log files

2. [ ] Run the following command to gather cluster logs:
isi_gather_info --esrs
If the upload is successful, output similar to the following appears:
> User-Agent: isi_gather_info

> Host: esrs.emc.com
> Accept: */*
> SerialNumber: G095200041J
> GUID: 003048f0f0022d4edb51db0b25df245f686e
> ProductFamily: Isilon
> Content-Length: 4715678
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< Date: Fri, 04 Oct 2013 19:19:27 GMT
< Server: Microsoft-HTTPAPI/2.0
< WWW-Authenticate: Basic realm="CT"

6 of 12
< Content-Length: 0
< Connection: close
< Content-Type: application/x-gzip
<
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
Cleaning up temporary data... done.
Gather-status unlocked
ote: If you see a response other than < HTTP/1.1 200 OK, as highlighted in the output above, the
upload process was not successful.
After the log gather is uploaded to Isilon Support, the files will be extracted by the log processor. You can
search for the log files on the IGS Tools log page, http://tools2.igs.corp/.
Verify the cluster details in SYR

Perform the following steps to verify that the cluster’s configuration details (serial number, cluster GUID,
OneFS version, node status, logical node number (LNN) and cluster name) are visible in SYR.
1. [ ] Log on to SYR at http://omega.eng.emc.com.

2. [ ] Click Click here to go to your Default Application.
3. [ ] Click the right arrow on the right side of the page until the Isilon link appears in the
product ribbon as shown below.
Figure 1
4. [ ] Click Isilon.

5. [ ] Click Detail Query.
6. [ ] Click Configuration Query.
7. [ ] In the Node SN box, type a node serial number of one of the nodes in the cluster.
8. [ ] Click Run Report.
Output similar to the following table appears:
Figure 2

7 of 12
Verify the node is visible in ServiceLink

1. [ ] Log on to https://esrs.emc.com using your RSA SecureID log in.
2. [ ] In the Browse for Devices section, enter a node serial number from any node in the
cluster.
3. [ ] Click Search.
The node serial number should appear in the Browse for Devices window similar to what is shown in the
following image:
Figure 3
ote: Note: For additional information about using the ServiceLink web user interface, click the Help link
in the upper right corner of the ServiceLink window.
Obtain the remotesuppport user password

You must obtain the remotesupport user account password in order to access the command shell on the
node. Have the customer run the following command to generate the password:
echo `isi_generate_remote_support_password`
ote: Note: The password is generated using the cluster’s global unique identifier (GUID). It is possible
for the root user to change the password for the remotesupport user using the passwd command. If you
are unable to log in with the generated password, contact the customer to obtain the new password.
Verify remote access using CLIviaSSH

1. [ ] Open a new browser window and log on to https://esrs.emc.com using your RSA
SecureID log in.
2. [ ] In the Browse for Devices section, enter a node serial number.
3. [ ] Click Search.
4. [ ] In the Browse for Devices window, click the serial number link in the Device column.
5. [ ] In the ESRS Remote Sessions pane, click CLIviaSSH.
6. [ ] Enter SR information and notes into the appropriate boxes.
7. [ ] Click Start Session.

8 of 12
8. [ ] When the Remote Session Applet window opens, wait for Ready to Launch to display
in the Session Status column.
9. [ ] Note the IP address listed in the IP column.
10. [ ] Launch PuTTY (or other terminal application).
11. [ ] Type the IP address from step 9 into the Host Name (or IP address) box.
12. [ ] Click Open.
13. [ ] At the login prompt, type remotesupport.
14. [ ] At the Password prompt, type the cluster password from step 3 in the previous section.
15. [ ] At the remotesupport> prompt, type one of the commands in the following table:
Table 2
Command Description
Help Lists all of the available commands that can be run.
GetATAControlAndGMirror Runs the atacontrol list & gmirror status
commands and uploads the output.
GetClusterEvents Runs the isi events command and uploads the
output.
GetClusterStatus Runs the isi status command and uploads the
output.
GetContactInfo Collects the contact information entered in the
General Settings page in the OneFS web
administration interface by parsing specific
information from the isi_gconfig celog
command.
GetContents-isi_client_stats Runs the isi_client_stats command to
collect pcap client statistics.
GetNfsClients Runs the isi_for_array mount -v command
to see if any nodes are being used as NFS clients
(the number one cause of failed gathers).
GetReadonlyStatus Runs the isi readonly command to report the
read-only status of all nodes in the cluster.
IsiGatherInfo Runs the isi_gather_info command to collect
and upload cluster log information.
IsiGatherInfo-Incremental Runs the isi_gather_info --incremental
command to collect the latest log set since the most
recent full isi_gather_info command.
IsiGatherInfo-Incremental-SingleNode Runs the isi_gather_info --local
--incremental command to collect and upload
cluster log information from the node that the
command is run on.
IsiGatherInfo-SingleNode Runs the isi_gather_info --local command
to collect and upload cluster log information from
the node that the command is run on.

9 of 12
clean_watch_folders Runs the rm --f command to delete all files in the

various SupportIQ upload directories in /var/crash/
generate_dashboard_file_daily Generates the dashboard.xml file.
generate_dashboard_file_interval Generates the dashboard.xml file.
get_contents_var_crash Runs the tar -zcvf /var/crash command to
collect the contents of /var/crash.
get_data_abr Runs the isi_gather_info -I --nologs
--group abr command to collect and upload as-
built records from CTO nodes.
get_data_application Runs the isi_gather_info -I --nologs
--group application command to gather and
upload data about OneFS applications,
(SnapshotIQ, SyncIQ, SmartQuotas), and firmware
status.
get_data_cluster Runs the isi_gather_info -I --nologs
--group cluster command to gather and
upload data about overall cluster configuration and
operation.
get_data_domain Runs the isi_gather_info -I --nologs
--group domain command to collect details
about the cluster’s Active Directory domain
membership.
get_data_fs Runs the isi_gather_info -I --nologs
--group fs command to collect and upload file
system information.
get_data_ib Runs the isi_gather_info -I --nologs
--group ib command to collect and upload data
about the configuration and operation of the
InfiniBand back-end network.
get_data_logs Runs the isi_gather_info --nologs
--group logs command to collect and upload
the contents of the /var/log directory.
get_data_messages Runs the isi_gather_info --nologs
--group messages command to collect and
upload the current active /var/log/messages files.
get_data_network Runs the isi_gather_info -I --nologs
--group network command to collect and
upload a list of network configuration and status,
open sockets, client statistics and FlexNet
configuration.
get_data_node Runs the isi_gather_info -I --nologs
--group node to collect node status information.
get_data_protocol Runs the isi_gather_info -I --nologs
--group protocol command to collect and
upload status information and configuration settings
for the NFS, SMB, FTP, and HTTP protocols.
get_data_usage Runs the isi_gather_info -I --nologs
--group usage command. Collects and uploads

10 of 12
current and information about node performance

and resource usage.
get_job_status Runs the isi job status –v command and
uploads the output.
restart_node Runs the shutdown -r now command and
restarts the node. Only run this command with the
customer’s approval.
tail_messages Runs the /usr/bin/tail -n 1000
/var/log/messages command to collect and
upload the last 1,000 lines in the /var/log/messages
log file from the node that the command is run on.
upload_messages Collects and uploads the /var/log/messages log file
from the node that the command is run on.
ote: All collected log files are uploaded using the --esrs option of the isi_gather_info command
to send the files securely to the ESRS gateway.
1. [ ] Type CTRL+D to close the PuTTY application and end the remotesupport session.
2. [ ] In the Remote Session Applet window, click the X icon to terminate the application.
3. [ ] Close the Remote Session Applet window.
4. [ ] Close the ServiceLink window.
Log Files
 /var/log/rscApiLog.log
 /var/log/ConnectEMC.log
 /var/log/ConnectEMCHttp.log
 /var/log/ConnectEMC_HTTP.log
 /var/log/GWExt.log
 /var/log/GWExtHTTPS.log
 /var/log/auth.log
 /opt/connectemc/logs/ConnectEMC.log
Known Issues
 Accessing ServiceLink from Mac OS browsers is not supported.
ESRS Resources
 ESRS Collateral & Resources, http://one.emc.com/clearspace/docs/DOC-76305
 ESRS Secure Remote Services, https://support.emc.com/products/31755_Secure-Remote-Support
 EMC Secure Remote Services Forum, https://community.emc.com/community/support/esrs
 Isilon Troubleshooting Guide: Administration - SupportIQ and ESRS

11 of 12

12 of 12

Isilon - ESRS Cluster

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Isilon - ESRS Cluster

Uploaded by

Copyright:

Available Formats

EMC ® Isilon ® SolVe Generator

Solution for Validating your engagement

Generated: 4:01 PM >

Copyright© 2012 – 2021 EMC Corporation. All rights reserved.

EMC CONFIDENTIAL version: 4.4.4.0

About EMC Secure Remote Services.....................................................................................3

EMC CONFIDENTIAL version: 4.4.4.0

How to Enable EMC Secure Remote Services on Isilon

About EMC Secure Remote Services

EMC CONFIDENTIAL version: 4.4.4.0

EMC CONFIDENTIAL version: 4.4.4.0

 SupportIQ disabled on the cluster.

Use the OneFS web administration interface

Use the OneFS command-line interface

Enable ESRS ConnectHome on the cluster

isi remotesupport connectemc modify --enabled=yes --primary-esrs-gateway=<IP1> --secondary-

3. [ ] Verify the settings by running the following command:

EMC CONFIDENTIAL version: 4.4.4.0

Primary Esrs Gateway: eng-sea-esrs1

ote: To disable ESRS on the cluster, run the following command:

isi remotesupport connectemc modify --enabled=no

Verify that Isilon receives a test event

isi events sendtest

Verify the cluster can send log files

If the upload is successful, output similar to the following appears:

> User-Agent: isi_gather_info

EMC CONFIDENTIAL version: 4.4.4.0

Verify the cluster details in SYR

1. [ ] Log on to SYR at http://omega.eng.emc.com.

4. [ ] Click Isilon.

Output similar to the following table appears:

EMC CONFIDENTIAL version: 4.4.4.0

Verify the node is visible in ServiceLink

Obtain the remotesuppport user password

Verify remote access using CLIviaSSH

EMC CONFIDENTIAL version: 4.4.4.0

EMC CONFIDENTIAL version: 4.4.4.0

clean_watch_folders Runs the rm --f command to delete all files in the

EMC CONFIDENTIAL version: 4.4.4.0

current and information about node performance

EMC CONFIDENTIAL version: 4.4.4.0

EMC CONFIDENTIAL version: 4.4.4.0

You might also like