Isilon OneFS 8.1.

2
Technical Update
 What is new in Isilon OneFS 8.1.2

• Hadoop 3 compatibility

• Isilon management pack for Ambari

• Enhanced WebHDFS support

• LLAP support

• Ranger with SSL support

2 © Copyright 2018 Dell Inc.

 Hadoop 3 Compatibility
Overview

Apache Hadoop 3 became GA in December 2017

HDP3 became GA on July 13 2018
Isilon HDFS rpc’s and existing integration points need to be
validated
Key Value to ensure compatibility
Propositions
• Existing Isilon Hadoop customers can upgrade to HDP3 without disrupting any workloads
Cloudera CDH 6 compatibility validation will be completed
• New customers can leverage the latest Hadoop3 ISV (and Apache) stacks with Isilon
when it goes GA

3 © Copyright 2018 Dell Inc.

 Hadoop 3 Compatibility

• Hadoop 3.0 and 3.1 introduce some small behavioral changes in the HDFS rpc calls

– OneFS needs to make adjustments to these protocols level interactions (HDFS client requests and
WebHDFS responses)

– Changes are totally transparent to Hadoop users or administrators

– Note that some of these changes are in Apache Hadoop 2.9, but no commercial ISV vendors have shipped
their stacks with 2.9

• Significant work was done in:

– Moving test infrastructure to CentOS 7 (required by the Java version that Hadoop 3.0 uses)

– Finding and documenting the configuration/behavior changes

4 © Copyright 2018 Dell Inc.

 Isilon Management Pack
Overview

Ambari will use Management Pack model to integrate with

third party products with HDP3
Isilon Management Pack is an agentless integration approach
that requires no Isilon update with new Ambari version
Key Value Propositions
Ambari UI will also be Isilon aware – only relevant config will
• Deployment and upgrading of HDP with Isilon become much simpler
• be shown
Ambari UI is Isilon-aware and managing/deploying Isilon in Ambari is much easier
Ambari Metrics with Isilon will continue to work with this new
model

5 © Copyright 2018 Dell Inc.

 Isilon Management Pack
• With Ambari Isilon Management Pack, Isilon OneFS is a new service in Ambari that can be easily
installed during Ambari deployment

• Eliminates unnecessary communication between Ambari Manager and OneFS

• Presents only settings that are needed to deploy OneFS (e.g. SmartConnect address)

• Isilon Management Pack is also release independent. Any update can be done outside of OneFS
release cycle.

6 © Copyright 2018 Dell Inc.

 Enhanced WebHDFS Support
Overview

Prior to OneFS 8.1.2, WebHDFS implementations had

limitations:
Chunked Encoding not supported, causing Knox
integration and other native WebHDFS usage problems
Key Value Proposition
Delegation
 Improved Token challenges that prevent many Ambari
WebHDFS implementation will enable all Hadoop Isilon customers to use key
Viewsandfrom
management working
security properly
Hadoop features seamlessly
 Isilon will support key Ambari functions and other security tools that need WebHDFS features similar
OneFS
to DAS 8.1.2 addresses both of these issues, with

incremental performance improvement leveraging multi-

threading enhancements
7 © Copyright 2018 Dell Inc.
 Enhanced WebHDFS Support
• No configuration required in OneFS; changes are made in the protocol level

– Delegation Tokens support over WebHDFS, completing our Kerberos story

– Chunked Encoding for writing data over WebHDFS

• Many usability improvements:

– Ambari Views work with Kerberos

– Hue works with Kerberos

– Apache Knox works natively with OneFS (without the need of httpfs workaround)

– All applications or tools that rely on WebHDFS should work with OneFS

– Multithreading for file writes over WebHDFS (instead of single threading today)

8 © Copyright 2018 Dell Inc.

 LLAP (Low Latency Analytical Processing) Support
Overview

HDP 2.6 began supporting LLAP with Hive for interactive

SQL
Hive with LLAP is a similar feature as CDH’s Impala
Isilon
Key uses support “reference by inode” to properly support
Value Propositions

LLAP caching consistency with HDFS layer

• Customer can run both batch mode and interactive SQL with Hadoop on Isilon
• Isilon supports LLAP similar to DAS

9 © Copyright 2018 Dell Inc.

 LLAP (Low Latency Analytical Processing) Support

• LLAP uses client-side caching of column data to reduce queries to the DataNode.

– Require the use of “Reference by inode” to ensure cache consistency with HDFS layer

• With “Reference by inode” in OneFS, HDFS clients will

– Get inode (Hadoop lin) for a file or directory

– Use that same inode on future queries to ensure the same resource is accessed (even after rename, move,
etc). More details at https://hortonworks.com/tutorial/interactive-sql-on-hadoop-with-hive-llap/

• Require no configuration on OneFS, but must be enabled on the clients.

• “Reference by inode” is currently only used by Hive/LLAP, but can also be used by other
applications in the future

10 © Copyright 2018 Dell Inc.

 Ranger with Secure Communication (SSL)
Overview
• Enterprise customers with high security standards want to encrypt all traffic in a Hadoop cluster
• HDP2.5 started supporting 1-way SSL in Ranger and Isilon OneFS 8.1.2 supports this

Key Value Proposition

 Enterprise customers who want secure integration between Ranger plug-in and Policy Manager can
achieve this with OneFS 8.1.2

11 © Copyright 2018 Dell Inc.

 Ranger with Secure Communication (SSL)

• Today, OneFS uses unsecure HTTP communication to download policies from Ranger Policy Server

• OneFS supports One-Way SSL with Kerberos to secure communication with Ranger Policy Server:

– After Kerberizing OneFS and HDP, OneFS uses SPNEGO (HTTP Kerberos principal) to communicate with Ranger.

– Ranger is configured to use a certificate authority and issues a certificate in its response to OneFS, completing One-
Way SSL.

• This work also enable secure SSL communication between OneFS and Ambari Server (for pre-HDP 3.0
deployments) and Ambari Metrics Server using Kerberos

– The new Isilon Management Pack does not require Ambari Agent/Server communication and hence this is not
applicable

12 © Copyright 2018 Dell Inc.

OneFS 8.1.1 Technical Update
 What is new with Isilon OneFS 8.1.1

• Cloudera Navigator support

• SMB3 over the wire encryption

• Google cloud support for CloudPools

• Patch signing

• New health check framework

• Multi-national language support

• Non-disruptive upgrades

14 © Copyright 2018 Dell Inc.

 Cloudera Navigator Background

• Provides comprehensive governance and data stewardship in Hadoop environments

• Application for end-to-end lineage and metadata management with Isilon

• Cloudera Navigator provides five categories of Data Management functions

Audit Lineage Metadata Mgmt. Encrypt Optimizer

• Isilon satisfies requirements addressed by 4 categories

Audit Lineage (NEW) Metadata Mgmt. (NEW) Encrypt ✔ Optimizer ✔

• Lineage used mainly in Financial Services and Healthcare

15 © Copyright 2018 Dell Inc.

 Cloudera Navigator and OneFS

• In HDFS workflows, Navigator monitors and tracks data as it is created and reused

• Isilon provides FSimage and iNotify integration with Navigator

• Access zone aware

16 © Copyright 2018 Dell Inc.

 File & Metadata Management in HDFS
In memory view of File System + Metadata

Blocks on disk
HADOOP

Persistent view of File System + Metadata

+

17 © Copyright 2018 Dell Inc.

 Metadata Management in HDFS

iNotify Log

18 © Copyright 2018 Dell Inc.

 Metadata Management in HDFS

Usually occurs on Secondary NameNode as this can be resource intensive

Then copied to NameNode

19 © Copyright 2018 Dell Inc.

 Navigator Polling HDFS for Metadata

Navigator

Navigator

Navigator

20 © Copyright 2018 Dell Inc.

 Metadata Management in OneFS Prior to 8.1

Isilon OneFS
Navigator
(prior to 8.1.x)

NO FSImage or iNotify Logs

21 © Copyright 2018 Dell Inc.

 Metadata Management with Isilon HDFS Now

OneFS
Navigator
8.1.1

1
1. Nightly FSImage job – tree walk,
rebuilds a new FSImage

2. Nodes combine logs to create a

unified iNotify log stream iNotify Log
NO Checkpointing in OneFS
2

22 © Copyright 2018 Dell Inc.

 Navigator Polling Isilon HDFS

Navigator OneFS

Navigator OneFS

Navigator
OneFS

23 © Copyright 2018 Dell Inc.

 Navigator Polling HDFS - skips
Navigator

Navigator

Navigator
..
…
Navigator LOG skip/error

Navigator

Navigator

24 © Copyright 2018 Dell Inc.

 Simple Lineage….

25 © Copyright 2018 Dell Inc.

 Review Table Based Lineage Data….

26 © Copyright 2018 Dell Inc.

 Custom Metadata and Tagging

27 © Copyright 2018 Dell Inc.

28 © Copyright 2018 Dell Inc.
29 © Copyright 2018 Dell Inc.
SMB3 Encryption
 SMB3 Encryption

• Adds the capability to allow data transfers to be secured by encrypting data in-flight

• No additional deployment requirements beyond configuring Isilon & using Windows

clients that support SMB3 Encryption

• SMB3 Encryption capability is found in Windows 8 / Server 2012 and newer clients

• Performance Optimizations
– Leverages AES-NI (Advanced Encryption Standard New Instructions) found in modern CPUs
– Further gains achieved leveraging the PCLMULQDQ extensions in Intel CPUs.

Dell - Internal Use - Confidential

31
 SMB3 Encryption

• SMB3 encryption can be enabled on individual shares, entire access zones or the
entire cluster

• Isilon Administrator can force the requirement of encryption

– Clients not capable of SMB3 encryption will be unable to access the system

• Performance impact of enabling encryption

– Up to ~ 25% for enabling encryption
– Variable based on clients and their capabilities

Dell - Internal Use - Confidential

32
 SMB3 Encryption
Configuration and Administration

 A simple on/off switch (“EncryptData”, False by default) in the configuration

 Can be set in:
– Share: All communication to that share will be encrypted (PCI, HR, financial data, etc.)
– Zone: All new shares created in a zone will be set to encrypted
– Global: All new shares created on the cluster will be set to encrypted

 Setting to force encryption (“RejectUnencryptedAccess”, True by default)

– When set to:
▪ False: Requires encryption from supporting clients (e.g. >= Windows 8.0) but allows access others as well (e.g. older than SMB3)
▪ True: Requires encryption from everyone – no exceptions. Clients not capable of SMB3 encryption will be denied access to each
share with this setting.

 Applies to share/zone/global depending on where encryption has been set

Dell - Internal Use - Confidential

33
 SMB3 Encryption
Configuration and Administration - Zone vs. Global settings

 Zone settings retain their autonomy

 Rules of engagement:
– No explicit zone-level settings mean that we fall back to global setting
– Any zone setting set explicitly overrides the corresponding global setting in that zone
– Changes in global settings do not overwrite existing (explicit) zone settings

 Global/Zone/Share do not exactly form a hierarchy as you can manually change the
encryption setting for an item.

Dell - Internal Use - Confidential

34
 CLI Changes
Setting share encryption:

# isi smb shares modify ifs --smb3-encryption-enabled yes

# isi smb shares view ifs | grep Encryption
Smb3 Encryption Enabled: Yes

# isi smb shares modify ifs --revert-smb3-encryption-enabled

# isi smb shares view ifs | grep Encryption
Smb3 Encryption Enabled: No

Dell - Internal Use - Confidential

35
 CLI Changes (contd.)
Setting global encryption:
# isi smb settings global modify --support-smb3-encryption yes
# isi smb settings global view | grep Encryption
Support Smb3 Encryption: Yes

# isi smb settings global modify --revert-support-smb3-encryption

# isi smb settings global view | grep Encryption
Support Smb3 Encryption: No
Setting System zone encryption:
# isi smb settings zone modify --zone=System --support-smb3-encryption yes
# isi smb settings zone view --zone=System | grep Encryption
Support Smb3 Encryption: Yes

# isi smb settings zone modify --zone=System --revert-support-smb3-encryption

# isi smb settings zone view --zone=System | grep Encryption
Support Smb3 Encryption: No

Dell - Internal Use - Confidential

36
 CLI Changes (contd.)
Allowing older clients not to encrypt (globally):
# isi smb settings global modify --reject-unencrypted-access no
# isi smb settings global view | grep Reject
Reject Unencrypted Access: No

# isi smb settings global modify --revert-reject-unencrypted-access

# isi smb settings global view | grep Reject
Reject Unencrypted Access: Yes
Allowing older clients not to encrypt (in System zone):
# isi smb settings zone modify --zone=System --reject-unencrypted-access no
# isi smb settings zone view --zone=System | grep Reject
Reject Unencrypted Access: No

# isi smb settings zone modify --zone=System --revert-reject-unencrypted-

access # isi smb settings zone view --zone=System | grep Reject
Reject Unencrypted Access: Yes

Dell - Internal Use - Confidential

37
 UI Changes
Only the global level encryption can be configured with WebUI (right now).

Dell - Internal Use - Confidential

38
 A Peek at the Wire – Unencrypted Connection

39 © Copyright 2018 Dell Inc.

 A Peek at the Wire – Unencrypted Connection

40 © Copyright 2018 Dell Inc.

 A Peek at the Wire – Encrypted Connection

41 © Copyright 2018 Dell Inc.

 A Peek at the Wire – Encrypted Connection (contd.)

42 © Copyright 2018 Dell Inc.

 A Peek at the Wire – Connect to Encrypted Share

43 © Copyright 2018 Dell Inc.

 A Peek at the Wire – Connect to Encrypted Share

44 © Copyright 2018 Dell Inc.

 Connection to Encrypted Cluster or Zone

45 © Copyright 2018 Dell Inc.

 Connection to Encrypted Cluster or Zone

46 © Copyright 2018 Dell Inc.

Google Cloud and
CloudPools
 Google Cloud and CloudPools

• Public cloud target for CloudPools – Google Cloud

– Implemented through use of generic S3 support

• Google object storage supports a continuum of archive needs which consist of

regional, multi-regional, nearline and coldline
– Standard Multi-Regional – Fastest access to data, most expensive
– Nearline – Not as fast as Standard, less expensive
– Coldline – Slower than both (but still sub-second access speed), least expensive

• Offerings provide customers opportunity to optimize on storage costs vs. transaction

costs
– Regional is more expensive to store but less cost for transaction; Coldline is cheap to store but
transaction costs are higher

48 © Copyright 2018 Dell Inc.

 Google Cloud and CloudPools

• New cloud provider in CloudPools webui drop down

• Storage Tier for the two CloudPools buckets must be set in the Google Cloud Administration Dashboard
(not part of Isilon’s UI)
• Storage Tier must be configured before customers archive to the cloud. Once data is in the cloud it
cannot change tiers

• Performance characteristics are similar to other cloud providers and are based on tier chosen
and internet connection speeds

• Isilon does not support the native Google Cloud Authentication method (OAuth2)

• Isilon requires the Google Cloud Storage account have ‘Interoperability Mode’ enabled and S3
key/secret to be generated
• This is implemented through the Google Cloud Administration Dashboard

49 © Copyright 2018 Dell Inc.

 Enable Interoperability Mode
From the Google Cloud dashboard select ‘Storage’

50 © Copyright 2018 Dell Inc.

 Select
Settings

51 © Copyright 2018 Dell Inc.

 Select
Interoperability

52 © Copyright 2018 Dell Inc.

 Enable Interoperability
mode and create a
key

53 © Copyright 2018 Dell Inc.

 Create Cloud Account
• Create a cloud account with cloud provider EMC ECS Appliance
• Use the previously created key and secret

54 © Copyright 2018 Dell Inc.

 Add Google Certificates (v8.0.1.0, 8.0.1.1)
If running OneFS prior to 8.0.1 MR 2, must add the Google SSL certificate to the cluster

1) Copy the root certificate of target service provider <cert.pem> into directory
- % cp <cert.pem> /ifs/.ifsvar/modules/cloud/cacert

2) Get the hash for each cert

- openssl x509 -hash -noout -in <cert.pem>

3) Create a symlink to the <cert.pem> using the output <hash-val>

- ln -s /ifs/.ifsvar/modules/cloud/cacert/<cert.pem> /ifs/.ifsvar/modules/cloud/cacert/<hash-val>.<suffix>

<suffix> starts as 0. If there is a collision of existing symlink file name, then use the next number as suffix.

55 © Copyright 2018 Dell Inc.

 To Get Certificates (1)
• Google “Google Internet Authority G2”
– https://pki.google.com/
Note: Google certificate expires Dec 31 2017, will need to download a new one

56 © Copyright 2018 Dell Inc.

 To Get Certificates (2)
• Google “geotrust global ca”
– https://www.geotrust.com/resources/root-certificates/

57 © Copyright 2018 Dell Inc.

 See also

• https://support.emc.com/kb/ 497931

58 © Copyright 2018 Dell Inc.

 CloudPools Targets

59 © Copyright 2018 Dell Inc.

Patch Signing
 Patch Signing

• Currently there is no way for customers to validate the authenticity of upgrade images or
patches

• Patch signing provides a means to verify authenticity of OneFS patches

• Patch signing provides improved customer supportability

• Dell EMC side: Package created containing patch files and manifest, detached signature file
provided for validation
– Transparent to customer

• Customer’s side: Signature verified, patch extracted and installed

61 © Copyright 2018 Dell Inc.

 Patch Installation (Customer’s side)

62 © Copyright 2018 Dell Inc.

 UI and CLI Changes

• No WebUI support in 8.1.1

• CLI changes:
• isi_signatures
•
Usage: isi_signatures [ -s <signature file> -p <package file> ]
-s <signature file>: file containing a digital signature and public key for a package
-p <package>: signed package
• Expected Output:
• Success: the signature is valid for this package
• Error: Failure: the signature is not valid for this package

63 © Copyright 2018 Dell Inc.

Health Check Framework
 Health Check Framework

• Flexible framework available to customers and Support personnel to pro-actively monitor cluster
and environment to identify problems before they become issues
• Unified way of creating, running and maintaining the scripts

• Improves RAS (Reliability, Availability, and Serviceability)

• Framework is checklist based

– Multiple checklist supported (eg ALL, self_test, smartconnect)
– Reusable checklist items

• Standardization in parameters and outputs

• Can control the execution of scripts to manage cluster load

• Not integrated with CELOG (and intended to supplement CELOG)

65 © Copyright 2018 Dell Inc.

 Health Check Framework

• No required configuration (some are optional)

• Useful to add some sort of cron job to periodically run checklists

• isi healthcheck command syntax (no WebUI):

66 © Copyright 2018 Dell Inc.

 Troubleshooting

• How to troubleshoot new features

– The Health check framework logs to /var/log/messages in the event of problems running checks or error
output from checks

• Health check data is available for analysis in log gathers from /ifs/modules/health-check/results

67 © Copyright 2018 Dell Inc.

Multi-language Support
NFSv3 encoding
 NFSv3 Encoding Support

• Prior to OneFS 7.2, encoding was not enforced for any NFSv3 exports

• With OneFS 7.2 and later, clients were forced to use the encoding specified in the OneFS exports

• With OneFS 8.1.1, enables exports to the NFS clients that uses multiple character encodings
– Restores Pre-OneFS 7.2 encoding behavior

• Changes made are compatible with CloudPools, audit etc

• Upgrades supported from OneFS 7.1.1.11 to OneFS 8.1.1

– NFSv3 character encoding should just work (ie. requires no cluster configuration).

69 © Copyright 2018 Dell Inc.