Professional Documents
Culture Documents
HOL-0543-01 - PowerScale DataIQ ECS Field Enablement
HOL-0543-01 - PowerScale DataIQ ECS Field Enablement
Table of Contents
Introduction .................................................................................................................................................. 4
PowerScale Field Enablement ......................................................................................................... 5
Lesson 2 - Cloudera Manager with CDH and OneFS Integration and Installation .....................36
Lesson 3 - Test and validate CDH operations and services ........................................................37
Lesson 4 - Kerberize Cloudera CDH - OneFS Integration with MIT Kerberos Or Kerberize
Cloudera CDH - OneFS Integration with Active Directory ............................................................38
Lesson 5 - Test and validate Kerberized CDH operations and services .....................................39
Optional Tasks................................................................................................................................40
Introduction
This lab provides the basic infrastructure to test and validate the procedures and configuration
for deployment and operation of different clients and applications against a PowerScale OneFS
cluster. The goal is to provide a baseline set of servers and services to allow the student to learn
and develop the skills necessary to be successful in the deployment, operation or support of
PowerScale OneFS integrations.
A few potential use cases and lab scenarios are presented in this guide, including Hadoop
cluster integrations, NFSv4, and Kerberos deployments. But many others not outlined here exist
and can be implemented.
This lab does not provide any step-by-step instructions for the student on how to accomplish
these tasks, the primary purpose is to provide a lab space where the student has the required
infrastructure and they can self-direct themselves and learn how to accomplish these tasks.
• Many PowerScale OneFS implementation scenarios can be configured and tested with this
lab
• Existing PowerScale labs, documentation and how-to guides can be used on this lab to
implement use cases
• Software packages should be downloaded directly for application testing and integration
• No support for any lab exercises exist
The primary purpose of this lab is to provide the core infrastructure components for the student
to self-direct learn how to integrate PowerScale OneFS with many complex scenarios not
addressed in the base labs.
Note: This installation of 9.0 has had all core available OneFS protocols enabled by default, this
is different from a fresh install on OneFS 9.0 where all protocols are now disabled by default.
The exception to this on this cluster is HDFS, since HDFS is licensed feature, the license must
be added and then the HDFS service enabled with #isi services enable hdfs and review the
service status with #isi services -al
Lab Overview
Lab Overview
In this lab, you will complete the following:
Architecture
The environment for this lab includes the following:
Note: This lab guide covers the basic setup used to install and test applications with PowerScale
storage.
Network Diagram
Below shows the network diagram for this environment.
Lab Credentials
Additional IP Pool range
These IP ranges can be allocated to additional SmartConnect Pools on the PowerScale cluster to
create facilitate the use of Access Zones and multi-tenancy as needed
• 192.168.1.XXX
• 128.221.252.XXX
Labs Scenario
This lab is built to support the deployment and integration of many different applications within the
environment as needed.
The recommended approach is to execute an application install to completion then if needed destroy
the lab and redeploy a new lab to test additional application if they do integrate with the existing
configuration.
The focus of this lab is to provide a fenced-in environment that allows you to configure the
PowerScale cluster for NFS, Kerberos or Hadoop distros installation then test and validate the
operation and services.
As stated previously this lab does not include any specific tasks to complete, rather it is an open
environment where you can undertake any installation tasks or integration you require to gain the
knowledge and understanding of how to deploy PowerScale and Hadoop. By providing this space
the student can leverage any materials or guides they see as beneficial in learning how to deploy
these services and tools. If you run into problems or issues, you can always just destroy and
redeploy the lab.
Some sample installation and configuration "How To's" can be found at https://github.com/brittup/
how_to these simple docs walk through some scenarios that can be used and tested in this field
enablement lab.
Access to Software
Since the basis of this lab is to provide a sandbox in which many different platforms and services
can be deployed. No specific software, packages or application are currently downloaded or
deployed.
If newer or additional software packages are needed you can just download them as needed;
Cloudera Manager CDH or Hortornworks Ambari HDP are great examples but many other software
packages could be installed and integrated.
Our goal by providing this flexibility is to provide the ability to install any version of these platforms
as they are updated with newer releases.
https://inside.dell.com/docs/DOC-231359
Reference Material
Using Hadoop with OneFS Info Hub
https://www.dell.com/support/article/en-us/sln318813/using-hadoop-with-onefs-info-hub
Hortonworks
http://hortonworks.com/
Cloudera
http://www.cloudera.com/
https://inside.dell.com/docs/DOC-231359
https://github.com/Isilon/isilon_hadoop_tools
This plug in will manage S3 dynamic access key ID and secret for accessing OneFS S3 buckets. This
is posted here if you wish to use this sandbox and explore this item within this setup.
NOTE:
On the launchpad, Data Drive E: you will find the following for your use.
• DataIQ and Plug-ins, these are also copied into the root directory on the DataIQ VM.
• PowerScale Custom Service Descriptor 1.0.0
• VDBench
• Wireshark
• CloudBerry and S3 Browsers
These is more on this data drive including an assortment of demo data for your use.
Module 1 - Multiprotocol
Integration and Validation
To setup, test and validate multiprotocol access with OneFS, Windows and Linux some
suggested exercises and configuration are:
Many other multiprotocol configurations and integration can be set up and validate in this lab.
Use the provided infrastructure to set up any configurations you wish to test.
Module Conclusion
Congratulations on completing Module 1.
In this module, you explored PowerScales multiprotocol implementation and operation, the setup of
the lab allowed you to create and validate many different components of OneFS multiprotocol to
understand OneFS behaviors. At any time additional scenarios can be explored in this lab using the
provided infrastructure.
Module 2 - Kerberos
Integration
To setup, test and validate Kerberos with OneFS, Windows and Linux some suggested exercises
and configuration are:
https://github.com/brittup/how_to/tree/master/krb5
Many other configurations and integration can be set up and validated in this lab. Use the
provided infrastructure to set up any configurations you wish to test.
Module Conclusion
Congratulations on completing Module 2.
In this module, you explored OneFS Kerberos implementation and operation, the setup of the lab
allowed you to create and validate many different components of OneFS to understand PowerScale
behaviors. At any time additional scenarios can be explored in this lab using the provided
infrastructure.
To setup, test and validate NFSv4 with OneFS, Windows and Linux some suggested exercises
and configuration are:
https://github.com/brittup/how_to/tree/master/nfs
Many other configurations and integration can be set up and validated in this lab. Use the
provided infrastructure to set up any configurations you wish to test.
Prepare and configure OneFS for Kerberized NFS integration and setup.
To setup, test and validate Kerberos with OneFS and NFS, suggested exercises and configuration
are:
https://github.com/brittup/how_to/tree/master/nfs
https://github.com/brittup/how_to/tree/master/krb5
Many other configurations and integration can be set up and validated in this lab. Use the
provided infrastructure to set up any configurations you wish to test.
Module Conclusion
Congratulations on completing Module 3.
In this module, you explored PowerScale NFSv4 and Kerberized NFS implementations and operation,
the setup of the lab allowed you to create and validate many different components of OneFS to
understand PowerScale behaviors. At any time additional scenarios can be explored in this lab using
the provided infrastructure.
This lab is intended to help you understand the deployment methodology for deploying
Hortonworks Ambari HDP against an PowerScale cluster.
The modules listed below highlights the overall deployment strategy that need to be
accomplished to successfully deploy and operate HDP. The bulleted task provide guidance
around the high level configuration tasks that need to be accomplished.
https://github.com/brittup/how_to/tree/master/hadoop
Prepare and configure OneFS for Hadoop integration; using the PowerScale implementation
guides for Hadoop setup the cluster for integration.
https://www.dell.com/support/article/en-us/sln318813/using-hadoop-with-onefs-info-hub
Begin by preparing OneFS for integration with Ambari by completing the following on OneFS
*It is suggested to use the OneFS Hadoop Tools (always get the latest) https://github.com/
Isilon/isilon_hadoop_tools
https://www.dell.com/support/article/en-us/sln318947/hadoop-distributions-and-products-
supported-by-onefs?lang=en
At the time of writing this doc; HDP 3.1.5 is the suggested install but any supported version can be
used.
https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/index.html
https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/installation.html
Having deployed and integrated HDP with OneFS we can test and validate Hadoop jobs and
services.
Having deployed an operational HDP Hadoop cluster against OneFS we can then Kerberize the
cluster to provide secure Hadoop services with PowerScale. We have the option of deploying
Kerberos based on an MIT KDC or an Active Directory. Follow the documentation depending on
which Kerberos provider you intend to deploy and integrate.
On completion of Kerberizing the Hadoop cluster we can now test and review the operations and
behavior of Kerberized jobs and access against an OneFS cluster.
Optional Tasks
Module Conclusion
Congratulations on completing Module 4.
In this module, you explored Hortonworks HDP and OneFS implementations and operation, the setup
of the lab allowed you to create and validate many different components of OneFS to understand
PowerScale behaviors. At any time additional scenarios can be explored in this lab using the
provided infrastructure.
Module 5 - Cloudera
Manager and CDH
deployment and integration
This lab is intended to help you understand the deployment methodology for deploying
Cloudera’s CDH stack against a PowerScale cluster.
The modules listed below highlights the overall deployment strategy that needs to be
accomplished to successfully deploy and operate Cloudera Manager and CDH. The bulleted
tasks provide guidance around the high-level configuration tasks that need to be accomplished.
https://github.com/brittup/how_to/tree/master/hadoop
Prepare and configure OneFS for Hadoop integration; using the PowerScale implementation
guides for Hadoop setup the cluster for integration.
https://www.dell.com/support/article/en-us/sln318813/using-hadoop-with-onefs-info-
hub?lang=en
Begin by preparing OneFS for integration with Cloudera CDH by completing the following on
OneFS
*It is suggested to use the OneFS Hadoop Tools (always get the latest) https://github.com/
Isilon/isilon_hadoop_tools
Setup and install Cloudera Manager and then deploy the CDH stack with OneFS integration.
Having deployed and integrated CDH with OneFS we can test and validate Hadoop jobs and
services.
Having deployed an operational Cloudera Hadoop cluster against OneFS we can then Kerberize
the cluster to provide secure Hadoop services with PowerScale. We have the option of deploying
Kerberos based on an MIT KDC or an Active Directory. Follow the documentation depending
which Kerberos provider you intend to deploy and integrate.
On completion of Kerberizing the Hadoop cluster we can now test and review the operations and
behavior of Kerberized jobs and access against a PowerScale cluster.
Optional Tasks
• Upgrade Cloudera Manager & CDH version (install a lower version and upgrade)
• Upgrade PowerScale OneFS
• Break and troubleshoot components
• Implement Cloudera Navigator
Module 6 - DataIQ
Integration
DataIQ Integration
This lab is intended to help you understand the deployment methodology for deploying and
integrating DataIQ with a PowerScale cluster.
Lesson 1 - DataIQ
DataIQ and OneFS integration and setup. There are two approaches here, first being the use of
the DataIQ documentation to read and follow in connecting the DataIQ 2.0 server to the
PowerScale clusters. The second approach is to go to democenter and follow the lab guide
there on getting started with DataIQ.
Search for the following lab: HOL-0537-03-DataIQ_Getting_Started and you can then download
the lab guide.
Module Conclusion
Congratulations on completing Module 6.
In this module, you explored DataIQ and PowerScales implementation and operation, the setup of the
lab allowed you to create and validate many different components of the integration of DataIQ with
OneFS behaviors. At any time additional scenarios can be explored in this lab using the provided
infrastructure.
ECS Integration
This lab is intended to help you understand the deployment methodology for deploying and
integrating ECS with PowerScale cluster and DataIQ.
Lesson 1 - ECS
ECS setup and integration with DataIQ and OneFS can be achieved via two approaches. First being
the use of the ECS documentation to read and follow in connecting to DataIQ 2.0 server and the
PowerScale clusters. The second approach is to go to democenter and follow the lab guide there on
getting started with ECS or any of the other ECS labs. The documents listed below are just some of
what can be gotten off our web sites.
Search for the labs by using ECS in the search field and you can then download the lab guide
Module Conclusion
Congratulations on completing Module 7.
In this module, you explored ECS with DataIQ / PowerScale implementation and operation, the setup
of the lab allowed you to create and validate many different components of the integration of ECS
with DataIQ and OneFS behaviors. At any time additional scenarios can be explored in this lab using
the provided infrastructure.