Effective Hadoop

TM
Cluster
Management
8ecorded verslon avallable aL
http://www.impetus.com/webinar_registration?event=archived&eid=51
utline
adoop
TM
- Quick overview
adoop
TM
clusters
Addressing the adoop
TM
cluster challenges
Operational
Cluster Setup
Cluster Management
Cluster Sharing
Quick provisioning of Cloud-based adoop
TM
clusters
Available solutions for cluster setup and management
2
8ecorded verslon avallable aL
http://www.impetus.com/webinar_registration?event=archived&eid=51
adoop
Nodes, roles, services
Configurations & modes
adoop sub-components
Feature /
CIuster Mode
Production DeveIopment & Testing usage
Distributed Pseudo Distributed StandaIone
How Services Run TruIy Distributed SingIe Machine - Separate Java processes SingIe Java Process
HDFS Yes Yes No (Uses IocaI FiIe System)
RepIication Yes No No
XML Conf Init required Yes Yes No
Services Start/Stop req. Yes Yes No
Node addition Yes No No
Hadoop
TM
verview
3
Hadoop
TM
Clusters- The Pain Areas
Operations
Cluster setup
Cluster management
adoop
TM
compatibility
Cluster sharing
4
8ecorded verslon avallable aL
http://www.impetus.com/webinar_registration?event=archived&eid=51
Hadoop
TM
Clusters- What is Missing?
Automation
Setup support
User friendly interface
Cluster management capabilities
Centralized management for multiple clusters
Monitoring & notifications support
Provision for cluster sharing
5
8ecorded verslon avallable aL
http://www.impetus.com/webinar_registration?event=archived&eid=51
Hadoop
TM
Clusters - Key Challenges
perations
Challenges
Manual operations
Console based interface
//7essing Challenges
Automated activities
User friendly GU as an alternate interface

8ecorded verslon avallable aL

http://www.impetus.com/webinar_registration?event=archived&eid=51
Hadoop
TM
Clusters - Key Challenges

CIuster Setup
Challenges
Manual execution
nteractive setup
Setup procedure varies as per the situation
Set-up environment - Cloud & local clusters
//7essing the challenges
Context aware, non-interactive automation
adoop
TM
components initialization
Click based cluster provisioning
8ecorded verslon avallable aL
http://www.impetus.com/webinar_registration?event=archived&eid=51
"uick Provisioning of Cloud Clusters
ction CIoud CIusters LocaI CIusters
Instances Launch #equired --- N.A. ---
Required additionaI Info
Machine image, nstance type,
Security groups, Key
CIuster machine types Homogeneous/ eterogeneous omogeneous /eterogeneous
S Normally same Could be different
Java Same on all Could be different
Password-Iess SSH Setup Already Exists Setup #equired
Hadoop Components Setup of components nearly remains same in both case
Instances Termination #equired, if requested --- N.A. ---

Understanding the difference

8ecorded verslon avallable aL
http://www.impetus.com/webinar_registration?event=archived&eid=51
Challenges
Manual operation
Efforts increases with cluster size
nterface switching
//7essing the Challenges
Automated provisioning
Scripts
Cloud APs
"uick Provisioning of Cloud Clusters

8ecorded verslon avallable aL

http://www.impetus.com/webinar_registration?event=archived&eid=51
Hadoop
TM
Clusters - Key Challenges

CIuster Management
Challenges
ack of appropriate management support
//7essing the challenges
Support for managing
Configuration
Nodes & services
Jobs
Keys & users
Monitoring & notifications
8ecorded verslon avallable aL
http://www.impetus.com/webinar_registration?event=archived&eid=51
Hadoop
TM
Clusters - Key Challenges

Hadoop
TM
CompatibiIity & Issues
Challenges
'ersion compatibility
Cluster modes
'ersion, vendor, bundle format differences
'endor specific features
//7essing the challenges
orkaround solution for compatibility issues
Automation for version, vendor, bundle format
Context aware management support
'endor specific tools
8ecorded verslon avallable aL
http://www.impetus.com/webinar_registration?event=archived&eid=51
Hadoop
TM
Clusters - Key Challenges
2
CIuster Sharing
Challenges
ays of sharing
No sharing / Sharing with limited security
//7essing these challenges
Sharing through alternate interface
Purpose specific role based sharing
8ecorded verslon avallable aL
http://www.impetus.com/webinar_registration?event=archived&eid=51
How HCM Tools can Help?
Quick turnaround
elps to minimize time, cost & efforts
ntegrated support (setup, management & monitoring)
Easy execution
Proven usefulness in production & testing
Adhering to best practices
3
8ecorded verslon avallable aL
http://www.impetus.com/webinar_registration?event=archived&eid=51
Setup
Automation for
setup of Cluster
Components local
& cloud
environments
Management
Nodes, Services,
Configuration, Keys,
Users, Jobs, Job
Templates,
Performance
Tracking
Upgrade, Auditing,
Monitoring,
mport/Export,
Event Mail/Alerts/
Notifications, Active
working
Sharing
Alternate access
interface
#ole based access
Security preserved
4
How HCM Tools can Help?
8ecorded verslon avallable aL
http://www.impetus.com/webinar_registration?event=archived&eid=51
$ome Available ptions
Amazon Elastic Map-#educe
hirr
Cloudera SCM
mpetus Ankush
5
8ecorded verslon avallable aL
http://www.impetus.com/webinar_registration?event=archived&eid=51

Slide 1S
CAN3 will add logos
arima rora aheshwari, 11/28/2011
Ankush - Impetus HCM Tool

8ecorded verslon avallable aL

http://www.impetus.com/webinar_registration?event=archived&eid=51
Ankush: Key Features
Centralized multiple cluster management
Common interface for cloud and local clusters
ntegrated support for setup, management & cloud provisioning
Anytime-anywhere access to cluster functionalities
Cloud independent
#educes cluster setup time by 60 - 80%
Bundle setup optimization

8ecorded verslon avallable aL

http://www.impetus.com/webinar_registration?event=archived&eid=51
$ummary
Automation facilitates smooth adoop
TM
working
Eases cloud cluster provisioning
Alternate U enables secure cluster sharing
adoop
TM
still in evolving stage
Cluster management tools for pertinent problem areas

8ecorded verslon avallable aL

http://www.impetus.com/webinar_registration?event=archived&eid=51
About Us
Strategic partners for software product engineering and #&D
Thought leaders in cutting-edge technologies
Mature processes and practices that are methodical, yet flexible
Diverse domain expertise
mpetus offers expert consulting, implementation, and support
services in Big Data and Analytics
8ecorded verslon avallable aL
http://www.impetus.com/webinar_registration?event=archived&eid=51
"uestions
PIease send in your questions
using the chat paneI
8ecorded verslon avallable aL
http://www.impetus.com/webinar_registration?event=archived&eid=51
Thank you
or more information,
write to us at inquiry@impetus.com
8ecorded verslon avallable aL
http://www.impetus.com/webinar_registration?event=archived&eid=51
impetuscaIIing