ADM 201 – Configure a MapR Cluster

L6-1
Learning Goals

6.1 Describe Volumes and Volume Properties

6.2 Configure Volumes

L6-2
Review: Storage Architecture

Storage pool Storage pool

Containers, striped
across storage pools

https://youtu.be/TOFQq9PHJHk L6-3
Review: Storage Architecture
Files are sharded into chunks and written to containers

L6-4
Review: Storage Architecture
Volumes are collections of containers

L6-5
What Is a MapR Volume?

• Logical unit of data organization and

management

• Comprised of containers
– Name containers
– Data containers

• May span the cluster

L6-6
Volume Properties: Name and Mount Path

/work/projects

projects /mapr/<cluster name>/work/projects

L6-7
Volume Properties: Permissions

Root directory
permissions

Volume
permissions

L6-8
Volume Properties: Permissions
Volume permissions can be granted to users or groups

Permission Allows Action(s)

dump Dump or back up the volume

restore Restore or mirror the volume

m Edit volume properties

d Delete the volume

fc Full control except set/edit ACLs

a Set and edit ACLs

L6-9
Volume Properties: Topology

/data /data

/data/r1 /data/r2 /data/r3 /data/r4 /data/r1 /data/r2 /data/r3 /data/r4

Volume topology set to /data Volume topology set to /data/r1

L6-10
Volume Properties: Quotas

Hard quota
No more data written to volume

Advisory quota
Alarm is generated

my_volume

L6-11
Volume Properties: Quotas

Only original, post-compression

data counts against a quota

• 10 GB source file: 10 GB

• Compressed to 8 GB: 8 GB

• Replicated 3 times: 24 GB

my_volume 8GB counts

against quota

L6-12
Volume Properties: Accountable Entity (AE)
• Entity (user or group) accountable for a volume’s usage

• Only one AE per volume

• AEs may also have quotas

• Every volume assigned to an AE

counts against the AE’s quota Vol-1
Vol-1
Vol-1
Vol-1

L6-13
Accountable Entities and Quotas

Use to manage An accountable Multiple volumes Volumes and

space consumed entity can be a can bill to the accountable
by different user or a group same accountable entities can both
projects, entity have disk use
departments, etc. quotas

L6-14
Volumes and Accountable Entities

Accountable Entities
Users
root

Cobra

users development Rattler

production
marje cobra-dev

cobra-prod rattler-prod

marje-rattler marje-cobra marje-vol

L6-15
Volumes and Accountable Entities

The volume marje is Accountable Entities

accountable to Users
Users
root

Cobra
Volumes under
marje can have users development Rattler
different accountable
entities

production
marje cobra-dev

cobra-prod rattler-prod

marje-rattler marje-cobra marje-vol

L6-16
Volumes and Accountable Entities

Accountable Entities
cobra-related volumes
are all accountable to Users
root
the group Cobra
Cobra

users development Rattler

production
marje cobra-dev

cobra-prod rattler-prod

marje-rattler marje-cobra marje-vol

L6-17
Volumes, Accountable Entities, and Quotas

Hard Quotas
root
AE: Cobra 40 TB 40 TB
Vol: cobra-dev 30 TB

Vol: cobra-prod 30 TB 62 TB
users development Vol: marje-cobra 2 TB

production
marje cobra-dev

cobra-prod rattler-prod

marje-rattler marje-cobra marje-vol

L6-18
Volume Property: Replication Factor

• Replication factor is set at the volume

Master level (default = 3)
container

• Applies to all containers in the volume

• Separate replication factors for name

containers and data containers

Tail container Intermediate

container(s)
L6-19
Volume Property: Replication Factor

• Replication factor
Master – Desired number of copies
container

• Minimum replication factor

– If actual replication count falls below
this, an alarm is raised and data is
re-replicated immediately

Tail container Intermediate

container(s)
L6-20
Volume Property: Replication Factor

• Disk failure triggers immediate

replication

• Node failure/offline triggers replication

after a configurable window
– default: 1 hour
– set with maprcli config save

L6-21
Volume Property: Replication Type

• Once set, cannot change

Master
container
• High throughput (chain)
– Default
– Appropriate for most volumes

Tail container Intermediate

container(s)
L6-22
Volume Property: Replication Type

• Once set, cannot change

Master
container
• High throughput (chain)
– Default
– Appropriate for most volumes

• Low latency (star)

– Best with small files
– Can impact network with large files

Tail container Intermediate

container(s)
L6-23
Learning Goals

6.1 Describe Volumes and Volume Properties

6.2 Configure Volumes

L6-24
Typical Volume Layout

/mapr/<cluster name>/

projects users opt var

build test markm shellyb mapr

L6-25
Volumes Best Practices

• Create multiple volumes

– Efficiently use snapshots and mirrors
– Separate and manage resources
vol-b – Establish ownership and accountability
vol-b
vol-a – Spread the load of access requests

vol-b • Create volumes based on business

requirements
vol-c – For users, departments, projects, etc.

L6-26
Create Volumes With MCS

L6-27
Create Volumes With MCS: Volume Setup

Volume name
must be unique

L6-28
Create Volumes With MCS: Volume Setup

Parent directories must

exist; mount point must
not exist

L6-29
Create Volumes With MCS: Volume Setup

Assign topology

L6-30
Create Volumes With MCS: Volume Setup

If desired, make the

volume read-only

L6-31
Create Volumes With MCS: Volume Setup

Assign accountable
entity (mapr by default)

L6-32
Create Volumes With MCS: Permissions

Root directory
permissions in
octal format

Volume permissions

L6-33
Create Volumes With MCS: Usage Tracking (Quotas)
Format: <num><units>

<units> is:

M (Megabytes)
G (Gigabytes)
T (Terabytes)

Advisory and
hard quotas

L6-34
Create Volumes With MCS: Replication

Replication factor for

data containers

Replication factor for

name containers

Replication pattern

L6-35
Create Volumes With MCS: Scheduling
Snapshots and scheduling are covered in a later module

L6-36
Create Volumes With MCS: Data Access
Create Access Control Expressions to define access to volume data

L6-37
Create Volumes With CLI
• Create a volume and mount it with:

$ maprcli volume create –name <name> -mount 1 –path <path>

• Type the command with no options to see syntax:

<path>
$ maprcli volume create

<name>

L6-38
Volume Create Options With CLI
-advisoryquota <quota> Sets advisory quota
-quota <quota> Sets hard quota
-ae <entity> Sets accountable entity
-aetype <type> 0=user; 1=group
-auditenabled Enables volume auditing if specified (no arguments)
-readonly <value> 0=read/write; 1=read-only
-replication <num> Replication factor of <num> for data containers

-minreplication <num> Minimum replication factor of <num> for data containers

-nsreplication <num> Replication factor of <num> for name containers

-nsminreplication <num> Minimum replication factor of <num> for name containers

-topology <path> Specifies volume topology

For a full list: http://maprdocs.mapr.com/home/ReferenceGuide/volume-create.html

L6-39
Set Permissions With CLI

$ maprcli volume create –name cluster –u joe:fc jenn:m –g adm:restore

-g <group name>:<perms> Provide a space-separated list of group:permission pairs

-u <user name>:<perms> Provide a space-separated list of user:permission pairs

Permission codes: dump Dump or back up the volume

restore Restore or mirror the volume
m Edit volume properties
d Delete the volume
a View and edit ACLs
fc Full control except view/edit ACLs

L6-40
Modify a Volume
• Through MCS:

Click volume name

• Through CLI:

$ maprcli volume modify –name <name> [options]

L6-41
Lab : Create Volumes and Set Quotas

1_Lab Seperator

L6-42
 Next Steps
ADM 201 – Configure a MapR Cluster

L6-43