3

Control Groups (cgroups)

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Objectives

After completing this lesson, you should be able to:

• Describe the purpose of control groups
• Describe control group subsystems and parameters
• Describe the control group hierarchy model
• Configure control groups using the
/etc/cgconfig.conf file
• Use the lssubsys utility
• Use the cgcreate and cgdelete utilities
• Use the cgset utility
• Assign tasks to control groups
• Use the cgrules.conf file
• Enable PAM to use control group rules
• Obtain information about control groups and parameters
3-2 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
 Control Groups: Introduction

Control groups (cgroups) enable you to allocate computing

resources to specific processes or tasks. You can:
• Assign a set of CPU cores and memory nodes to a specific
group of tasks
• Specify the relative share of CPU time available to the
tasks in a cgroup
• Specify memory limits for the tasks in a cgroup
• Suspend the tasks in a cgroup
• Report the CPU time and memory used by the tasks in a
cgroup
• Assign different priorities to network packets from different
cgroups
• Deny or allow tasks in a cgroup access to specific devices

3-3 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
 Cgroup Subsystems (Resource Controllers)
A subsystem is a kernel resource controller that applies limits
or acts on a group of processes. Subsystems include:
• cpuset – Assigns individual CPUs and memory nodes to
cgroup tasks
• cpu – Schedules CPU access to cgroup tasks
• cpuacct – Reports the total CPU time used by cgroup
tasks
• memory – Reports or limits memory use of cgroup tasks
• devices – Grants or denies access to devices
• freezer – Suspends or resumes cgroup tasks
• net_cls – Tags outgoing network packets with an
identifier, which can then be assigned a different priority
• blkio – Reports or controls I/O bandwidth for block
devices
3-5 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
 Cgroup Subsystem Parameters

• Each subsystem has specific parameters that enable

resource control and reporting mechanisms.
• Subsystem parameters allow you to set limits, restrict
access, or define allocations for each subsystem.
• Understanding the specific parameters helps you
understand the possibilities for controlling resources with
cgroups.
• Each subsystem cgroup also has a set of common files,
including the tasks file:
– The tasks file keeps track of processes associated with the
cgroup and the associated subsystem parameter settings.
– The tasks file contains all the process IDs (PIDs) assigned
to the cgroup.

3-6 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
 Cgroup Hierarchy

Eight separate hierarchies Eight associated root cgroups

/cgroup/cpuset cpuset

/cgroup/cpu cpu

/cgroup/cpuacct cpuacct
Each root
/cgroup/memory memory cgroup can
have child
/cgroup/devices devices cgroups.
/cgroup/freezer freezer

/cgroup/net_cls net_cls

/cgroup/blkio blkio

3-7 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
 Cgroup Configuration Rules and Constraints
• A hierarchy can have multiple subsystems attached.
– For example, the /cgroup/cpu-mem hierarchy can have
both cpu and memory subsystems attached to it.
– Both cpu and memory subsystem parameters reside in the
single /cgroup/cpu-mem hierarchy.
• A subsystem cannot be attached to a second hierarchy if
the second hierarchy has a different subsystem attached.
– The cpu subsystem is attached to the /cgroup/cpu
hierarchy.
– The memory subsystem is attached to the
/cgroup/memory hierarchy.
– Attaching the cpu subsystem to the /cgroup/memory
hierarchy fails.
• A task cannot be a member of two different cgroups in the
same hierarchy.

3-8 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
 Cgroup Configuration

• Install the libcgroup software package:

# yum install libcgroup
• Enable the cgconfig service to start at boot time:
# chkconfig cgconfig on
• The main configuration file for cgroups is
/etc/cgconfig.conf. It is used to define control
groups, their parameters, and mount points.
• The /etc/cgconfig.conf file contains two types of
definitions.
– mount: Defines the virtual file systems to mount subsystems
– group: Defines a cgroup, access permissions, subsystems
to use, and parameter values for those subsystems
• Start the cgconfig service to read the configuration file.

3-9 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
 /etc/cgconfig.conf “mount” Section

mount {
cpuset = /cgroup/cpuset;
cpu = /cgroup/cpu;
cpuacct = /cgroup/cpuacct;
memory = /cgroup/memory;
devices = /cgroup/devices;
freezer = /cgroup/freezer;
net_cls = /cgroup/net_cls;
blkio = /cgroup/blkio;
}

3 - 10 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 lssubsys Utility

Use the lssubsys command to list the hierarchies.

# lssubsys –am
cpuset /cgroup/cpuset
cpu /cgroup/cpu
cpuacct /cgroup/cpuacct
memory /cgroup/memory
devices /cgroup/devices
freezer /cgroup/freezer
net_cls /cgroup/net_cls
blkio /cgroup/blkio

3 - 12 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Sample “mount” Implementation

• Edit /etc/cgconfig.conf to create the cpu_ram

hierarchy and attach the cpu, cpuset, and memory
subsystems.
mount {
cpuset = /cgroup/cpu_ram;
cpu = /cgroup/cpu_ram;
memory = /cgroup/cpu_ram;
}

• Alternative command-line method (not persistent):

# mkdir /cgroup/cpu_ram
# mount –t cgroup –o cpu,cpuset,memory cpu_ram
/cgroup/cpu_ram

3 - 13 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 /etc/cgconfig.conf “group” Section

group <cgroup_name> {
[perm {
task {
uid = <task_user>;
gid = <task_group>;
}
admin {
uid = <admin_user>;
gid = <admin_group>;
}
}
<subsystem> {
<subsystem.parameter> = <value>;
}
}

3 - 15 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 View of the New Cgroup Hierarchy

cpuset cpu memory

/cgroup/cpu_ram/

dbgrp

Cgroup Hierarchy

3 - 17 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 cgcreate and cgdelete Utilities

• Alternatively, you can use the cgcreate command to

create a cgroup from the command line. The syntax is:
# cgcreate [-t uid:gid] [-a uid:gid] -g subsystems:path
• For example, to create a new cgroup named group2 in
the cpu subsystem hierarchy:
# cgcreate –g cpu:group2
• To create a single cgroup named cpu_mem in two different
subsystem hierarchies (cpu and memory):
# cgcreate –g cpu,memory:cpu_mem
• Use the cgdelete command to remove a cgroup:
# cgdelete cpu:group2
# cgdelete cpu,memory:cpu_mem

3 - 18 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 cgset Utility

• Alternatively, use the cgset command to set the para-

meters of cgroups from the command line. The syntax is:
cgset [-r <name=value>] <cgroup-path>
cgset --copy-from <source_cgroup-path>] <cgroup-path>

• Example: Suppose that the cpuset subsystem is attached

to the group1-web and group2-db cgroups. To allocate
CPU and memory nodes for the cgroups:

# cgset -r cpuset.cpus=‘0,2,4,6,8,10,12,14’ group1-web

# cgset -r cpuset.cpus='1,3,5,7,9,11,13,15' group2-db
# cgset -r cpuset.mems=‘0’ group1-web
# cgset -r cpuset.mems='1' group2-db

3 - 19 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Assigning Processes to a Cgroup

• Use the cgclassify command to move existing

processes to a cgroup. For example, to move PID 1683
into the group1-web cgroup:
# cgclassify -g cpuset:group1-web 1683
• Alternatively, you can also use the echo command:
# echo 1683 > /cgroup/cpuset/group1-web/tasks
• Use the cgexec command to launch a process in the
specified cgroup. For example, to execute the httpd
service in the group1-web cgroup:
# cgexec -g cpuset:group1-web httpd
• Alternatively, edit the associated file in /etc/sysconfig
to allocate the service to a cgroup automatically.

3 - 20 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Cgroup Rules Configuration File

• You can also set up rules so that processes are

automatically assigned to particular groups by cgred.
• The cgred daemon moves tasks into cgroups according
to the settings in the /etc/cgrules.conf file.
• The syntax to define a cgroup and associated subsystems
for a user is as follows:
user_name [:command_name] subsystem_name [,...]
cgroup_name

• For example, to assign the tasks run by the oracle user

to the cpu, cpuset, and memory subsystems in the
dbgrp cgroup:
oracle cpu,cpuset,memory dbgrp

• You can also use the * and @ characters.

3 - 22 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Enabling PAM to Use Cgroup Rules

• You can configure Pluggable Authentication Modules

(PAM) to use the rules that you define in the
/etc/cgrules.conf file.
• Install the libcgroup-pam software package:
# yum install libcgroup-pam
• Edit the /etc/pam.d/su configuration file and add the
following line:
session optional pam_cgroup.so
• For a service that has a configuration file in
/etc/sysconfig, add the following line to start the
service in a specified cgroup:
CGROUP_DAEMON=“*:cgroup_name”

3 - 23 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Getting Information About Cgroups

• To determine which process belongs to which cgroup, use

the following command:
# ps –O cgroup
• If the PID is known, use the following command:
# cat /proc/PID/cgroup
• The /proc/cgroups file contains subsystem information:
# cat /proc/cgroup
• The tree command lists all hierarchies and cgroups:
# tree /cgroups
• The lscgroup command lists the cgroups on a system:
# lscgroup

3 - 24 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 cgget Utility

• Use the cgget command to view the value of a subsystem

parameter. The syntax is:
cgget -r cgroup_file cgroups_list
• For example, to view the memory statistics for the cgroup
dbgrp:
# cgget -r memory.stat dbgrp
• Use the following syntax to list the values of all the
subsystem parameters:
cgget -r subsystem /
• For example, to list all the memory subsystem parameter
values:
# cgget -r memory /

3 - 26 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Quiz

Cgroups enable you to do which of the following?

a. Specify the relative share of CPU time available to the
tasks in a cgroup.
b. Specify memory limits for the tasks in a cgroup.
c. Suspend the tasks in a cgroup.
d. Report the CPU time and memory used by the tasks in a
cgroup.

3 - 27 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Quiz

Which of the following statements are true?

a. Cgroups can be hierarchical: each group inherits
characteristics from its parent group.
b. Many different hierarchies of cgroups can exist
simultaneously on a system.
c. Each time a new hierarchy is created on the system, all the
tasks on the system are initially members of the default
cgroup of that hierarchy, which is known as the root
cgroup.
d. When you create a child cgroup, the child inherits the
parent’s processes.

3 - 28 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Quiz

Which of the following commands allow you to assign

processes to a cgroup?
a. cgcreate
b. cgclassify
c. cgexec
d. echo

3 - 29 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Summary

In this lesson, you should have learned how to:

• Create control groups
• Assign tasks to control groups
• Set subsystem parameter values for control groups
• Use the configuration files for control groups and control
group rules
• Enable PAM to use control group rules
• Obtain information about control groups

3 - 30 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Practice 3: Overview

This practice covers the following topics:

• Exploring the default cgroup hierarchy
• Creating cgroups in the default hierarchy
• Creating a custom cgroup hierarchy
• Using cgroups to allocate CPU resources
• Configuring cgroup rules
• Configuring PAM to move processes to cgroups

3 - 31 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.