Lab 1: Installing pre-requisites for Ceph storage cluster

Introduction
In this Lab, you will install and configure the pre-requisites for deploying and managing a
Ceph cluster.
In this setup, we have:
1. Ceph Monitor - Monitors the cluster state, OSD map, and CRUSH map.
2. Ceph OSDs - Handles the data store, data replication, and recovery.

Below is the table, which contains details of the servers that we will be using for setting the
CEPH Storage cluster.

Role Hostname IP-Address

MON, MGR and MDS ceph-mon1 192.168.100.51
MON ceph-mon2 192.168.100.52
MON ceph-mon3 192.168.100.53
OSD ceph-osd1 192.168.100.61
OSD ceph-osd2 192.168.100.62
OSD ceph-osd3 192.168.100.63
OSD ceph-osd4 192.168.100.64
CLIENT ceph-client 192.168.100.71

Note: We need to perform the below steps on all the nodes. First, let us begin with the
ceph-mon1 node to understand the steps and after that, we can rerun the commands on
other nodes as well.

Student Material – Do Not Re-distribute. For any queries contact:

naushad.p.tech@gmail.com or https://www.linkedin.com/in/naushadpasha/
1. Login into the ceph-mon1 node as root user with the password linux.

1.1 Make sure that the SELinux is disabled for this lab purposes.

Security-Enhanced Linux (SELinux) is a mandatory access control (MAC) security

mechanism implemented in the kernel.

SELinux has three basic modes of operation, of which Enforcing is set as the
installation default mode.
• Enforcing: The default mode which will enable and enforce the SELinux
security policy on the system, denying access and logging actions
• Permissive: In Permissive mode, SELinux is enabled but will not enforce
the security policy, only warn and log actions. Permissive mode is useful
for troubleshooting SELinuxissues.
• Disabled: SELinux is turnedoff

# sed -i 's/enforcing/disabled/g' /etc/selinux/config

# setenforce 0
# sestatus

Output:

Student Material – Do Not Re-distribute. For any queries contact:

naushad.p.tech@gmail.com or https://www.linkedin.com/in/naushadpasha/
1.2 Enable and Start Chrony service (NTP Server).
Chrony is a versatile implementation of the Network Time Protocol (NTP).
The chrony suite installed by default and provides:
chronyd - Chrony daemon and chronyc - Command-line utility.

# systemctl enable --now chronyd

# systemctl status chronyd

Output: -

# chronyc sources
Output: -

Student Material – Do Not Re-distribute. For any queries contact:

naushad.p.tech@gmail.com or https://www.linkedin.com/in/naushadpasha/
1.3 Add an entry to /etc/hosts file for local name resolution.

# cat > /etc/hosts <<EOF

192.168.100.51 ceph-mon1.example.com ceph-mon1
192.168.100.52 ceph-mon2.example.com ceph-mon2
192.168.100.53 ceph-mon3.example.com ceph-mon3
192.168.100.61 ceph-osd1.example.com ceph-osd1
192.168.100.62 ceph-osd2.example.com ceph-osd2
192.168.100.63 ceph-osd3.example.com ceph-osd3
192.168.100.64 ceph-osd4.example.com ceph-osd4
192.168.100.71 ceph-client.example.com ceph-client
127.0.0.1 localhost
EOF

1.4 Verify the /etc/hosts file updated successfully.

# cat /etc/hosts

1.5 Test network connectivity between servers to ensure name resolution is working.

# ping -c 5 ceph-mon2

1.6 Let us remove the default container management tool - podman.

# yum -y remove podman

1.7 Let us add docker repository and install docker.

# yum config-manager --add-

repo=https://download.docker.com/linux/centos/docker-
ce.repo
# yum install docker-ce --nobest -y

Student Material – Do Not Re-distribute. For any queries contact:

naushad.p.tech@gmail.com or https://www.linkedin.com/in/naushadpasha/
1.8 Let us enable and start docker.

# systemctl enable --now docker

# systemctl status docker

Below is the reference image for understanding ceph cluster traffic flow.

2. Ensure that the server is updated with the latest packages by running the below
commands.

# yum update --nobest -y

Output:

Student Material – Do Not Re-distribute. For any queries contact:

naushad.p.tech@gmail.com or https://www.linkedin.com/in/naushadpasha/
3. Run the below group of commands on ceph-mon2, ceph-mon3, ceph-osd1, ceph-osd2,
ceph-osd3, ceph-osd4 and ceph-client servers.

systemctl enable --now chronyd

chronyc sources
sed -i 's/enforcing/disabled/g' /etc/selinux/config
setenforce 0
cat > /etc/hosts <<EOF
192.168.100.51 ceph-mon1.example.com ceph-mon1
192.168.100.52 ceph-mon2.example.com ceph-mon2
192.168.100.53 ceph-mon3.example.com ceph-mon3
192.168.100.61 ceph-osd1.example.com ceph-osd1
192.168.100.62 ceph-osd2.example.com ceph-osd2
192.168.100.63 ceph-osd3.example.com ceph-osd3
192.168.100.64 ceph-osd4.example.com ceph-osd4
192.168.100.71 ceph-client.example.com ceph-client
127.0.0.1 localhost
EOF
yum -y remove podman
yum config-manager --add-
repo=https://download.docker.com/linux/centos/docker-ce.repo
yum install docker-ce --nobest -y
systemctl enable --now docker
yum update --nobest -y

Student Material – Do Not Re-distribute. For any queries contact:

naushad.p.tech@gmail.com or https://www.linkedin.com/in/naushadpasha/