LPIC 201

Chapter 1: starting a system

Process:

1. firmware, Power-On-Self-Test (POST), boot loader

2. finds kernel
3. kernel starts other programs
see during the boot
dmesg -> boot messages (kernel ring buffer)
most distros save this in /var/log/boot (debian) or /var/log/boot.log (rh)
older IBMs firmware is called BIOS (Basic Input Output System) and newer
ones are called Unified Extensible Firmware Interface (UEFI)

BIOS

simple, booted from only one sector of hdd -> we needed 2 part loaders for
modern OSs: bootloader + OS. boot loader has a config file.
Master Boot Record is the first sector on first hard drive. BIOS checks MBR
and boots from it.
boot loader can point to another bootloader!

UEFI

Intel created it in 1998 and 2005 others joined. UEFI (Unified Extensible
Firmware Interface) has EFI System Partition (ESP) to store bootloader pro-
grams so we can have any size bootloader and multiple bootloaders. it is FAT
and most of the times on /boot/efi. UEFI uses its on mini bootloaders which
are called boot-managers. efibootmgr lets you add / remove boot entries or
change boot order.

Linux boot loaders

• Linux Loader (LILO)

• Grand Unified Bootloader (GRUB) Legacy
• GRUB2

1
we started with LILO in 1990 but no one uses it anymore. file is /etc/lilo.conf.
NO UEFI.
Grub came into life at 1999 and GRUB2 in 2005. They support UEFI and grub
2 has more features.
it is possible to boot the whole kernel from version 3.3.0 on UEFI.
but we DONT because GRUB has cool features.

Grub Legacy

the config is in /boot. on debuan it is called menu.lst and on RH grub.conf. It

has global definitions first and then definitions of each OS. globals are: - color
white/blue - default - fallback - hiddenmenu - splashimage - timeout
then you will have OSs: - title - root (hd0, 0) - kernel - initrd (initial RAM disk,
drivers nessesary for kernel to interact with hardware) - rootnoverify (for non
linux boot partitions)
sample:
default 0
timeout 10
color white/blue yellow/blue

title CentOS Linux

root (hd1,0)
kernel (hd1,0)/boot/vmlinuz
initrd /boot/initrd

title Windows
rootnoverify (hd0,0)
the grub-install /dev/sda install the grub on MBR (same ast grub-install
'(hd0)'.

GRUB2

a newer version but kind of similiar. in /boot/grub/ (so you can have both!)
now the config is like this:
menuentry "CentOS Linux" {
set root=(hd1,1)
linux /boot/vmlinuz
initrd /initrd
}

2
menuentry "Windows" {
set root=(hd0,1)
}
Two big differences: - set root= - hd(0,1) instead of (hd0,0)
the config is /boot/grub/grub.cfg but is created by global configs in
/etc/default/grub and OS configs in /etc/grub.d forlder and grub-mkconf >
/boot/grub/grub.cfg.
Lets see some interactions. You can use arrows, use E, F10, Boot, . . .

Others

there ar eothers like systemd-boot & syslinux & ISOLINUX (for live CDs)

Secure bootloaders

UEFI support something called secure boot. here UEFI only manages images
which are signed. they say this is for security but makes headaches for linux.
Solutions: - Disable it on UEFI boot manager - Purchase a digital signature and
sign your images! - use bootloader image signed by others
The 3rd is about companies invested, signed a mini-bootloader which starts
normal boot loaders. We have two. One from Linux Foundation (preloader) and
Fedora (shim).

Process Initialization

Linux OS consist of MANY programs and it needs many services to be running

in background. the init process, handles this and the whole process is called
initialization process. When the Kernel is up, it looks for a program called init
in /sbin/init, /etc/init & /bin/init (if all failes, it runs /bin/sh and if it also
fails, it goes in panic mode)
what init does, is based on system configuration. there are 3 main init programs:
- Unix System V (SysV) - systemd - Upstart
Hisotry: SysV based on UNIX. shell scripts. runlevels. but systems became more
complicated, now we have systemd with its targes (like runlevels) by redhat.
Ubuntu created upstart.

checking / changing runlevels

you can change the runlevel with init. so init 6 works like reboot. There are
also specific commands like ‘shutdown, halt, poweroff and reboot.

3
shutdown -h now

SysV

runlevels. stops and starts services based on runlevels. 0 to 6:

0: shutdown
1: single
2: multisuer graphical on debian
3: multiuser text on RH
4: not used
5: graphical on RH
6: reboot
the SysV has two ways of settings runlevels: /etc/inittab & startup scripts.
this is how each line of /etc/inittab looks like:
id:runlevel:action:process
id:3:initdefault:
actions are:
• boot: start at boot
• bootwait: start & wait till finished
• initdefault: enter this runlevel after system is up
• once: run when the runlevel is entered
• powerfail: start when powered down
• powerwait: start when powered down and wait for it
• respawn: start if terminated
• sysinit: start before any other at boot
• wait: start and wait for finish
the scripts are all in /etc/init.d/rc[0-6].d or /etc/rc[0-6].d. filenames
indicate if they have to be stopped or started and in which order.
to config the runlevels we have chkconfig (RH) and “‘update-rc.d““(Debian).
chckconfig --list network
network 0:off 1:off 2:off 3:off 4:off 5:off 6:off

chkconfig –-levels 12345 network on #with no levels, turns on on the defulat run level
On debian systems, we use update-rc.d.
update-rc.d program remove #wont start at default runlevel
update-rc.d –f program start 40 2 3 4 5 . stop 80 0 1 6 . # 40 & 80 are the orders

4
systemd

becomming default. controversy. monolistic. it has UNITS and TARGETS.

unit: service or action on system. name + type + config file (id is name.type).
we have 8 type of units: - automount - device - mount - path - service - snapshot
- socket - target
#sytemctl
qUNIT LOAD ACTIVE SUB DESCRIPTION
proc-sys-fs-binfmt_misc.automount loaded active waiting Arbitrary Executable File For
sys-devices-pci0000:00-0000:00:02.0-drm-card0-card0\x2deDP\x2d1-intel_backlight.device loa
abrtd.service loaded active running ABRT Automated Bug Reporting Tool
accounts-daemon.service loaded active running Accounts Service
alsa-state.service loaded active running Manage Sound Card State (restore and
atd.service loaded active running Job spooling tools
auditd.service loaded active running Security Auditing Service
avahi-daemon.service loaded active running Avahi mDNS/DNS-SD Stack
bluetooth.service loaded active running Bluetooth service
bolt.service loaded active running Thunderbolt system service
chronyd.service loaded active running NTP client/server
colord.service loaded active running Manage, Install and Generate Color Pr
crond.service loaded active running Command Scheduler
cups.service loaded active running CUPS Scheduler
targets looks like runlevels. a target is a different group of services running on
the system. at the moment systemd tries to be like SysV and it has targets like
runlevel0.target
unit configs are in /lib/systemd/system:
[root@funlife system]# cat tor.service
[Unit]
Description=Anonymizing overlay network for TCP
After=syslog.target network.target nss-lookup.target
PartOf=tor-master.service
ReloadPropagatedFrom=tor-master.service

[Service]
Type=notify
NotifyAccess=all
ExecStartPre=/usr/bin/tor --runasdaemon 0 --defaults-torrc /usr/share/tor/defaults-torrc -f
ExecStart=/usr/bin/tor --runasdaemon 0 --defaults-torrc /usr/share/tor/defaults-torrc -f /et
ExecReload=/bin/kill -HUP ${MAINPID}
KillSignal=SIGINT
TimeoutSec=30
Restart=on-failure
RestartSec=1

5
WatchdogSec=1m
LimitNOFILE=32768

# Hardening
PrivateTmp=yes
DeviceAllow=/dev/null rw
DeviceAllow=/dev/urandom r
ProtectHome=yes
ProtectSystem=full
ReadOnlyDirectories=/run
ReadOnlyDirectories=/var
ReadWriteDirectories=/run/tor
ReadWriteDirectories=/var/lib/tor
ReadWriteDirectories=/var/log/tor
CapabilityBoundingSet=CAP_SETUID CAP_SETGID CAP_NET_BIND_SERVICE CAP_DAC_READ_SEARCH
PermissionsStartOnly=yes

[Install]
WantedBy = multi-user.target
or a target:
[root@funlife system]# cat graphical.target
# SPDX-License-Identifier: LGPL-2.1+
#
# This file is part of systemd.
#
# systemd is free software; you can redistribute it and/or modify it
# under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation; either version 2.1 of the License, or
# (at your option) any later version.

[Unit]
Description=Graphical Interface
Documentation=man:systemd.special(7)
Requires=multi-user.target
Wants=display-manager.service
Conflicts=rescue.service rescue.target
After=multi-user.target rescue.service rescue.target display-manager.service
AllowIsolate=yes
the default is /etc/systemd/system/default.target.
program to sue with systemd is systemctl: - list-units: current status - default:
change default - isolate: start one and stop all others - start name - stop name -
reload name: reload the config - restart: shutdown and restart - status name/PID:
status - enable name - disable name

6
we can use isolate with targets to move between targest:
systemctl isolate rescue.target
It has its own logging which is called journalctl.

Upstart

ubuntu tried to craete its own SysV replacement. Everything is moved to

/etc/init folder. the files are called name.conf for service names and are like
this:
# tty1 - getty
#
# This service maintains a getty on tty1 from the point the system is
# started until it is shut down again.
start on stopped rc RUNLEVEL=[2345] and (
not-container or
container CONTAINER=lxc or
container CONTAINER=lxc-libvirt)
stop on runlevel [!2345]
respawn
exec /sbin/getty -8 38400 tty1
$
main feature is it can start services when a device is connected, not only runlevels.
stop bluetooth
start network

System Recovery

we will talk about kernel / device problems. ### kernel it is possible to use an
older kernel in Grub. it is possible to boot in single user mode if needed, passing
some kernel params may help

drive failure
separated partitions
booting with another disk / usb
mouting the root drive
mount /dev/sda1 /media
fsck /dev/sda1

7
Chapter 2, Maintain the system

keeping users informed

fluid messages are the ones active users get.

wall

logged in users who have their messages set to “yes”. use /bin/mesg to set your
meesage (mesg, mesg y, mesg n)
# who -T # + signs means the write access is granted
# write sara tty2
Hi there! Ctrl-D
to broadcast:
# wall message
# wall
hi there ctrl-D

> systemd sends a wall message in case of emergency, half, reboot, ...

notify-send

is a GUI utility in the form of notify-send title message. To send to others, you
have to use “w” to find the DISPLAY and then use something like this:
DISPLAY=:0 sudo su -c "notify-send "\hi\" \"please call jadi\"

### shutdown
we know it for reboot / halt but can do other things too!
shutdown [options] time [wall message]

-H: halt
-P: power off
-r: reboot
-c: cancel
-k: no new login
--no-wall: no wall

time can be hh:mm, +10, 0, now, ...

### static messges

based on files. called logon messges.

8
- /etc/issue: tty terminal logins. system access policy, outages, ..
- /etc/issue.net: for remote logins (telnet). to enable for ssh, add Banner /etc/issue.net t
- /etc/motd: message-of-the-day. shown after use logged in and before command line prompt.

## System Backup
backup strategy. Data categories. Value of each data category. Maximum data inaccessibility

Media: Magnetic disks (1 cent per gig, tapes upto tera!), CD, HDD, SSD, cloud

> Keep your backups safe!

Types: Full. Incremental. Differential. Snapshot.

Recovery: single/multi file recovery. Partition recovery. system recovery (RTO!)

Where to backup: show all dirs /etc, /home, /opt, /root, /srv, /usr, /var

### Softwares
There are some specific ones like BackupPC, Duplicity, Bareos, Bacula but you can use normal

tar

-c craete
-u update (add only new /modified)
-z compress gzip
-j compress bzip2
-J compress xz
-v verbose
-g filename create full/inc based on file

-d check the tar with existing files

-t show contents
-x open
-z uncompress with gunzip
-j uncompress with bzip2
-J uncompress sz

tar cfz backup.tar.gzip *.dat

touch newdat.dat
tar ufz backup.tar.gzip *.dat

### magnetic tapes

logic is like a road or writing on a roll of paper. they are called /dev/st[0-7] or /dev/nst

9
mt [-f device] operation [count] [arg] status: display status load: load tape (if not
automatically done) erase: erase everything fsf count: skip count (forward) bsf
count: rewind count tell: get current position eod: end of current data rewind:
to the beginning eject: rewind and unload offline: rewind and unload
mt -f /dev/sd0 status tar -Jcvf /dev/st0 /home/jadi
cd /tmp tar xf /dev/st0

rsync

can be used locally and remotely. -a is archive -v verbos -h more human readable
–progress
to use over network you need ssh to be installed (it is secure!)

dd

do not use on mounted! dd if=input of=output bs=4096 count=10 you can use
to write iso to the USB you can use to write ZERO on a disk you can use to
clone . . .

Installing from source

1. Download
2. Unpack
3. Read the docs
4. compile
5. errors? install dependencies
6. compile
7. move the output to somewhere permanent

Manage Resource Usage

“You can’t manage what you can’t measure.” ### memory free htop sar top
vmstat #virtual memory

CPU

htop iostat #per device mpstat #multi proccessor ps sar top w

10
Process

htop lsof #files and network pmap ps pstree top w

Network

iftop ip iptraf lsof netstat ntop sar ss #socket stat tcpdump

uptime / load

uptime htop top

Device IO

iostat iotop sar

sar

you can see sar works in many places. If you run sar with a particular option,
such as networking or disk information, and you get the response “ requested
activities not available in file ,” you will need to modify your sadc configuration.
On Red Hat–based systems, modify the file /etc/sysconfig/sysstat and add the
desired option to the SADC_OPTIONS line. On a Debian-based distribution,
modify the file /etc/default/sysstat and, within the sadc section, either add the
option to the SA1_OPTIONS line or make sure that ENABLE=“true” is set.
sar 1 4 #cpu
htop iftop iostat

to predict usage

understand softwares and you can have helper programs like Cacti, collectd,
MRTG, Nagios, RRDTool

Chapter 3: Kernel
What is kernel. Hardwas > Kernel > GUI/GNU > Applications kernel manages
memory, softwares, hardwares and filesystem. Monolitic
/proc/meminfo
to cehck shared memory between processes

11
ipcs
drivers are compiled in kernel or added to the kernel as modules. device files
can be - character device: data as a char: modems and terminals - block: disks -
network: packets
ls /dev/
file system managemet: ext, ext2, ext3, ext4, msdos, NFS, NTFS, ReiserFS
(good performance and recovery), SMB, XFS (high performance 64bit).

kernel parts

bzImage: large zipped kernel kernel: uncompressed kernel vmlinux: uncom-

pressed kernel binary, not as the final boot version vmlinuz: generic compressed
kernel binary ZImage: small binary with GNU Zip
kernel modules are at /lib/modules insmod modprobe
source site is officially kernel.org and in /usr/src/linux. we also have pathces to
prevent from downloading the whole thing again.
header files; C concept. needed to compile modules /usr/src/linux (deb) or
/usr/src/kernels (RH)
docs are at /usr/src/linux/Documentations /usr/src/kernels/

versions

• original: started with 0.01

• version 1 series: On 1994, 1.0 released after 0.95. continued with 1.x.y (x
is odd for tests)
• version 2: after version 1.3, we went for 2. same as 2.x.y logic
• version 2.6: on 2003 we had 2.6.0. stable and we kept it as 2.6.x.y, all were
production and -rc indicated release candidates
• version 3: for 20th anniversary! 3.x.y and -rc
• version 4: in 2015. 4.x.y -rc

compile a kernel

you dont need it in normal cases but these are the steps: 1. obtain the source
(kernel.org, tar at /usr/src craete short link as /usr/src/linux) 2. create a config
file (what features? /usr/src/linux/.config, you can run make config, or easier
make defconfig make menuconfig) 3. compile (make / make bzImage) 3.5. install
kernel (cp bzImage /boot/vmlinuz-4.3, System.map is for debugging. or do with
make install) 4. compile and install module files (make modules and then make

12
modules_install install in lib/modules/kernel version/) 5. creating initial ram
disk (mkinitrd outputfile version (RH), mkinitramfs -o outputfiel version (deb))
and move to /boot 6. add to grub (edit or update-grub)cat

maintain the kernel

/lib/modules are the modules or in /lib/modules/4.14.75 /etc/modules (de-

bian) or /etc/modules-load.d are ones that kernel will load configs are at
/etc/conf.modules and /etc/moduels.dep DKMS (dynamic kernel module sup-
port) can help you to introduce your modules and how they should be compiled
when a new kernel is released.
#lsmod #list modules
#modinfo iwlwifi

to install:
# insmod /lib/modules/4.18.11-200.fc28.x86_64/kernel/drivers/net/wireless/intel/iwlwifi/iwlw
# modprobe iwlwifi
insmod is based, need the exact file and does not understands dependencies.
important switches of modprob:
-r remove -n dry run -v verbose -c show current config

hardware

lspci
lsusb
understand the cold/hot plug
udev listens to hotplugs (/etc/udev/udev.conf). it matches the kernel message
with rules (/etc/udev/rules.d and /lib/udev/rules.d)

troubleshoot kernel

uname -a all -s name -n network host -r release -v version -m hw name -p

processor type -o operating system
/proc dynamic pseudo direcotry contains info on interrupts, ioports, direct
memory access (DMA). you can use cat
lsdev shows the data from /proc
kernel info in /proc/sys/kernel
echo “1” > /proc/acpi/ibm>kbdlight

13
or you can use “sysctl” utility

Chapter 4: Manage file system

we need to understand, be able to fix, attach, retach, ..

Understand filesystem structures

concept of partitioning partitions can span whole disk or even larger! LVM
highlevel formatting inode table file nameis not in inode, it is in a table
there are many filesystem types. The natives are:
• btrfs, newer, large files and filesystem size, its own RAID, .. COW (copy
on write)
• ext2, one of the originals, deprecated
• ext3, (2TB, 16TV), with journaling
• ext4, (16TB, 1EB)
• reiserFS (before ext3)
and non natives:
• ntfs
• vfat
• xfs
• zfs
you can learn more with man filesystem.

makig filesystems

mkfs -t ext4 mkfs.ext4

parted -l
sudo blkid

attach

mount -t fstype device mount_point

lost+Found used to recover files in ext2, ext3 & ext4
mount -a all -r readonly -o options: check=none (no integrity check) exec: permit
binary file exec group: tells which group can mount owner: allows a user to

14
mount ro: read only rw: read write sync: write buffer on every write user/users:
allows this users to mount
you can check attacheds with
mountpoint /

detach

umount vi test_file lsof test_file fuser test_file

attach removable manually

mount -t vfat /dev/sdb1 /mnt sync

attach persistently

/etc/fstab - partition/volume: /dev/sda3 or UUID - mount point - filesystem

type - mount options - backup (for dump utility) - integrity check (fsck)

mounting and systemd

systemd has mount unit files. for EACH mount point, a new file will be created
at /etc/systemd/system/*.mount

Additional Topics

virtual or memory based fs

/dev, /proc, /sys, /run

btrfs

new, special ideas (snapshots and subvolumes), COW, large files, uses B-Tree,
checksum, its own RAID, compression, ..

make one
mkfs -t btrfs /dev/sdb /dev/sdc #2 for RAID mkdir btrfs-test mount /dev/sdb
btrfs-test vi bt/file1.txt btrfs filesystem show

15
subvolumes
can act as subdirectories but can be mounted separately. they are not block
devices.
mount /dev/sdb bt btrfs subvolume create bt/new_subvolume btrfs subvolume
list bt #-t for table
when parent mounted, subvolumes are mounted too.
btrfs subvolume get-default bt umount bt mkdir subv mount -o sub-
vol=new_subvolume /dev/sdb subv

snapshots
snapshots are like subvolumes. easy to create. parent must be mounted
btrfs subvolume snapshot volume_mount_point snapshot_name
mount /dev/sdb bt btrfs subvolume snapshot new_subvolume new_subvolume_snapshot
btrfs subvolume list bt

optical fs

mostly for CD & DVD. - El Torito: lets CD to boot - HFS: created for mac.
read only on linux - HFS+ - ISO 9660: CD/DVD classic - Joliet: additions to
9660 by microsoft (longer filename, unicode, ..)
/dev/cdrom or /dev/dvd
normal mount or even mkfs. you can also mount iso

swap filesystem

concept
free
swapon swapoff

Network based filesystems

concept CIFS (Common Internet FS) & NFS (later) & Samba we can also include
Network Attached Storage (NAS) which is NFS or CIFS & Storage Attached
Networks (SAN) which is iSCSI.

16
Auto Mounting

mounting hotplugs to /run or /media. this is done by udev or older HAL

(hardware abstraction layer) used /mnt

AutoFS
AutoFS mounts NFS filesystems. can be at /etc/fstab but AutoFS is better in
performance. config is in /etc/auto.master (called master map) in the form of:
mount-point map-name [options]
and again.. systemd has units for this.

Encrypted FS

dm-crypt uses cryptsetup utility eCryptfs newer. layered on top of current file
system. just need “ecryptfs-utils” on the system! no new tools:
mount -t ext4 /dev/sdd1 /home mount -t eCryptfs /home /home
you can use /etc/fstab for this
Linux Unified Key Setup (luks) is an improved dm-crypt

Maintaining Linux File system

say fine tuning, changing labels, ..

adjusting

adjusting ext
there many utilities. including: - debugfs interarctive to modify metadata -
e2label change labels - resize2fs enlarge or shrink fs - tune2fs tune (UUID, labels,
...)
to change a label:
blkid /dev/sdc1 uuidgen sudo tune2fs /dev/sdc1 -U NEW_UUID

adjusting xfs
• xfs_admin: tune (UUID, label, ..)
• xfs_fsr: improvements
• xfs_growfs: expand

17
adjusting btrfs
• btrfs balance: relocates and balances
• btrfs-convert: convert and extended to btrfs (and vice versa)
• btrfstune: tune btrfs property set: set values like labels

checking and reparing

problems can happen! outage, physical problems, .. we should monitor and

repair.

checking ext
• fsck.* : check and optionaly repair -> will create files in lost+Found. An
example? if a file has an inode but no directory lists its name. fsck.xfs &
fsck.btrfs does nothing! Only fsck will check all the fastab
• debugfs: interactive / extract data
• dumpe2fs: display fs info
• tune2fs: lots of data. use -l to list attributes

checking xfs
• xfs_check: checks but does not repairs
• xfs_repair: checks and reprais. with -n, it will dry run (no fix)
• xfsdump: dumps fs data and attributes
• xfs_metadump: dumps meta to a file
• xfs_info: display and check. like xfs_grow -n
• xfsrestore: restore data and attributes

checking btrfs
• btrfs check: check and optionaly repair fs
• btrfs get property : set property
• btrfs rescue: recovers a damaged fs
• btrfs restore: restores files from a damaged fs
• btrfs scrub: checks all data
• btrfsck: replaced by btrfs check

SMART

Self Monitoring and Reporting Tech. smart devices are SSDs or SCSI and can
inform the software about the status. the package is “smartmontools”. the
daemon is “smartd” and the command line is “smartctl”.
logs are in /var/log/smartd, /var/log/messages & /var/log/syslog

18
configs in /etc/smartd.conf or /etc/smartmontools/smartd.conf
-smartctl -i /dev/sda1 : view devices -smartctl -t [logn|short|selftest] /dev/sda1:
test the device -smartctl -a /dev/sda1 : lots of info -smartctl -H /dev/sda1 :
short summary

Chapter 5 - Advanced Storages

RAID

Redundant Array of Independent(used to be Inexpensive) Disks or RAID is a

set of multiple physical disk partitions combined in a virtual single drive. Can
be called disk stripping. can be HW, SW or even firmwares.
under stand raid from a url like https://www.booleanworld.com/raid-levels-
explained/.
0: disk striping. faster rw. no fault tolerance. 1: disk mirroting. min of 2 disks
(and even!). good: fault. bad: cost 10: 1 + 0 5: disk striping with parity. faster
write (HW). min 3 disks. better data read time than 4, can revoer one faulty
disk in a timely manner. not recommended because if 2 fails. . . 6: disk striping
with double parity. min 4 disk. slower write but faster read and tolerates 2
faulty disks.
your kernel should be 2.6 and up. you should have /proc/mdstat or
you can check lsmod | grep raid
the command is mdadm (older one was raidtools2)
Have same sized partitions! Hex code should be da for MBR or fd
for older GPTs.
lsblk
fdisk /dev/sdb
fdisk /dev/sdc
fdisk /dev/sdd
losblk
create your RAID array using ‘mdadm’ (check the man page):
mdadm -C /dev/md0 -l 6 -n 4 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
if superblocks already created on devices, you can use -A to assemble them into
RAID. also check mdadm --create --help to see options for each mode
when created, you can check with /pro/mdstat. or you can check with
mdadm --misc --detail /dev/md0

19
then it is normal usage. format, mount, . . .
Tehcnically you dont need a config file (you have superblocks) but it is good to
have it:
mdadm --verbose --detail --scan /dev/md0 >> /etc/mdadm.conf

Monitoring

To monitor, ‘mdadm can be used. these are the event/alerts:

• DeviceDisappered
• RebuildStarted
• RebuildNN #tells percentage
• RebuildFinished
• Fail
• FailSpare #the spare you added is failing before synced
• SpareActive #spare became active
• NewArray
• DegradedArray #array member missing
• MoveSpare
• SpareMissing
• TestMessage
mdadm --monitor --help
mdadm --monitor options devices
some options: - –mail= - –program=
and you can change –monitor with –folow so it will check every 60secconds

spare devices

cocepts (extra device which can be added to array if one disk fails)
mdadm --misc --detail /dev/md0 | grep Spare
mdadm --manage --add /dev/md0 /dev/sde1

Remove RAID

First you have to stop it and then remove the super block of devices:
mdadm --manage --stop /dev/md0
mdadm --zero-superblock /dev/sdf1 /dev/sdg1 /dev/sdh1

20
Adjusting Storages

Not used much but is good to have an idea.

PATA. SATA. ATAIP. SCSI. SAS. iSCSI. AHCI (SSDs). NVMe (SSD on PCI:
/dev/nmve*).
DMA (direct memory access): so other parts wont be occupied Write Back
Cache: OS will be said “done” but .. you need sync.
one command is hdparm. It can change drive settings (but why? they are in
optimum).
sudo hdparm -I /dev/sdc
another is sdparm. It is hdparm for SCSI.
Also we have sysctl (do not confuse with systemctl). It can show/change kernel
params.
to see all parameters:
sysctl -a
all /proc/sys files are also editable kernel params.
sudo sysctl dev.cdrom.autoeject
sudo sysctl -w dev.cdrom.autoeject=1
these are reflected on /proc/sys/dev/cdrom/autoeject. to make them permanent
edit /etc/sysctl.conf. also you can edit that and then sysctl -p.

other tools

for SMART devices, there is smartctl and smartd. We can scheduled tests with
this daemon.
for nvme devices: nvme help
SSDs has some problems. specially kidn of fragmentation and too much writes
(can no simply edit). so you need trimming. the command is fstrim. if supportd:
sudo hdparm -I /dev/sda | grep TRI fstrim /home
iSCSI -=-=-=- Internet Small Computer System Interface (iSCSI) RFC3720.
lets remote storages appear as SCSIs. remtoe system with SCSI disk is ‘target’
and the system using the disk is ‘initiator’.
It is kind of SAN (Storage Area Network) which are attached storage devices.
there are protocols other than iSCSI too:
Fibre Channel Protocol: Fibre Channel SAN upto 32gig per seccond. Expensive
using FCP to move SCSI commands

21
ATA over Ethernet Protocol (AoE). runs on level 2 and transfers ATA commands
over Ethernet. can be shared with TCP/IP packets and make it cheaper. Easy
and secure.
FiberC Channel over Ethernet: FCoE not as expensvie as fiber. Encapsulates
Fiber Channel on Ethernet Networks.
iSCSI is cheaper than FCP and can coexists with TCP/IP
Logical Unit Number (LUN) is a number to identify one logical SCSI device on
target. start at 0 (first is lun0). can have aliases.
iSCSI Qualified Name (IQN) is a unique address:
iqn.domain-date.domain:unique-scsi-name (domain-date is year-month, when
the network is registered).
to setup, review the page 237 of LPIC-2: Linux Professional book.
== LVM Logical Volume Management lets multiple partitions to be grouped
and used as one partition.
Physical Volume: PV created by pvcreate. tells LVM to use one partition / disk
Volume Group: VG vgcreate command. adds PV to storage pool
Logical Volume: LV lvcreate. can be formatted as a linux partition
Physical Extent: smallest block size on PV (4MB default but can be changed
with -s switch).
lvm
lvm> help
we need 5 steps:
1. create PV
2. create VG
3. create LV
4. format LV
5. mount LV
lsblk
pvcreate /dev/sdb1
pvcreate /dev/sdb2
pvcreate /dev/sdb3
pvdisplay
for step 2:
vgdisplay
vgcreate vg00 /dev/sdb1 /dev/sdb2 /dev/sdb3
vgdisplay
step 3:

22
lvcreate -L 1G vg00
lvdisplay /dev/vg00/lvol0
lvscan
lvs
other steps are as before:
mkfs.ext3 /dev/vg00/lvl0
mount /dev/vg00/lvl0 /mnt

managing LVM

increase VG & LV
say we have a new vg! lets add it:
pvcreate -d /dev/sdc1
pvscan
pvdisplay /dev/sdc1
vgextend vg00 /dev/sdc1
lvextend -L 2g -v /dev/vg00/lvol0 #2g is the new total size
# during previous action, backup is metadata only
lvdisplay --maps /dev/vg00/lvol0

LV Snapshots
logic is having a snapshot using COW. first only metadata is copied and on each
write, COW happens.
lvcreate -v -L 500m -s -n my_snap /dev/vg00/lvol0
lvdisplay /dev/vg00/lvol0
lvdisplay /dev/vg00/my_snap
switches: - -s: snapshot - -n: name - -L: size! how much space should be used.
grows with each write command and if runs out of space, will be ususable
after using, to remove we can do:
umount /dev/vg00/my_snap
lvremove /dev/vg00/my_snap
sometimes snapshots helps us to create backups. you create a snap-
shot, let the system continue its writes & backup from the snapshot.

renaming
lvrename /dev/vg00/lvol0 /dev/vg00/mylv lvrename vg00 mylv yourlv

23
config file
/etc/lvm/lvm.conf man lvm.conf

Mapper

Device Mapper Kernel module, maps physiacal voluems ot virtual storage blocks.
dmsetup info.
dmsetup info /dev/vg00/yourlv

Chapter 6: Network

Understanding Network

Standard OSI layer has 7 layers: - Application - Presentation - Session - Transport

- Network - Data-Link - Physical
But sometimes we say 4 layers:
• Application
• Transport
• Network (How data is sent, local, internet, IP, hostname, default route,
netmask)
• Physical (Wired, WiFi, Switch, ..)

Physical Layer

Wifi, cable, Fiber, . . .

Network Layer

IP
like 127.0.0.1 or fed1::08d3:1319:8a2e:0370:7334, MAC

Default Route
logic, should be reachable

24
Netmask
using 1s in a 32 bit address, we can decide what is local and what should be
given to router. Say 255.255.255.0 means 8 bits are dedicated to local addresses.
can be shown by /24 (32-8).

Hostname
IP is difficult. Humans use hostnames (DNS)

DHCP
Dynamic Host Configuration Protocol

Transport

• Ports
• User Datagram Protocol
• Transmission Control Protocol

Application Layer

Here is where network programs process data. There are some wellknown ports:
- 21 TCP FTP - 22 TCP SSH - 23 TCP Telnet - 25 TCP SMTP - 53 UDP DNS
- 80 TCP HTTP - 143 TCP IMAP - 443 TCP HTTPS
you can check them in /etc/services

Configuring Network

You should be able to configure these: - host address - network address - default
route / default gateway - hostname - DNS server address
You can configure using the config files, commands or GUI apps

Files

Unfortunately Debian & RH based systems are different.

• Debian: /etc/network/interfaces
• RH: /etc/sysconfig/network-scripts
for Debian, this can be a sample file:

25
auto eth0
iface eth0
address 192.168.1.77
netmask 255.255.255.0
gateway 192.168.1.254
inet static

auto eth1
iface eth1 inet dhcp
And this is sample from RH world:
DEVICE="eth0"
NM_CONTROLLED="no"
ONBOOT=yes
TYPE=Ethernet
BOOTPROTO=static
NAME="System eth0"
IPADDR=192.168.1.77
NETMASK=255.255.255.0
IPV6INIT=yes
IPV6ADDR=2003:aef0::23d1::0a10:00a1/64
for DNSs we have /etc/resolv.conf file:
domain mydomain.com
search mytest.com
nameserver 192.168.1.1
but you can also have /etc/hosts file for commonly used domains or the ones
defined by your own.
/etc/hostname

GUIs

These days, many systems use Network Manager. There are others too.

Command lines tools

ifconfig, iwconfig, route, ip

ifconfig
ifconfig -a
ifconfig eth0 up 192.168.1.67 netmask 255.255.255.0
ifconfig eth0 hw ether AA:BB:CC:DD:EE:FF #change mac!

26
also ifup and ifdown can bring up / down an already configured interface.
When working with wireless:
iwlist wlan0 scan
iwconfig wlan0 essid "MyNetwork" key s:mypassword
the new tool is iw.

routing

netstat -nr
route
route del default gw 192.168.1.1
route add default gw 192.168.1.1

dhcp

to use dhcp you use one of these 3: - dhcpd #server - dhclient - pump

ip

In new versions of linux distributions, the ip command is beccomming popular.

It can do most of the things we saw:
ip addr show
ip route show
ip addr add 192.168.10.63/24 dev eth0
ip link set eth0 [up/down]
sudo ip route del default
sudo ip route add default via 192.168.1.1
sudo ip route add 10.0.0.0/8 dev tun0

Troubleshooting

it is always good to have a look at logfiles. it can be in dmesg or /ar/log/dmesg,

/var/log/syslog or /var/log/messages. Also check netgwork logs etc.
Address Resolution Protocol (ARP) is used to convert ip address to the physical
address.
arp
ss can investigate sockets. like ss -l (listen) or ss -a (all) or -m (memory usage).
ping ping6

27
traceroute traceroute6 mtr google.com (my trace route: ping + traceroute)
nc (netcat or swiss army knife) can read / write from TCP and UDP you can
create a chat session with “nc -l 1234” in one terminal and “nc localhost 1234”
on another window. You can also scan using it but. . . we have:
nmap can map the network, discover hosts, services, even versions, . . . nmap -v
google.com nmap -p 80 192.168.1.1/24
find host info: host google.com dig www.linux.org nslookup
lsof -i #only network lsof -i6
netstat normally shows all the open connections netstat -u #udp netstat -t #tcp
netstat -l #what applications are listening netstat -s #stats
tcpdump can monitor tcp flows: - -D : show available interfaces - -i ens33 : listen
on this one - -w dumpfile.pcap : write to file - -r dumpfile.pcap : read from this
file - -A : ascii - -c N : count of packets before exit - -i eth0 src 192.168.1.100
port 80 - -i eth0 dst 200.200.1.1

Security

tcp-wrapper works as a proxy. /etc/inet.conf you can have also have access lists:
/etc/hosts.allow /etc/hosts.deny

28