gracco / sysadmin-interview-questions Public

399 stars 98 forks Activity

Star Notifications

Code Issues 1 Pull requests

master Go to file

Gracco Guimaraes Kubernetes questions … on Sep 14, 2021 12

View code

README.md

Linux System Administrator/DevOps

Interview Questions
A collection of Linux sysadmin/Devops interview questions. Feel free to contribute via pull
requests, issues, or email messages.

Table of Contents

1. Contributors
2. General Questions
3. Simple Linux Questions
4. Medium Linux Questions
5. Hard Linux Questions
6. Expert Linux Questions
7. Networking Questions
8. MySQL Questions
9. DevOps Questions
10. Fun Questions
11. SRE Questions
12. Kubernetes Questions
13. Demo Time
14. Other Great References

####[⬆] Contributors:

moregeek
typhonius
schumar
negesti
peter
andreashappe
quatrix
biyanisuraj
pedroguima
Ben

####[⬆] General Questions:

What did you learn yesterday/this week?

I was reading about differences types of encryption, and their usage, when use an
asymmetric key or a symmetric key and what are the limits and applications of
GCP when need to encrypt backups for example.

Talk about your preferred development/administration environment. (OS, Editor,

Browsers, Tools, etc.)

OS - Linux editor - Vscode Browser- chromium/firefox tools - terraform, vim, git,

ZSH, ohmyzsh, Python, ansible, terminator, packer, meld, helm, kubectl, aws tools,
kubetail, eksctl distro - Arch Linux environment - i3wm + lxde

Tell me about the last major Linux project you finished.

My major project in life was an e-commerce creation/migration project using

opensource tools like centos, java, springcloud, mongodb, centos, rabbitmq,
cassandra, kafka+zookeeper, Azure, percona-mysql, NetflixOSS (as API gateway),
nodejs, took 2 years to complete, I joined since the very beginning.

Tell me about the biggest mistake you've made in [some recent time period] and how
you would do it differently today. What did you learn from this experience?
 I hired someone that it's a very good engineer but has very bad social skills. This
new person broke completely the team synergy and at the end the team itself. I
learned that friendly > technical skills

Why we must choose you?

I have a lot of experience, with different types of environments, I worked with all
three main cloud providers ( GCP, AWS and Azure), in different size of companies.
I have a huge experience scaling services, learning new technologies, automating
simple tasks, a cool mind to troubleshoot and solve problems in critical
environments. I have been working as tech lead, and product owner for SRE team.

What function does DNS play on a network?

It's on the core for any environment, responsible to translate IP addresses into
names, DNS can also provides a load balancer layer using geolocation, service
discovery using SRV entry and a lot of others features, like domain ownership
confirmation using TXT entries ( useful to generate SSL certs, for example )

What is HTTP?

HTTP (hypertext transport protocol) it's a protocol that defines how messages
are formated and transmitted via web, and what actions webservers and
browsers should take in response of various commands.

What are HTTP status codes?

HTTP status codes are predefined status of the task at the server
1xx - represents informational responses
2xx - represents succesful responses
3xx - represents redirect responses
4xx - represents client errors
5xx - represents server errors
The most commmons status codes are:
200 Success/OK
201 - CREATED - used by POST or PUT methods
304 - NOT MODIFIED - used in conditional GET Request to reduce bandwitdth
use
400 - BAD REQUEST - This can be due to validation errors or missing input
data
404 - NOT FOUND - Resource method is not available
 500 - INTERNAL SERVER ERROR - server threw some exceptions while
running the method
502 - BAD GATEWAY - Server was not able to get the response from another
upstream server

Describe the most common HTTP methods/verbs, and give examples:

GET - Read only operation, used to fetch detail from the server, downloads
POST - This method is used for the creationg of new resources on the server
PUT - This method is used to update existing resource on the server or to replace
the resource, PUT it's indepotent, and POST isn't, with PUT you can update a
resource N times, but if you try with post you will create N resources. PUT can
create resources.
PATCH - Applies a partial update to a resource and doesn't create a new resource
DELETE - This method is used to delete the resource on the server
TRACE - Provides a loop back test along the path to the target resource providing
a useful debugging mechanism.
OPTIONS - Fetches the list of supported options of resources present on the
server.

What is an HTTP proxy and how does it work?

An HTTP proxy it's a service that forwards HTTP connections, for example a user
A want's to access a server www.b.com, but for security reasons the user cannot
have direct access on internet, so the user's browser will ask for the HTTP proxy
to access the website, the webserver from www.b.com will receive a HTTP
message from the proxy and will answer as usual, but in the http HEADER from
the message will be changed by the HTTP proxy server who will add or change
some headers like User-Agent, X-Forwarded-For, used a lot by companies that
need to control their users Internet access.

Describe briefly how HTTPS works.

HTTPS use the same HTTP protocol but creates a security layer(tunnel) using
SSL/TLS, on top of it, this prevents anyone modify or inspect what's happening
inside this tunnel and ensure the client it's communicating with the right server.
The SSL handshake is established and after that all HTTP responses are send by.
The SSL uses both types of encryption, symmetrical and assymetrical, first when
the key exchange happens, a assymetrical encriptions is put in place to the result
of the key exchange be a symmetrical encryption.

What is SMTP? Give the basic scenario of how a mail message is delivered via SMTP.
 SMTP ( Simple Mail transport protocol) works in the application layer, and uses a
process called "store and forward", working close to a MTA (Mail Tranfer Agent),
this MTA service sends via SMTP a package with the messages, when this
message arrives at the destination, the client will use POP3/IMAP to download it.

What is RAID? What is RAID0, RAID1, RAID5, RAID10? RAID (Redundant Array of
Independent Disks) it' s a data storage virtualization technology that combines
multiple physical disks in one logical volume.

RAID0 - Striping, the data is striped between 2 or more disks, improving speed but
compromising availability.
RAID1 - Mirroring, the data is mirrored between 2 or more disks, improving
availability (the server could lose N - 1 disks) but compromises the speed
(replication time).
RAID5 - Blocks striped but uses distributed parity, RAID5 uses minimum 3 disks,
that stripe data between them, but replicates blocks too, it's a good RAID for
databases, in this scenario read operations are good, but write can be slow.
RAID10 - It's a implementation using RAID0 + RAID1, uses 4 disks, striping data
between 2 of then and mirroring this data in another 2 disks.

What is a level 0 backup? What is an incremental backup?

Level0 backup it's a full backup (all blocks), an after a level 0 backup, we can
initializes an incremental backup rotine ( only the difference between the blocks)

Describe the general file system hierarchy of a Linux system.

/ - root folder
/etc - configuration files provided by the package manager
/bin - binaries files
/sbin - system binaries files (important binaries for the OS)
/boot - Static files for boot processes ( boot loader)
/dev - Device files
/lib - Essential shared libraries and kernel modules
/usr - Secundary hierarchy
/mnt - Mounting point for temporary filesystem
/media - Mounting point for removable media
/opt - Add-on application software packages
/srv - Data service provided by this system
/tmp - Temporary files
 /var - Variable data
/root - Root user folder
/home - Home users folders

####[⬆] Simple Linux Questions:

What is the name and the UID of the administrator user?

root, 0

How to list all files, including hidden ones, in a directory?

ls -a or find .

What is the Unix/Linux command to remove a directory and its contents?

rm -R

Which command will show you free/used memory? Does free memory exist on Linux?

free
Sure exists, but the Linux kernel creates file caches in ram, so when we see the
output from free command, sometimes can show us that we are without memory
but this memory is cached by the OS.

How to search for the string "my konfi is the best" in files of a directory recursively?

grep -Rin "my konfi is the best" /folder

How to connect to a remote server or what is SSH?

ssh user@server - simple way

SSH (secure shell) it's a cryptographic network protocol, used for remote login to
computer systems by users.

How to get all environment variables and how can you use them?

env, set or printenv, will show every variable that login session, we can use setting
as variables ex.: TEST=something or using export the variable will be global (can
be used by all system users)

I get "command not found" when I run ifconfig -a . What can be wrong?

your PATH variable doesn't have the full path for the ifconfig command.

What happens if I type TAB-TAB?

 It depends which program the user is trying to complete

What command will show the available disk space on the Unix/Linux system?

df -h

What commands do you know that can be used to check DNS records?

dig +trace
nslookup
whois
host

What Unix/Linux commands will alter a files ownership, files permissions?

chmod, chown, chattr

What does chmod +x FILENAME do?

Add a execute permission to a file for all users

What does the permission 0750 on a file mean?

Add a permission to the owner of the file write, execute and read, the group of the
owner execute and read, and others do nothing

What does the permission 0750 on a directory mean?

Add a permission to the owner of the folder, create, get access and list the
directory files,for the group of the owner only enter and list the directory, and
nothing for others.
+r - User can list the files
+w - User can create a file inside the directory
+x - User can get into the folder

How to add a new system user without login permissions?

useradd username -s /bin/false

How to add/remove a group from a user?

usermod -a -G groupname username #to add a user in a new group

usermod -G [all groups that you want the user into] username #You don't remove
the user from a group, you add the user in all groups that this user is suppose to
be.
 newgrp - Updates shell session with new group permissions

What is a bash alias?

It's a shortcut for some bash command

How do you set the mail address of the root/a user?

creating a file called .forward in the user folder

editing the file /etc/aliases

What does CTRL-c do?

send a SIGINT to the terminal (it's a polite kill)

What is in /etc/services?

A mapping for services and ports, when a service call a function getportbyname()
usually this function goes in this file to check.
Example the command netstat or ss without the -n parameter

How to redirect STDOUT and STDERR in bash? (> /dev/null 2>&1)

1> redirect the STDOUT

1>> redirect the STDOUT in append mode
2> redirect the STDERR
2>> redirect the STDERR in append mode
&> redirect both STDERR and STDOUT
&>> redirect both STDERR and STDOUT in append mode
2>&1 redirect STDERR to STDOUT

What is the difference between UNIX and Linux.

Linux it's a UNIX "clone" using the same POSIX(Portable Operating System
Interface) standards, but UNIX it's a brand, has different copyrights and tools.

What is the diference between Telnet and SSH?

SSH it's encrypted and telnet isn't.

Telnet can ommit authentication
SSH adds overhead to the bandwidth

Explain the three load averages and what do they indicate. What command can be
used to view the load averages?
 The three load averages indicate the processor usage estimated in 1 minute,
estimated in 5 minutes and 15 minutes.
top or uptime

Can you name a lower-case letter that is not a valid option for GNU ls ?

####[⬆] Medium Linux Questions:

What do the following commands do and how would you use them?
tee

copies the STDOUT to a file, but continues to show the STDOUT.

awk

awk it's a programming language designed for text processing.

tr

tr or translate, it's a command to substitute characters.

cut

cut is a command for text processing and extracts a portion of a text

tac

tac it's a reverse cat, pritting the file bottom to up.

curl

curl or cURL is a tool to transfer data from or to a server, using one of the supported
protocols. cURL can be called a CLI browser, you can use to authenticate, change the
HEADER, and do a lot of stuffs with it.

wget

wget is a tool for retrieving files using HTTP, HTTPS , or FTP.

watch

Watch it's a tool that runs a specified command repeatedly and displays the result on
standard output.

head
 It's a command that shows the first lines of a file, the default it's 10 lines

tail

It's a command that shows the last lines of a file, the default it's 10 lines

What does an & after a command do?

Makes the command run in a background sub shell, and becomes a job.

What does & disown after a command do?

disown control jobs that are running in the system, without any paramenter or ID
removes the last job on the job table.

What is a packet filter, and how does it work?

Packet filter it's the process of passing or blocking packets at a network interface
based on source and destination address, port or protocols. The packet filter
examines, the header of every packet who passed through and based in the rules,
ACCEPT, DROP or REJECT the packet, it's well know as firewall.

What is Virtual Memory?

Virtual memory it's the amount of memory available for the system, physical
memory + swap memory (hard disk memory).

What is swap and what is it used for?

Swap it's a disk partition used by the Linux when the physical memory is full, if
the system needs more memory resources some inactive pages are copied to
swap, it was a common way to increase the computer/server memory using the
disk.

What is an A record, an NS record, a PTR record, a CNAME record, an MX record?

A record stands for address, indicates an IP address for a domain

NS stands for Name Server record indicates which DNS server is authoritative for
that domain ( Where the actual DNS entries are)
CNAME stands for canonical name and servers to make one domain to another
domain name.
MX stands for mail exchange, it's a list of mail exchange servers used by the
domain.

Are there any other RRs and what are they used for?
 Yes.
PRT record stands for pointer record and maps an IPV4 address to a CNAME
SOA record stands for State of Authority and is easily one of the most important
DNS records because stores information like when the domain was last updated.
SRV record stands for Service Record, is a record that specifies hostname and
port number for a specific service, it can be used for service discovery.
TXT record stands for Text Information, used by various purposes, as domain
ownership for example.

What is a Split-Horizon DNS?

It's the feature/configuration of the DNS server answer a different resolution to a

query based on the source of the query. A common use it's when the DNS server
needs to diferentiate internal and external queries, for the same domain. We can
use views to configure this.

What is the sticky bit?

It's a permission bit that is set on a file or a directory that let only the owner of the
file/directory or the root user to delete or rename the file.

What does the immutable bit do to a file?

It makes the file immutable, any user can change the state of the file or create
hard links.

What is the difference between hardlinks and symlinks? What happens when you
remove the source to a symlink/hardlink?

All files in the linux filesystem are a link to a inode, a hard link is a new link to the
same inode (if you remove or rename the old or the new link, the file will be intact,
but any change in the data on the inode is reflected in all files that refer to that
inode), the file system will only delete the inode if you don't have any link for this
inode. Because of this characteristic a hardlink only works on files that are in the
same file system.

A softlink, it's a link that points the link from the inode, so it's a link from a link if
the first link change the name or be deleted, the soft link will break, but can be
used between differents filesystems.

What is an inode and what fields are stored in an inode?

 Each object in the filesystem is represented by a inode that stores all the
information about the file, like file type, permissions, owner, group, file size, file
access, change and modification time (never birth time), file deletion time,
number of links, extended attributes. Each inode has a unique number and it can
acessed using stat filename

How to force/trigger a file system check on next reboot?

Create a file named forcefsck in the root folder

What is SNMP and what is it used for?

SNMP stands for simple network monitoring protocol, it's a protocol to monitor
devices, works in the application layer, has 3 versions now, they are not
compatible between each other, and V3 introduced encryption. Messages are
transported via UDP.

What is a runlevel and how to get the current runlevel?

Runlevel it's a preset operational system state, so based in this level, the OS
starts the corresponding services, or scripts. To get the current runlevel uses the
command runlevel, or who -r

What is SSH port forwarding?

SSH Port forwarding it's a way to create a tunnel between your machine in a
destination using ssh.

What is the difference between local and remote port forwarding?

Local port forwarding creates a tunnel between a local server and a local client, a
remote port uses a local server but with internet IP address to connect a internal
service that doesn't have access in the internet.

What are the steps to add a user to a system without using useradd/adduser?

Edit /etc/passwd with the new username, configure the home, and shell
Edit /etc/groups add this new username to some groups
Create the user home folder and set the right permissions
Reset the user password with passwd username

What is MAJOR and MINOR numbers of special files?

The MAJOR number will set to the kernel with kind of device it is, and MINOR
number will set a special characteristics of the device, example if a machine have
 2 disks, the MAJOR number will be the same for both, but the MINOR doesn't.

Describe the mknod command and when you'd use it.

mknod command creates a new device in /dev but actually udev creates
automatically each device, if something very terrible happen we can recreate
some devices using mknod to fix or make some backup.

Describe a scenario when you get a "filesystem is full" error, but df shows there is
free space.

When a filesystem it's out of inode, it can happen when you have a huge amount
of small files, in a small filesystem. df -i will show.

Describe a scenario when deleting a file, but 'df' not showing the space being freed.

when a process it's still appending that file. We can use lsof to check it.

Describe how 'ps' works.

the ps command read files from /proc and the content of these files are
generated by the kernel.

What happens to a child process that dies and has no parent process to wait for it and
what’s bad about this?

creates a zombie process, one zombie process it's not a big problem, but each
process uses a little size of ram, and uses a PID that's a finite number of it.

Explain briefly each one of the process states and all signals

CREATED or NEW STATE, in this moment the process wait the admission to the
ready state, by the scheduler
RUNNING/RUNNABLE (R) the process has been loaded into main memory and is
awaiting execution by the CPU, or it's using CPU core right now
SLEEPING a sleeping process is a process waiting for a resource to be available,
I/O operation to complete for example, or an event to happen. There is two states
of SLEEPING process
Interrruptible Sleep (S) - Process that can be terminated before the wake up
condition is fulfilled without any consequences.
Uninterruptible Sleep (D) - Process that can't be killed, in the example of I/O
operation, the act the process it's in uniterruptible sleep (D) until a the I/O
operation to complete and wake up.
 STOPPED (T) - A process becomes stopped when it receives the SIGSTOP signal,
when stopped the process execution is suspended and only signals it will handle
are SIGKILL and SIGCONT
Zombie (Z) it's a state after completing the execution or being explicitly killed, but
the process remains as a zombie until the parent process call the wait system
call to read its exit status, and finally ending the process lifetime.
Process SIGNALS are one of the ways process communicate among themselves
and with the kernel. Exceptionally SIGKILL and SIGSTOP signals cannot be
handled or blocked.
SIGTERM - the default signal sent by kill command, Asks the process to
terminate voluntarily
SIGKILL - unlike SIGTERM, forces the process to terminate, can't be blocked
or handled
SIGSTOP - suspend the process execution, putting in stopped state. In this
state, the process will do nothing but accept SIGKILL or SIGCONT.
SIGSTP - almost identical to SIGSTOP, the only difference is it can be blocked
or handled, this is the signal sent when you type <ctrl>+z in the terminal
SIGCONT - if a process is in stopped state, it will put it back in the
RUNNING/RUNNABLE state and resume it execution. If the process is in any
other state, it's silently ignored.
SIGINT - generated when the user type <ctrl>+c in the terminal, it interrupts
the current command processing and wait for user's next command.
SIGQUIT - generated when eht user type <ctrl>+\ in the terminal, normally
it will force the process to produce a core dump and terminate.
SIGALARM - signal used to wake up sleeping process, normally scheduled by
alarm system call.
SIGCHLD - sinal send from a child process to its parent process when its
state changes.
SIGHUP - the signal indicates the terminal handling the process has been
disconnected and/or parent process terminated. To run a process that won't
terminate when the terminal disconnects, you can start it using the
command nohup .

How to know which process listens on a specific port?

lsof -i :$PORT or netstat -lp | grep $port or ss -lp | grep $port

What is a zombie process and what could be the cause of it?

 A zombie process is a process whose execution is completed but it still has an
entry in the process table. Zombie process usually occurs for child process, as
the parent process still needs to read its child exit status.

You run a bash script and you want to see its output on your terminal and save it to a
file at the same time. How could you do it?

Use the tee command

script | tee file

Explain what echo "1" > /proc/sys/net/ipv4/ip_forward does.

Disable ipv4 ip_forward function from the kernel, as well the IPV4 routing function

Describe briefly the steps you need to take in order to create and install a valid
certificate for the site https://foo.example.com.

Create a key file

Uses this key file to create a csr file
Send this csr file to a ssl certificate provider
Get the crt from the certificate provider with the CA chain and configure into the
webserver
Or you can use certbot to simplify.

Can you have several HTTPS virtual hosts sharing the same IP?

Yes using virtualhosts, but the client needs to support http/1.1, to use name-
based virtual host configuration.

What is a wildcard certificate?

It's a certificate that can be used by differents hostnames from a single domain.

Which Linux file types do you know?

Regular file
Directory file
Special files
Block file
character file
named pipe file
symbolic link file
socket file
What is the difference between a process and a thread? And parent and child
processes after a fork system call?

A fork it's an identical process as the parent but with new PID, it has a own
memory share, and runs independently from the parent. A thread it's a lightweight
process and usually it's just a CPU state with the process containing the
remainings. A threads require less overhead then forking or spawning a new
process, because doesn't have a new system virtual memory space and
environment.
Both child and parent process have different PIDs, neither process access the
variables of each other, the child process ctime, uptime, stime, cutime and cstime
subrotines are set to 0.

What is the difference between exec and fork?

A fork in a simple way, it's a process copy only changing the pid and resource
limits, a exec it's a call that basically replaces the entire current process with a
new program. It loads the program into the current process space and runs it
from the entry point. Example, when we call the find command, our bash forks
itself, and in this new fork context, uses exec call to execute the find program.

What is "nohup" used for?

It's used to create process that are independent from user login, starting a
process with nohup it's telling the process to ignore SIGHUP calls, that the signal
sent by the kernel when the parent shell is closed.

What is the difference between these two commands?

myvar=hello

export myvar=hello

The first one create the variable only in the user context, the second in a global
context, so this variable can be used by all users.

How many NTP servers would you configure in your local ntp.conf?

4 it's minimum recommended by RedHat

What does the column 'reach' mean in ntpq -p output?

It's a octal number, that show the last 8 transactions with the ntp server, this
number is a FIFO log, so if the same packet doesn't arrive (it's UDP), this number
can be different based in the order of the checks.
 You need to upgrade kernel at 100-1000 servers, how you would do this?

I would be using Ansible, but before I would test in a controled group to check if
something bad can happen.

How can you get Host, Channel, ID, LUN of SCSI disk?

cat /proc/scsi/scsi

How can you limit process memory usage?

Calling the program with limit command, or set the ulimit in the console, or in the
/etc/security/limits.d, or in the systemd init script.

What is bash quick substitution/caret replace(^x^y)?

sed -e 's/^x/^y/g'

Do you know of any alternative shells? If so, have you used any?

I use zsh, it's 100% bash compatible

What is a tarpipe (or, how would you go about copying everything, including hardlinks
and special files, from one server to another)?

It's a way to copy a directory to a server from another preserving permissions and
the files, usually I don't copy files from a server to another, I use automation to do
the job to recreate the server for me, but if I really need to copy, we could use a
tarpipe, or dd.

####[⬆] Hard Linux Questions:

What is a tunnel and how you can bypass a http proxy?

We can create a ssh tunnel with ssh -R and redirect the http proxy to a server that
has access to the internet.

What is the difference between IDS and IPS?

IDS detect the problem inspecting the packet header and payload and creates a
log.
IPS detect the problem inspecting the packet header and payload and drops the
packet if finds something problematic, based in some pre defined rules.

What shortcuts do you use on a regular basis?

 ai package - sudo apt install package

ll - ls -lha

gitcm - git commit -m

gl - git pull

gp - git push

.. - cd ..

... - cd ../..

.... - cd ../../../

What is the Linux Standard Base?

It's a joint project projected by several Linux distributions under the organizational
structure of the Linux Foundation to standardize the sofware system structure,
including filesystem hierarchy.

What is an atomic operation?

Atomic operations are program operations that run until completion

independently from any other process.

Your freshly configured HTTP server is not running after a restart, what can you do?

I would try to see the logs and check what's the problem. jounalctl -xe
systemctl status httpd

What kind of keys are in ~/.ssh/authorized_keys and what it is this file used for?

Public keys, they are used to authenticate users in the server.

I've added my public ssh key into authorized_keys but I'm still getting a password
prompt, what can be wrong?

The permission for authorized_keys file, and .ssh folder, as the path from
authorized_keys file needs to be /home/user/.ssh/authorized_keys and right
spelled, or your private key has the wrong permission.

Did you ever create RPM's, DEB's or solaris pkg's?

yes using fpm, and using dpkg or rpm.

What does :(){ :|:& };: do on your system?

creates a forkbomb
How do you catch a Linux signal on a script?

using the command trap inside the script

Can you catch a SIGKILL?

No, by security proposes

What's happening when the Linux kernel is starting the OOM killer and how does it
choose which process to kill first?

OOM will kill the process that will free more memory and the least important for
the OS.

Describe the linux boot process with as much detail as possible, starting from when
the system is powered on and ending when you get a prompt.

BIOS/UEFI
BIOS performs startup based in the hardware, POST ( Power On Self Test)
processs to initiallize the hardware, after complete, and calls the bootloader.
bootloader
The bootloader (GRUB2) present options to the user select, GRUB supports
unix-like OS, and chain-load Windows OS, and loads the kernel into memory
and supplies it with some parameters.
kernel
The kernel will decompress itself and will setup essential hardware and
memory paging, and calls start_kernel() function, and it will perform the
majority of system setups like device and driver initialization, scheduler, idle
process and then starts separately in the user space the init process (pid 1).
init -The init it's scripts executed by shell (sysV, runit) or configuration files that
are executed by binaries (upstart, systemd), init has specific levels, that are
passed as variables at the call, with consists of specifics set of daemons. These
will provide various non-operating system services and structures and form the
user environment.
User environment
The typical desktop environment begins with a daemon that calls everything
needed. To shutdown, it's the inverse, the kernel kills every process, and
shutdown.

What's a chroot jail?

Chroot jail it's a way to isolate a process and its children from the rest of the
system. The idea is that you create a directory tree where you copy or link in all
 the system files needed for a process to run, usually we use bind to mount some
folder inside a chroot.

When trying to umount a directory it says it's busy, how to find out which PID holds the
directory?

lsof directory

What's LD_PRELOAD and when it's used?

LD_PRELOAD it's a variable that can be used to load some library before the
default C library, can be used to test a new version for a library, or for
development proposes.

You ran a binary and nothing happened. How would you debug this?

strace binary, and see what's happening.

What are cgroups? Can you specify a scenario where you could use them?

Cgroups are a Linux kernel feature that allow limit the resource use for a group of
process(CPU, memory, disk I/O). A scenario to use could be to test a software in
a physical machine that has a big hardware, and make this software run a
minimum configuration, a very common sofware that uses cgroups it's in
contairners ( docker, crio).

####[⬆] Expert Linux Questions:

A running process gets EAGAIN: Resource temporarily unavailable on reading a

socket. How can you close this bad socket/file descriptor without killing the process?
using gdb, attach the gdb in the process and close the file descriptor or socket
that is problematic, and detach.

####[⬆] Networking Questions:

What is localhost and why would ping localhost fail?

Localhost it's the internal interface in Linux, that some programs can use to talk
to each other inside the server. The ping will fail if the lo interface is down, or if we
don't resolve localhost in our /etc/hosts file or some firewall rule is dropping
ICMP packets or blocking lo interface.

What is the similarity between "ping" & "traceroute" ? How is traceroute able to find the
hops.
 Both use ICMP (Internet control message protocol) packets to archive their
proposes, but traceroute sends the packets gradually increasing the TTL value,
starting with TTL 1. The first router receives the packet, decrements the TTL value
and drops the packet because the TTL has zero. The router sends an ICMP Time
Exceeded message back to the source.

What is the command used to show all open ports and/or socket connections on a
machine?

lsof -i

netstat -a

ss -a

Is 300.168.0.123 a valid IPv4 address?

no

Which IP ranges/subnets are "private" or "non-routable" (RFC 1918)?

10.0.0.0/8
192.168.0.0/16
172.16.0.0/16

What is a VLAN?

It's a virtual lan created to separate networks inside a switch, making the
broadcast domain shorter, and for security proposes. Works in the network layer
(OSI Layer 2)

What is ARP and what is it used for?

It's a layer 2 protocol that maps IP address to MAC address.

What is the difference between TCP and UDP?

TCP ( Transmissions control protocol ) and UDP ( User Datagram Protocol ), both
works in the layer 3 of the OSI model, and are different methods to send
information across networks, or Internet. TCP is used in scenarios when reliability
is important, and is used by the majority of layer 7 protocols, like HTTP, FTP,
SMTP. TCP is connection-oriented ( after estabilishes the connection between
two devices, maintains until the transfer process finishes), and uses a process
called three-way handshake ( SYN, SYN-ACK, ACK)..
UDP it's a connectionless protocol ( doesn't estabilish a connection before hand)
it's much simple and used in situations when data loss is acceptable, because
 doesn't guarantee all data is successfully transferred.

What is the purpose of a default gateway?

The default gateway it's the way to get in other networks.

What is command used to show the routing table on a Linux box?

route
netstat -r
ip route list
ip r

A TCP connection on a network can be uniquely defined by 4 things. What are those
things?

remote-ip-address
remote-port
source-ip-address
source-port

When a client running a web browser connects to a web server, what is the source
port and what is the destination port of the connection?

source port it's dynamic based on net.ipv4.ip_local_port_range defined between

32768 - 61000, destination port 80 or 443.

How do you add an IPv6 address to a specific interface?

using ip -6 addr command, or using ifconfig ip inet6, or editing the OS file for
network interfaces.

You have added an IPv4 and IPv6 address to interface eth0. A ping to the v4 address
is working but a ping to the v6 address gives yout the response sendmsg: operation
not permitted . What could be wrong?

can be a firewall rule

What is SNAT and when should it be used?

SNAT stands for Source Network Address Translation - changes the source
address in IP header of a packet. The typical usage is to change the private
address/port to a public address/port for packets leaving the network.
 Explain how could you ssh login into a Linux system that DROPs all new incoming
packets using a SSH tunnel.

We could login using some DRAC interface if the machine it's physical, or we can
use the libvirt, or vmware console if virtual, or aws console.

How do you stop a DDoS attack?

You try to block the source address of the attack, or we use some CDN.

How can you see content of an ip packet?

capturing the packet with tcpdump, and opening with wireshark.

####[⬆] DevOps Questions:

Can you describe your workflow when you create a script?

First I think about the problem, then create some concept to solve a small part of
the problem, than I interate on top of if, adding more layers of complexity, at the
end I test, and run some lint, create a pull request and submit to the team
evaluate the solution proposed.

What is GIT?

It's a SCM (source code management), it's a software that controls source code,
and helps developers to work together and to share code.

What is a dynamically/statically linked file?

dynamically uses the dynamic path for the file example ../../file
statically uses the static path for the file example /usr/local/bin/file

What does "./configure && make && make install" do?

Compiles a source code, creates a binary file and copy to a pre defined path

What is puppet/chef/ansible used for?

Automatize infrastructure, you create a code and run this code against an
infrastructure

What is Nagios/Zenoss/NewRelic used for?

Monitoring infrastructure and services.

What is the difference between Containers and VMs?

 VMs have all the OS stack, devices and so, containers uses the cgroup
implementation from the kernel, and have a smaller footprint because of that.

How do you create a new postgres user?

createuser command

What is a virtual IP address? What is a cluster?

Virtual IP address, it's IP address defined in a virtual interface, a cluster it's 2 or

more machines working together to delivery a service.

How do you print all strings of printable characters present in a file?

using the command strings

How do you find shared library dependencies?

ldd /fullpath/command

readelf -d /fullpath/command

What is Automake and Autoconf?

They are programming tools to automate parts of the compilation process.

./configure shows an error that libfoobar is missing on your system, how could you fix
this, what could be wrong?

Installing the dev or devel version of the lib, that will contain the source code to
be compiled

What are the advantages/disadvantages of script vs compiled program?

Scripts are easy to correct and see how works, compiled are much faster

What's the relationship between continuous delivery and DevOps?

Continous delivery it's one subject inside the DevOps methodology, this subject
explain and show us how to delivery sofware or services fast, without human
intervention.

What are the important aspects of a system of continuous integration and

deployment?

testing, automatize the process, decrease human error, automate quality of code,
check of vulnerabilities, lint to have a more reliable and trustful process/delivery
####[⬆] Fun Questions:

A careless sysadmin executes the following command: chmod 444 /bin/chmod -

what do you do to fix this?

find the library that controls the chmod command, with ldd, after that call the
/lib64/ld-linux-x86-64.so.2 /bin/chmod +x /bin/chmod

I've lost my root password, what can I do?

you can use another administrator account to access and reset, or you can reset
the server passing in the grub2 init=/bin/bash and reset the password.

I've rebooted a remote server but after 10 minutes I'm still not able to ssh into it, what
can be wrong?

network issues, or some fsck it's running, or some service locked the startup.

If you were stuck on a desert island with only 5 command-line utilities, which would
you choose?

cd, vim, cat, awk, find

You come across a random computer and it appears to be a command console for
the universe. What is the first thing you type?

lshw

Tell me about a creative way that you've used SSH?

a for loop to send via ssh a command to 50 vms.

to connect in an internal deployed gitlab service, using a IPSec VPN connected in
a bastion host.

You have deleted by error a running script, what could you do to restore it?

I could copy the file descriptor from the /proc/PID/fd/number_of_file_descriptor

into a new file.

What will happen on 19 January 2038?

The standard time libray from C was developed using 4-byte to storage time and
4-byte integer its, a 2 trillion number, that in seconds, translates to January 2038,
in that time some mainframes could have some issue, openbsd is already
patched.
####[⬆] SRE Questions:

Can you explain what SLA means?

SLA ( Service Level Agreement) defines the level of service expected by a

customer from a supplier.

What's the “five nines” (“nine fives”, “two and a half nines”) uptime?

FIve nines measure the 99,999% of availability of a service, meaning that service
can be only offline 5.26 minutes in a year.

What would be the good SLI for an API service? How would you use an SLI to meet the
SLO?

User latency, error rate are good SLI. Every indicator that measures the user
experience it can be a good. Based in the SLI you can change, improve the
system or process to follow the SLO.

What are the SRE Signals?

Latency: Or response time, it's the time taken to serve a request. The increase of
latency it's key indicator of degradation in an aplication.
Traffic: It's the number of requests flowing across your network. Monitoring traffic
can help to identify capacity problems and plan ahead future demand.
Errors Rate: Indicates the rate of requests that fail, it's important to know when
your application it's running with errors, and if they are increasing.
Saturation: Measures the usage of a service, and how health is it. CPU, Memory,
IO are good metrics, increase of latency it's often a saturation issue.

True or false, you should always aim to make your service as reliable as it can
possibly be?

False, you need to keep your service reliable until it's interesting for the end user,
to avoid costs.

Explain the differences between GKE ingress and Nginx Ingress

GKE Ingress it a controller that it will spin up a cloud based ( GCP in the case)
Loadbalancer that will be responsible to intepretate ingress rules defined in your
Kubernetes cluster and expose them to the internet.
NGINX Ingress it a webserver running as a Ingress Controller inside your
kubernetes, that it will interpretate the ingress rules defined and expose them to
the internet.
####[⬆] Kubernetes and Docker Questions:

Describe a simple approach to efficient manage container resources K8S cluster

In the pod or deployment specification you can set limits for that container or
requests

Seting a request for Containers in a pod, the scheduler it will use this information
to decide which node to run that container, the container can if needed use more
resource than what it was set in the request.
Setting a limit it will limit the container to use maximum that amount of memory
or CPU cycles, this is enforced by the kubelet the container. If only limits is set the
kube scheduler it will set the request that matches the limit.
When a process tries to use more than the allowed amount of memory the
system kernel terminates the process with a OOM Error ( out of memory )
Configure a HPA

What are the layers in Docker and why are they useful?

Docker layer it's a file generated from running some command during a docker
build, and be accessed in the docker host /var/lib/docker/aufs/diff , they can
be used as cache.
docker history [image name] shows all layers

Because of the usage of AUFS ( Advanced multi-layered unification filesystem )

when docker mounts those layers a diff is applied and layers are reused.
Each command in a dockerfile generates a layer diff, so if you install a software
with a command uses the same "subshell" to cleanup caches and other files that
are not important for the container lifecycle. Only RUN, COPY and ADD creates
layers.

What features of the Linux kernel enable Docker to work?

CGroups
Namespaces

What is the difference between the COPY and ADD commands in a Dockerfile?

ADD and COPY have the same usage but ADD supports tar and remote url
handling, so you can download files or untar directly in the image. But the official
documentation suggest uses COPY because of the better control of the output.

Describe the lifecycle of a Pod? What is the phase of a Pod?

 Pods are considered ephemeral rather than durable entities, and are created,
assigned an unique ID and scheduled to a node, do not self-heal by themselves, if
the node goes down, a new pod with new ID is rescheduled.
Pods have status and a PodStatus field which has phase field and it can be:
Pending: The Pod has been accepted by Kubernetes Cluster, but one or more
containers has not been set up. This includes the time that a Pod spends
waiting download container images.
Running: The Pod has been bound to a node, and all of the containers have
been created.
Suceeded: All containers in the Pod have terminated in success, and will not
be restarted.
Failed: All containers in the Pod have terminated, and at least one conetainer
has terminated in failure. That is the container either exited with non-zero
status or was terminated by the system.
Unknown: For some readon the state of the Pod could not be obtained. This
phase typically occurs dut to an error in communicating with the node where
the Pod should be running.

What are three types of handlers to perform a diagnostic by Kubernetes engine?

A probe is a diagnostic performed by the kubelet on a Container, to diagnost the

kubelet calls a handler implemented by the container.
ExecAction: Executes a specified command inside the container. The diagnostic
is considered successful if the command exits with a status code of 0.
TCPSocketAction: Performs a TCP check agains the Pod's IP address on a
specific port. The diagnostic is considered successful if the command the port is
open.
HTTPGetAction: Performs an HTTP GET request against the Pod's IP address on
a specified port and path. The Diagnostic is considered succesful if the response
has a status code greater or equal to 200 and less than 400.

When should you use a liveness, readiness and startup probe?

The kubelet use liveness probes to know when to restart a container, one
example is a deadlock. Readiness probe is used when the kubelet needs to know
if the container is ready to accept traffic, and this signal is used to control which
Pods are used as backend Services. When is not ready is removed from Service
load balancer. The startup probe is used by kubelet to know when a container
application has started, if is configured, it disables liveness and readiness checks
until succeeds. This can be used to adopt liveness checks on slow starting
 containers, avoiding them getting killed by the kubelet before they are up and
running.

What are Init Containers? How do they differ from regular containers? Why are they
useful?

Init containers are used by Pods to run before the app container start. They differ
from app containers, as they always need to run to completion, so they don't have
probes, and each init container must complete successfully before the next one
starts, if not the kubelet it will apply the restartPolicy defined in the Pod, until
the init container is succesful. They are useful because they can be used by a
blocker or delay of dependencies, to start Pods in parallel, they can run utilities
that would make the app container less secure.

postStart & preStop events?

Kubernetes has component lifecycle hooks and they are exposed at the container
level:
PostStart is executed immediately after a container is created.

PreStop is called immediately before a container is terminated due to an

API request or management event, such as probes.

Why use a Statefulset?

Stateful sets different than deployments, have a unique network identifiers, and
DNS names inside of the kubernetes network, stable storage, and pod name label,
deployment and scaling guarantees like deploy and termination order.

What is PDB (Pod Disruption Budget)?

The PDB limits the number of Pods of a replicated service that are down
simultaneously from voluntary disruptions, a node drain for example ( Kubectl
drain)

What is an Operator? Why do we need Operators?

Operators are sofware extensions to Kubernetes that make use of custom

resources to manage applications and their components. The pattern aims to
capture the key aim of a human operator who is managing a service of set of
services. Kubernetes controllers concepts lets you extend cluster's behaviour
modifying the code of Kubernetes itself. Operators are clients of Kubernetes PI
that act as controllers for a custom resource ( extensions of the kubernetes API).
 We need operators to manage complex clusters automatically in a logical way,
like a mongodb replicaset or a elasticsearch cluster for example.

What is the role of the Kubelet, Kube-scheduler, Kube-apiserver, and Kube-proxy, kube-
controller-manager?

Kubelet, it's the primary node agent. It can register the node with the API server,
ensure the containers described in the PodSpec are running and healthy.
Kube-scheduler is a control plane process which assigns Pods to Nodes and
determines which nodes are valid placements for a Pod in the schedule queue
according to constaints and available resources. Multiple different schedules can
be used in a cluster, and kube-scheduler is the reference implementation.
Kubernetes API Server validates and configures data for the API objects which
includes pods, services, replicationcontrollers and others. The api server services
REST operations and provides frontend to the cluster shared state.
Kube-proxy it's the kubernetes network proxy that runs in each node, this reflects
services as defined in the Kubernetes API on each nod and can do simple TCP,
UDP and SCTP( Service Transmission Control Protocol) fowarding accross a set
of backends.
The kubernetes controller manager is a deamon that embeds the core control
loop, it's a non terminating loop that regualtes the state of the system. In
Kubernetes the control loop that watches the shared state of the cluster through
the apiserver and endpoints controller, namespace controller and serviceaccount
controller.

Explain the role of CRD (Custom Resource Definition) in K8?

CRD or custom resource definitions are extentions of the Kubernetes API.

How POD to POD communication works?

Each pod is assigned a unique IP, every container in a pod shares network namespace,
including the IP and network ports. Inside of a pod containers that are part can
communicate using localhost. When pods communicate with entities outside the
pods, they must coordinate how they use shared network resources.

How POD to service communication works?

Kubernetes service is a abstract way to expose an application running on a set of

pods, as a network service.

What's a CNI?
 How to troubleshoot if the POD is not getting scheduled?

How to run a POD on particular node?

What are the various things can be done to increase the K8 security?

How to monitor K8 cluster?

####[⬆] Service Mesh Questions:

####[⬆] Demo Time:

Unpack test.tar.gz without man pages or google.

tar xf test.tar.gz

Remove all ".pyc" files from testdir recursively?

find testdir -name '.pyc' -delete

Search for "my konfu is the best" in all .py files.

grep 'string' .py

Replace the occurrence of "my konfu is the best" with "I'm a linux jedi master" in all
*.txt files.

find . -name '*.txt' | xargs sed -i 's/test/teste/g'

Test if port 443 on a machine with IP address X.X.X.X is reachable.

telnet x.x.x.x 443

Get http://myinternal.webserver.local/test.html via telnet.

wget

How to send an email without a mail client, just on the command line?

using telnet

Write a get_prim method in python/perl/bash/pseudo.

Find all files which have been accessed within the last 30 days.

find . -atime +30

 Explain the following command (date ; ps -ef | awk '{print $1}' | sort |
uniq | wc -l ) >> Activity.log

date; ps -ef will get the actual date and all process that the user who runs the
command can access
awk '{ print $1}' will filter the first collunm before space
sort will sort, first numbers than characters
uniq will merge equal entries
wc -l will count the amount of lines and print the number
Activity.log will save this command appending in the file Activity.log

Write a script to list all the differences between two directories.

In a log file with contents as <TIME> : [MESSAGE] : [ERROR_NO] - Human readable

text display summary/count of specific error numbers that occurred every hour or a
specific hour.

cat log.log | grep $HOUR | uniq -c

####[⬆] Other Great References:

Some questions are 'borrowed' from other great references like:

https://github.com/darcyclarke/Front-end-Developer-Interview-Questions
https://github.com/kylejohnson/linux-sysadmin-interview-
questions/blob/master/test.md
http://slideshare.net/kavyasri790693/linux-admin-interview-questions

Releases

No releases published

Packages

No packages published