Operating Systems (OS)

Tasks and Layers of an OS

• OS as mediator between hardware and applications

• OS offers interfaces (API) for services

− Creation of processes
− Manage files
− I/O devices

Users
Utility Programs (shell, editor, …)
User Mode
Libraries (open, close, read, …)
Kernel Mode OS (Management of Memory, Processes, …)
Hardware (CPU, RAM, Disks, …)
[Tane]
Seite 23 Prof. Dr. A. Lehmann | IT Security
Memory Layout
Memory partitions for OS and programs

Problem: Job A
• Protect programs against each other
• Protect OS against programs Job B

Solution:
Job C
• Separate address space
− Kernel space, lower addresses
− User space, higher addresses
Operating
• Operating modes System
Memory
− User mode
Partitions
− System mode

[Tane]
Seite 24 Prof. Dr. A. Lehmann | IT Security
System and User Mode
System Mode
• Hardware access only through privileged commands

• OS service normally operate in system mode (special rights)

User Mode
• Applications are running in user mode

• Only unprivileged commands are available

• System calls are used for system services

• System calls will interrupt for change to system mode

[Tane]
Seite 25 Prof. Dr. A. Lehmann | IT Security
Device Driver
Device drivers are components of the OS
• Will be executed in privileged mode

• Booted into memory as an executable program

• Can be reloaded dynamically

Application
• Reliability? Virus?

User Driver User Mode

Device Driver Kernel Mode

[Tane]
Seite 26 Prof. Dr. A. Lehmann | IT Security
Virtual Address Space
• Each process owns a virtual address space (e.g. 4 GB)

• Abstracted from physical memory

• Virtual address space is segmented or paged

[Baun]

[Tane]
Seite 27 Prof. Dr. A. Lehmann | IT Security
Memory Allocation

4 GByte
local variables,
Stack register,
parameters
(stack grows)

global data and constants,

Heap malloc()
(heap may grow during runtime)
machine code
Code
protected
0 GByte

Stack: used to execute procedures and functions

[Tane]
Seite 28 Prof. Dr. A. Lehmann | IT Security
Dynamic and Static Linking
Static Libraries Application Application
• Statically linked into an executable A B
Static Static
Library Library

Dynamic Libraries Application Application

• Shared libraries A B

• Not physically linked into an executable

Shared
• Physically loaded into memory once Library

• Used by several processes

Seite 29 Prof. Dr. A. Lehmann | IT Security

Security Issues
• Procedure or system calls
− Storage place of return address and parameters on the stack

− Starting point for buffer overflow attacks

+ To inject code (e.g. Trojans) and run (e.g. with root rights)

− Infect a shared library with virus for common programs

int main()
{
char src[] = “itsecpasswd”;
char dest[7];
strcpy(dest, src);
return 0;
}
Buffer 8 bytes Overflow

i t s e c p a s s w d
[Ecke]
Seite 30 Prof. Dr. A. Lehmann | IT Security
Exploits and Defense
To exploit any type of buffer overflow the attacker needs
• To identify vulnerability in some program
• To understand how buffer will be stored in processes memory

Compile-time defence
• Choose high-level language (e.g. JAVA)
• Encourage safe coding standards
• Use safe libraries
• Additional code to detect corruption (Stackguard)

Runtime
• Executable address space protection
• Address space randomization

[Stal1]
Seite 31 Prof. Dr. A. Lehmann | IT Security
 Access Control and Rights
• Users are identified by system (authentication, password, username)

• User profile can specify permissible operations and file access

• OS enforces rules based on user profile

File System Access Control in Linux

• Index node (inode) contains links to addresses and blocks of file content

• Inode stores name of owner and rights

$ ls -lai /home
2 drwxr-xr-x 22 root root 4096 Sep 9 2016 .
2 drwxr-xr-x 22 root root 4096 Sep 9 2016 ..
262312 drwxr-xr-x 3 root root 4096 Sep 9 2016 home
2 d rwxr-xr-x 22 root root 4096 Sep 9 2016 .
inode file type rights link owner group size modification date name
counter [Stal1]
Seite 32 Prof. Dr. A. Lehmann | IT Security
Access Control in Linux

$ ls -l /home
drwxr-xr-x 3 root root 4096 Sep 9 2016 home

Position File
1 Type of file: d (directory) – (ordinary file) l (symbolic link)
2-4 Owner permissions: r (read) w (write) x (execute)
5-7 Permissions for other users in same group
8-10 Permissions for all users

Permission File Directory

read User can read content User can list files in directory
write User can modify content User can create and remove files
execute File is executable Directory can be accessed

Seite 33 Prof. Dr. A. Lehmann | IT Security

Special Rights
Permission Value Permission Value
Read 4 Set-UID 4
Write 2 Set-GID 2
Execute 1 Sticky 1

Permission chmod (octal) Symbolic

read, write and execute 7 rwx
read and write 6 rw-
read and execute 5 r-x
read only 4 r--
write and execute 3 -wx
write only 2 -w-
execute only 1 --x
No permissions 0 ---

Seite 34 Prof. Dr. A. Lehmann | IT Security

Special Rights

Symbol Meaning Description

s Set-UID Set User ID ensures that a program always runs with owner
privileges. (no meaning for directories)
s (S) Set-GID Set Group ID ensures that a program always runs with the
privileges of the primary group of the owner.
t (T) Sticky The sticky bit only has one feature. If it is applied to a folder, files
or directories created in it can only be deleted or renamed by the
file owner. This is used e.g. for /tmp

Seite 35 Prof. Dr. A. Lehmann | IT Security

 Access Control Lists (ACL)
Linux rights allow only distinction of access rights for owner, group and others

To give rights to one other person:

• New group has to be created
• Group only contains exactly the users who should get the rights
• For any combination of users a separate group is needed

ACL allow to set rights for the following parties:

• Exactly one owner
• Named users
• Exactly one owning group
• Named groups
• Others

$ setfacl -m u:USER:-,g:GROUP:RIGHTS,... FILE ...

Seite 36 Prof. Dr. A. Lehmann | IT Security

File/Data Encryption
Encryption and Decryption can be handled by different levels

• Application or user encryption

• Operating system encryption through e.g. mount encrypted partition

• Device encryption (hardware)

Application Application Application

OS OS OS

Hard Hard Hard

Drive Drive Drive

[Kapp]
Seite 37 Prof. Dr. A. Lehmann | IT Security