SW Vulnerabilities and Control Hijacking

Lecture 4 – Software Threats and
Vulnerabilities
CS Department
City University of Hong Kong
Slides partially adapted from lecture notes by M. Goodrich&R. Tamassia,

W. Stallings&L. Brown, Dan Boneh, and Dawn Song.
An Information Security Short Course
1
(Summer 2020)
Why Software Security?
• You may have a perfect design, a perfect
speciﬁcation, perfect algorithms, but still have
implementation vulnerabilities.
• After conﬁguration errors, implementation errors are

probably the largest single class of security errors
exploited in practice.
• Flaws in software break certain assumptions

important for security

2
(Summer 2020)
Operating Systems Concepts

3
(Summer 2020)
A Computer Model
• An operating system has to deal with the fact
that a computer is made up of a CPU, random
access memory (RAM), input/output (I/O)
devices, and long-term storage.
0
1
2
3
I/O CPU 4
5 RAM Disk Drive
6
7
8
4
An Information Security Short Course 9
(Summer 2020)
.
OS Concepts
• An operating system (OS) provides the interface
between the users of a computer and that
computer’s hardware.
– An operating system manages the ways applications access
the resources in a computer, including its disk drives, CPU,
main memory, input devices, output devices, and network
interfaces.
– An operating system manages multiple users.
– An operating system manages multiple programs.

5
(Summer 2020)
Multitasking
• Give each running program a “slice” of the CPU’s time.
• The CPU is running so fast that to any user it appears that the
computer is running all the programs simultaneously.

6
(Summer 2020)
The Kernel
• The kernel is the core component of
the operating system. It handles the
management of low-level hardware
resources, including memory, User Applications Userland
processors, and input/output (I/O)
devices, such as a keyboard, mouse, Non-essential OS
or video display. Applications
• Most operating systems define the Operating System
tasks associated with the kernel in The OS Kernel

terms of a layer metaphor, with the
hardware components, such as the CPU, Memory,
Hardware
Input/Output
CPU, memory, and input/output
devices being on the bottom, and
users and applications being on the
top.
7
(Summer 2020)
Input/Output
• The input/output devices of a computer include things
like its keyboard, mouse, video display, and network
card, as well as other more optional devices, like a
scanner, Wi-Fi interface, video camera, USB ports, etc.
• Each such device is represented in an operating system
using a device driver, which encapsulates the details of
how interaction with that device should be done.
– The application programmer interface (API), which the
device drivers present to application programs, allows those
programs to interact with those devices at a fairly high level,
while the operating system does the “heavy lifting” of
performing the low-level interactions that make such devices
actually work.
8
(Summer 2020)
System Calls
• User applications don’t communicate directly with
low-level hardware components, and instead
delegate such tasks to the kernel via system calls.
• System calls are usually contained in a collection of
programs, that is, a library such as the C library (libc),
and they provide an interface that allows applications
to use a predefined series of APIs that define the
functions for communicating with the kernel.
– Examples of system calls include those for performing file
I/O (open, close, read, write) and running application
programs (exec).
9
(Summer 2020)
Processes
• A process is an instance of a program
that is currently executing.
• The actual contents of all programs are
initially stored in persistent storage,
such as a hard drive.
• In order to be executed, a program
must be loaded into random-access
memory (RAM) and uniquely identified
as a process.
• In this way, multiple copies of the same
program can be run as different
processes.
– For example, we can have multiple copies of
MS Powerpoint open at the same time.
10
(Summer 2020)
Process IDs
• Each process running on a given computer is identified by a
unique nonnegative integer, called the process ID (PID).
• Given the PID for a process, we can then associate its CPU
time, memory usage, user ID (UID), program name, etc.

11
(Summer 2020)
Memory Management
• The RAM memory of a computer is its address space.
• It contains both the code for the running program, its
input data, and its working memory.
• For any running process, it is organized into different
segments, which keep the different parts of the
address space separate.
• As we will discuss, security concerns require that we
never mix up these different segments.

12
(Summer 2020)
Control Hijacking
Buffer-overflow and
Memory Safety

13
(Summer 2020)
An exploit is any input (i.e., a piece of software, an
argument string, or sequence of commands) that
takes advantage of a bug, glitch or vulnerability
in order to cause an attack

14
(Summer 2020)
Control hijacking
• Attacker’s goal:
– Take over target machine
• Execute arbitrary code on target by hijacking application
control flow
– Webserver à server target
– Email reader , web browser à client target
• Examples.
– Buffer overflow attacks
– Integer overflow attacks
– Format string vulnerabilities
– Race condition vulnerabilities
15
(Summer 2020)
Example 1: buffer overflows
• Extremely common bug in C/C++ programs.
– First major exploit: 1988 Internet Worm. fingerd. (a simple background
process on server)
# of vulnerable 600
software product
500 »20% of all vuln.
400 2005-2007: » 10%
300
200
100 Source: NVD/CVE

0
1995 1997 1999 2001 2003 2005
What is needed
• Understanding C functions, memory manager, the stack, the heap.
• Know how system calls are made
• The execve() system call
• Attacker needs to know which CPU and OS used on the target

machine:
– Our examples are for x86 running Linux or Windows
– Details vary slightly between CPUs and OSs:
• Little endian vs. big endian (x86 vs. Motorola)
– Decimal 1000 (hexadecimal 3E8) in two bytes
» big endian 03 E8
» little endian E8 03
• Stack Frame structure (Unix vs. Windows)
18
(Summer 2020)
structure that grows downwards
and keeps track of the activated
method calls, their arguments
Newly allocated variables and local variables
will stay below the previous
(existing) variables in stack.
e.g. libc
data dynamically generated

during the execution of a
process. e.g., malloc()
static program variables
initialized in the source code
prior to execution.
machine code of the program,

compiled from the source code 19
Stack Frame
Every time a function call is executed, a new frame is pushed onto the stack.
high
Arguments from the

arguments function being called
The address control

return address flow is supposed to
return, when the
stack frame pointer function terminates.
exception handlers e.g.: Try-catch block
local variables Stack

Growth
callee saved registers
SP low
20
(Summer 2020)
21
(Summer 2020)
out[i] = ‘\0’;

22
(Summer 2020)
What are buffer overflows?
Suppose a web server contains a function: void func(char *str) {
char buf[128];
When func() is called stack looks like: strcpy(buf, str);
do-something(buf);
}
argument: str
return address
stack frame pointer
char buf[128]
SP
23
(Summer 2020)
• One of the most common OS bugs is a buffer overflow
– The developer fails to include code that checks whether an input
string fits into its buffer array
– An input to the running process exceeds the length of the buffer
– The input string overwrites a portion of the memory of the process
– Causes the application to behave improperly and unexpectedly
• Effect of a buffer overflow

– The process can operate on malicious data or execute malicious code
passed in by the attacker
– If the process is executed as root, the malicious code will be
executing with root privileges
24
(Summer 2020)
What if *str is 136 bytes long? void func(char *str) {
After strcpy: char buf[128];
strcpy(buf, str);
do-something(buf);
}
argument: str
return address
stack frame pointer
*str Problem:
no length checking in strcpy()
char buf[128]
SP An Information Security Short Course

25
(Summer 2020)
Another Example with strcpy() Vulnerability
Top of
Memory
domain.c 0xFFFFFFFF
Main(int argc, char *argv[ ]) Stack
/* get user_input */ Fill
{ Direction
char var1[15];
char command[20];
strcpy(command, “whois "); var1 (15 char)
strcat(command, argv[1]);
strcpy(var1, argv[1]); command
printf(var1); (20 char)
system(command);
} ..
.
• Retrieves domain registration info Bottom of
• e.g., domain brown.edu Memory
An Information Security Short Course 0x00000000
26
(Summer 2020)
Another Example with strcpy() Vulnerability
domain.c Top of
Main(int argc, char *argv[]) Memory
/*get user_input*/ 0xFFFFFFFF Stack
{ Fill
char var1[15]; Direction
char command[20];
strcpy(command, “whois "); argv[1]
var1argv[1]
(15 char)
strcat(command, argv[1]); (15 char)
(20 char)
strcpy(var1, argv[1]); Overflow
printf(var1); command
exploit
system(command); (20 char)
}
..
• argv[1] is the user input .
• strcpy(dest, src) does not check buffer Bottom of
• strcat(d, s) concatenates strings Memory
0x00000000
27
(Summer 2020)
A Deeper View of Buffer Overflow

28
(Summer 2020)

29
(Summer 2020)

30
(Summer 2020)

31
(Summer 2020)

32
(Summer 2020)

33
(Summer 2020)

34
(Summer 2020)

35
(Summer 2020)

36
(Summer 2020)

37
(Summer 2020)

38
(Summer 2020)

39
(Summer 2020)
Buffer Overflow Attack in a Nutshell
• First described in
Aleph One. Smashing The Stack For Fun And Profit. e-zine
www.Phrack.org #49, 1996
• The attacker exploits an unchecked buffer to perform a
buffer overflow attack
• The ultimate goal for the attacker is getting a shell that
allows to execute arbitrary commands with high
privileges
– Vulnerability scenarios: the targeted victim program
has root privileges (setuid)
40
(Summer 2020)
Basic Stack Exploit
• Overwriting the return address allows an attacker
to redirect the flow of program control
• Instead of crashing, this can allow arbitrary code
to be executed
– Code segment is called “shellcode”
• Example, the execve system call is used to
execute a file
– With the correct permissions, execve(“/bin/sh”) can
be used to obtain a root level shell
41
(Summer 2020)
Basic stack exploit high
Suppose *str is such that Program P

after strcpy stack looks like:
Program P: execve(“/bin/sh”)
return address
When func() exits, the user gets shell !
Note: attack code P runs in stack.
char buf[128]
low
42
(Summer 2020)
Shellcode of execve
How to develop shellcode that runs as execve(“/bin/sh”)?
(exact shell code by Aleph One)

43
(Summer 2020)
44
(Summer 2020)
45
(Summer 2020)
46
(Summer 2020)
47
(Summer 2020)
48
(Summer 2020)
49
(Summer 2020)
50
(Summer 2020)
The NOP slide high
Problem: how does attacker

determine ret-address?
Solution: NOP slide

• Guess approximate stack state
when func() is called
– Install the program and perform random
fuzzy testing
• Insert many NOPs before program P:

nop ‘/x90’ , xor eax,eax , inc ax
low
51
(Summer 2020)
52
(Summer 2020)
More on Stack Smashing
• Some complications:
– Shellcode should not contain the ‘\0’ character.
– Overflow should not crash program before func()
exists.
• Sample remote stack smashing overflows:

– (2007) Overflow in Windows animated cursors (ANI).
LoadAniIcon()
– (2005) Overflow in Symantec Virus Detection
test.GetPrivateProfileString "file", [long string]

53
(Summer 2020)
Many unsafe libc functions
strcpy (char *dest, const char *src)
strcat (char *dest, const char *src)
gets (char *s)
scanf ( const char *format, … ) and many more.
• “Safe” libc versions strncpy(), strncat() are misleading
– Function strncpy() copies the string by specifying the number n of
characters to copy
• e.g., strncpy(dest, src, n); dest[n] = ‘\0’
– strncpy() may leave string unterminated.
• If source string is longer than the destination string, the overflow
characters are discarded automatically
– You have to place the null character manually
• Windows C run time (CRT):

– strcpy_s (*dest, DestSize, *src):
An Information Securityensures
Short Course proper termination
54
(Summer 2020)
Buffer overflow opportunities
• Exception handlers: (Windows SEH attacks)
– Overwrite the address of an exception handler in stack
frame.
• Function pointers: (e.g. PHP 4.0.2, MS MediaPlayer Bitmaps)

Heap
buf[128] FuncPtr or
stack
– Overflowing buf will override function pointer.
• Longjmp buffers: longjmp(pos) (e.g. Perl 5.003)

– Overflowing bufAnnext to pos overrides value of pos.
Information Security Short Course
55
(Summer 2020)
Corrupting Function Pointers in Heap
• Compiler generated function pointers (e.g. C++ code)
FP1
method #1
ptr FP2 method #2

FP3
method #3
data vtable
Object T
NOP shell
• After overflow of buf : slide code
data
buf[256] vtable
ptr
An Information Security Short Course 56
(Summer 2020) object T
General Control Hijacking:
shellcode
shellcode
shellcode
Overwrite Step: Overwrite entries in a vtable for Object T

Activate Step: Call any method from Object T
57
(Summer 2020)
General Control Hijacking:
Overwrite Step: Overwrite pointer to vtable on heap to point

to a crafted vtable.
Activate Step: Call any method from Object T (Summer 2020) 58
General Control Hijacking
Overwrite Step: Find some way to modify a Control Flow

Pointer to point to your shellcode, library entry point, or
Other code of interest..
Activate Step: Find some way to activate that modified
Control Flow Pointer
59
(Summer 2020)
Instances of Control Hijacking

60
(Summer 2020)
Finding buffer overflows
• To find overflow:
– Run web server on local machine
– Issue malformed requests (ending with “$$$$$” )
• Many automated tools exist (called fuzzers – next module)
– If web server crashes,
search core dump for “$$$$$” to find overflow location
• Construct exploit (not easy given latest defenses)

61
(Summer 2020)
Control Hijacking
More Control
Hijacking Attacks

62
(Summer 2020)
More Hijacking Opportunities
• Integer overflows: (e.g. MS DirectX MIDI Lib)
• Double free: double free space on heap.

– Can cause memory mgr to write data to specific location
– Examples: CVS server
• Format string vulnerabilities

63
(Summer 2020)
Integer Overflows (see Phrack 60)
Problem: what happens when int exceeds max value?
int m; (32 bits) short s; (16 bits) char c; (8 bits)
c = 0x80 + 0x80 = 128 + 128 ⇒ c=0

s = 0xff80 + 0x80 ⇒ s=0
m = 0xffffff80 + 0x80 ⇒ m=0
Can this be exploited?

64
(Summer 2020)
An example
void func( char *buf1, *buf2, unsigned int len1, len2) {
char temp[256];
if (len1 + len2 > 256) {return -1} // length check
memcpy(temp, buf1, len1); // cat buffers
memcpy(temp+len1, buf2, len2);
do-something(temp); // do stuff
}
What if len1 = 0x80, len2 = 0xffffff80 ?

⇒ len1+len2 = 0
Second memcpy() will overflow heap !!
65
(Summer 2020)
End of Segment

66
(Summer 2020)

SW Vulnerabilities and Control Hijacking

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

SW Vulnerabilities and Control Hijacking

Uploaded by

Copyright:

Available Formats

Lecture 4 – Software Threats and

Slides partially adapted from lecture notes by M. Goodrich&R. Tamassia,

• After conﬁguration errors, implementation errors are

• Flaws in software break certain assumptions

An Information Security Short Course

An Information Security Short Course

An Information Security Short Course

An Information Security Short Course

tasks associated with the kernel in The OS Kernel

An Information Security Short Course

An Information Security Short Course

An Information Security Short Course

An Information Security Short Course

100 Source: NVD/CVE

• Attacker needs to know which CPU and OS used on the target

data dynamically generated

machine code of the program,

Arguments from the

The address control

exception handlers e.g.: Try-catch block

local variables Stack

An Information Security Short Course

• Effect of a buffer overflow

SP An Information Security Short Course

An Information Security Short Course

An Information Security Short Course

An Information Security Short Course

An Information Security Short Course

An Information Security Short Course

An Information Security Short Course

An Information Security Short Course

An Information Security Short Course

An Information Security Short Course

An Information Security Short Course

An Information Security Short Course

An Information Security Short Course

Suppose *str is such that Program P

(exact shell code by Aleph One)

Problem: how does attacker

Solution: NOP slide

• Insert many NOPs before program P:

• Sample remote stack smashing overflows:

An Information Security Short Course

• Windows C run time (CRT):

• Function pointers: (e.g. PHP 4.0.2, MS MediaPlayer Bitmaps)

– Overflowing buf will override function pointer.

• Longjmp buffers: longjmp(pos) (e.g. Perl 5.003)

ptr FP2 method #2

Overwrite Step: Overwrite entries in a vtable for Object T

Overwrite Step: Overwrite pointer to vtable on heap to point

Overwrite Step: Find some way to modify a Control Flow

An Information Security Short Course

• Construct exploit (not easy given latest defenses)

An Information Security Short Course

An Information Security Short Course

• Integer overflows: (e.g. MS DirectX MIDI Lib)

• Double free: double free space on heap.

• Format string vulnerabilities

An Information Security Short Course

Problem: what happens when int exceeds max value?

int m; (32 bits) short s; (16 bits) char c; (8 bits)

c = 0x80 + 0x80 = 128 + 128 ⇒ c=0

Can this be exploited?