Network Packet Capture in Linux Kernelspace

Introduction
Network stack
Packet ingress flow
Methods to capture packets
University of Sao Paulo - USP
Network packet capture in Linux kernelspace

An overview of the network stack in the Linux kernel
Beraldo Leal
beraldo@ime.usp.br
http://www.ime.usp.br/~beraldo/
Institute of Mathematics and Statistics - IME
25th October 2011
Beraldo Leal
25th October 2011
1 / 25
Introduction
Network stack
Packet ingress flow
Outline
Introduction
Network stack
Packet ingress flow
Beraldo Leal
25th October 2011
2 / 25
Introduction
Network stack
Packet ingress flow
Introduction
Sniffers;
Improvements in packet reception;
Linux kernel network subsystem;
Beraldo Leal
25th October 2011
3 / 25
Introduction
Network stack
Packet ingress flow
Sniffers
tcpdump, wireshark, snort, etc;

Using the well-known library libpcap;
Not suitable for > 10 Gbps;
Packet loss;
Beraldo Leal
25th October 2011
4 / 25
Introduction
Network stack
Packet ingress flow
Improvements in packet reception
Commodity hardware for packet capture;

3COM
Intel
endace, ...
Many Interruptions
NEW API or NAPI (interruption coalescence)
zero-copy
Direct Memory Access - DMA
mmap()
Beraldo Leal
25th October 2011
5 / 25
Introduction
Network stack
Packet ingress flow
Linux kernel network subsystem
Kernel number of files: 36.680
1 2
net/ number of files: 1.293 ( 3.5% )

drivers/net/ number of files: 1.935 ( 5.27% )
Kernel SLOC: 9.723.525
net/ SLOC: 480.928 ( 5% )
drivers/net/ SLOC: 1.155.317 ( 12% )
1
2
kernel 3.0.0
source: wc, find, cat, etc..
Beraldo Leal
25th October 2011
6 / 25
Introduction
Network stack
Packet ingress flow
Network stack
L5: Application
http, ftp, ssh, telnet, ... (message)
L4: Transport
tcp, udp, ... (segment)
L3: Network
ipv4, ipv6, ... (datagram/packet)
L1/2: Link / host-to-network

ethernet, token ring, ... (frame)
Beraldo Leal
25th October 2011
8 / 25
Introduction
Network stack
Packet ingress flow
Important data structs:
net device
include/linux/netdevice.h
sk buff
include/linux/skbuff.h
Beraldo Leal
25th October 2011
9 / 25
Introduction
Network stack
Packet ingress flow
net device (include/linux/netdevice.h)

unsigned int mtu
unsigned int flags
unsigned char dev addr[MAX ADDR LEN]
int promiscuity
Beraldo Leal
25th October 2011
10 / 25
Introduction
Network stack
Packet ingress flow

sk buff (include/linux/skbuff.h)
struct sk buff *next;
struct sk buff *prev;
ktime t tstamp;
struct net device *dev;
unsigned int len;
unsigned int data len;
u16 mac len;
u8 pkt type;
be16 protocol;
sk buff data t transport header; (old h)
sk buff data t network header; (old nh)
sk buff data t mac header; (old mac)
Beraldo Leal
25th October 2011
11 / 25
Introduction
Network stack
Packet ingress flow
Important sk buff routines
alloc skb();
dev alloc skb();
kfree skb();
dev kfree skb();
skb clone();
skb network header(skb);
skb transport header(skb);
skb mac header(skb);
Beraldo Leal
25th October 2011
12 / 25
Introduction
Network stack
Packet ingress flow
Packet ingress flow

When working in interrupt driven model, the nic registers an
interrupt handler;
This interrupt handler will be called when a frame is received;
Typically in the handler, we allocate sk buff by calling
dev alloc skb();
Copies data from nics buffer to this struct just created;
nic call generic reception routine netif rx();
netif rx() put frame in per cpu queue;
if queue is full, drop!
net rx action() decision based on skb->protocol;
This function basically dequeues the frame and delivery a copy
for every protocol handler;
ptype all and ptype base queues
Beraldo Leal
25th October 2011
13 / 25
Introduction
Network stack
Packet ingress flow
Packet ingress flow
ip v4 rcv() will receive the ip datagram (if is a ipv4 packet);

ip checksum, check ip headers, ....
ip rcv finish() makes route decision (ip forward() or
ip local delivery())
ip local delivery() defrag fragmented packets, and call
ip local deliver finish()

ip local deliver finish() find protocol handler again;
tcp v4 rcv(), udp rcv(), or other L4 protocol handler
...
Beraldo Leal
25th October 2011
14 / 25
ip_local_deliver_nish()
(net/ipv4/ip_input.c)
nd protocol handler or
send icmp_dst_unreach
NF_IP_LOCAL_IN
NF_IP_PRE_ROUTING
<continue>
NF_IP_FORWARD
Layer 3
Network
ip_local_deliver()
defrag fragmented
packets
ip_rcv_nish()
nd route and handle
IP options
ip_forward()
(net/ipv4/ip_forward.c)
handle route alert;
send redirect if
necessary;
decrease TTL;
verify if frag is
possible (mtu)
ip_error()
(net/ipv4/route.c)
routing error, send
icmp pkt
ip_rcv()
packet_rcv()
arp_rcv()
<tcpdump_process>
(handle arp requests
verify skb, IP headers
<dhcpd process>
and replies)
and IP checksum
<...>
netif_rx()
(net/core/dev.c)
input_queue
[cpu]
Network Drivers
(drivers/net/*)
<...>
net_rx_action()
(net/core/dev.c)
decision based on
skb->protocol eld
Layer 1/2
Physical/Link
Application
userspace
kernelspace
Socket Layer
(net/core/sock.c)
__tcp_v4_lookup()
(net/ipv4/tcp_ipv4.c)
check for socket in
LISTEN, with dst_port
tcp_v4_do_rcv()
check for socket state
tcp_v4_rcv()
check for tcp headers
udp_rcv()
(net/ipv4/udp.c)
check for udp headers
ip_local_deliver_nish()
nd protocol handler or
send icmp_dst_unreach
NF_IP_LOCAL_IN
NF_IP_PRE_ROUTING
generate ICMP
error
<...>
Layer 4
Transport
<continue>
NF_IP_FORWARD
Layer 3
Network
ip_local_deliver()
defrag fragmented
packets
ip_rcv_nish()
nd route and handle
IP options
ip_forward()
(net/ipv4/ip_forward.c)
handle route alert;
send redirect if
necessary;
decrease TTL;
verify if frag is
possible (mtu)
ip_error()
(net/ipv4/route.c)
routing error, send
icmp pkt
Introduction
Network stack
Packet ingress flow
protocol handler
register a function to handler packets with dev add pack()
netfilter hooks
userspace tools;
socket AF PACKET, libpcap, ...
Beraldo Leal
25th October 2011
17 / 25
Introduction
Network stack
Packet ingress flow
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
struct packet type my proto;

int my packet rcv(struct sk buff skb, struct net device dev,
struct packet type pt, struct net device orig dev) {
printk(KERN ERR + 1!\n);
kfree skb(skb);
return 0;
}
static int hello init(void) {
printk(<1> Hello world!\n);
my proto.type = htons(ETH P ALL);
my proto.dev = NULL;
my proto.func = my packet rcv;
dev add pack(&my proto);
return 0;
}
static void hello exit(void) {
dev remove pack(&my proto);
printk(<1> Bye, cruel world\n);
}
module init(hello init);
module exit(hello exit);
Beraldo Leal
25th October 2011
18 / 25
Introduction
Network stack
Packet ingress flow
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
int my packet rcv(struct sk buff skb, struct net device dev, struct packet type pt, struct net device
orig dev)
{
switch (skb>pkt type) {
case PACKET HOST:
printk(PACKET HOST );
break;
case PACKET BROADCAST:
printk(PACKET BROADCAST );
break;
case PACKET MULTICAST:
printk(PACKET MULTICAST );
break;
case PACKET OTHERHOST:
printk(PACKET OTHERHOST );
break;
case PACKET OUTGOING:
printk(PACKET OUTGOING );
break;
case PACKET LOOPBACK:
printk(PACKET LOOPBACK );
break;
case PACKET FASTROUTE:
printk(PACKET FASTROUTE );
break;
}
printk(%s 0x%.4X 0x%.4X \n, skb>dev>name, ntohs(skb>protocol), ip hdr(skb)>protocol)
kfree skb(skb);
return 0;
}
Beraldo Leal
25th October 2011
19 / 25
Introduction
Network stack
Packet ingress flow
Netfilter hooks
iptables = userspace;
netfilter = kernelspace;
Netfilter is merely a series of hooks in various points in a
protocol stack;
packet filtering, network address [and port] translation
(NA[P]T) and other packet mangling;

www.netfilter.org
Beraldo Leal
25th October 2011
20 / 25
Introduction
Network stack
Packet ingress flow
References
br.kernelnewbies.org/node/150 has many links
Beraldo Leal
25th October 2011
23 / 25
Introduction
Network stack
Packet ingress flow
Thankyou! Question?
Beraldo Leal
25th October 2011
24 / 25
Introduction
Network stack
Packet ingress flow

An overview of the network stack in the Linux kernel
Beraldo Leal
beraldo@ime.usp.br
http://www.ime.usp.br/~beraldo/
Institute of Mathematics and Statistics - IME
25th October 2011
Beraldo Leal
25th October 2011
25 / 25

Network Packet Capture in Linux Kernelspace

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Network Packet Capture in Linux Kernelspace

Uploaded by

Copyright:

Available Formats

Introduction

University of Sao Paulo - USP