You are on page 1of 4

Although AWS is known for its wide array of features and reliability, problems can still occur.

How do I troubleshoot TCP/IP connectivity problems in Linux?

Issue

I'm having trouble connecting to an Amazon EC2 Linux instance ?.

Short Description

Connectivity problems between nodes of a TCP/IP network are usually the result of incorrect routing
information caused by, but not limited to, the following:

Rogue DNS server(s) If a client requests information from an unauthorized DNS server that is
incorrectly configured, the information returned is not reliable.

Incorrect HOSTS file entries Most operating systems implement a HOSTS file where host-to-IP
mappings can be statically defined. On Linux operating systems this file is usually kept in /etc/hosts and
has the following format:

Where 1.2.3.4 is the physical IP address that will be used when referencing hostname.

Hardware failure If a DNS server has a hardware failure, the DNS records maintained by the DNS
server(s) are not updated as expected, potentially causing incorrect host-to-IP mappings to be sent to
clients.

Blocked TCP/IP ports If a port required by an application layer protocol or service is blocked, network
communications are blocked.

Incorrect client TCP/IP configuration If a client is not configured correctly, the client can have varying
degrees of connectivity problems. This can happen when a client is configured to use automatic TCP/IP
configuration and is unable to retrieve configuration information or receives incorrect configuration
information.

Resolution

Use the ping utility to verify connectivity between nodes on a TCP/IP network

The ping utility sends an Internet Control Message Protocol (ICMP) packet from a client to the specified
network name or IP address:

Many publicly accessible servers deprioritize incoming ICMP packets. As a result, it is not uncommon for
a ping request to return a 100% packet loss statistic. When initiating a ping request, it is important to
check the following:

Does the ping request resolve the name to an IP address? For example, in the preceding example, the
name www.amazon.com resolved to the IP address 54.239.19.16 even though no ping response was
returned. If a different host name had been used, say www.bamazon.com, name resolution would fail
and no ping requests would be attempted:
If the name resolves to an IP address, there is a good chance that the destination is accessible via
another application layer protocol or service. For example, even if a ping request to www.amazon.com
times out, connectivity to http://www.amazon.com typically succeeds because this destination is a web
server that accepts browser requests over port 80.

If the ping request is successful and a reply is returned, verify that replies were returned within a
reasonable time (<250 ms). Use the -c switch to specify the number of ping requests. If less than one
half of the specified ping requests are returned or if the average response time exceeds 250 ms,
additional troubleshooting steps may be required. To specify the timeout for a ping request in
milliseconds, use the -W switch. To continually ping the destination until stopped, use the -W switch
with a value of 0; for example:

sudo ping W 0 www.ubuntu.org

Press the Ctrl+C key combination to stop generating ping requests.

Does the address resolve to a hostname? To test address-to-hostname resolution, use the Linux
nslookup command and specify the IP address of the destination. For example, the following non-
interactive mode nslookup request tests address-to-hostname resolution for one of the IP addresses
associated with www.ubuntu.org:

ubuntu@ip-10-0-0-252:~$ nslookup 82.98.134.233

Server: 10.0.0.2

Address: 10.0.0.2#53

Non-authoritative answer:

233.134.98.82.in-addr.arpa name = hl231.dinaserver.com.

Note that the response returned the name hl231.dinaserver.com, which is one of the load-balanced web
servers handling requests for http://www.ubuntu.org.

Use the Linux traceroute utility to determine packet loss or routing problems

The Linux traceroute utility identifies the path that is taken from a client node to a specified destination
node and the time in milliseconds for each router identified in the path to respond to a request. By
default, the traceroute utility uses a default maximum value of 30 hops; if the path between the client
and destination requires more than 30 routers or hops, you can increase the maximum number of hops
with the -h switch. Because traceroute requests depend on responses to ICMP requests, some hops on
the route may drop requests in favor of higher-priority network traffic.

The following traceroute command was issued from an Amazon Web Services EC2 instance of Ubuntu
Linux 14.04. The arguments -T -p 80 -n perform a TCP-based trace on port 80 and return IP addresses
rather than hostnames. The Linux traceroute option to specify a TCP-based trace rather than relying on
ICMP is useful because most Internet devices deprioritize ICMP-based trace requests. The Linux
traceroute option to retrieve only IP addresses is useful when the hostnames that are returned are
wrong or misleading:

ubuntu@ip-10-0-0-252:~$ sudo traceroute -T -p 80 -n www.ubuntu.org

traceroute to www.ubuntu.org (82.98.134.233), 30 hops max, 60 byte packets

A few timed-out requests is typically not an issue to be concerned about. Instead, watch for packet loss
to the destination or in the last hop of the route. Packet loss that accumulates over several hops can
also indicate a problem.

Important: When troubleshooting network connectivity using the traceroute utility, it is often beneficial
to run the command in both directions, from the client to the server and then from the server back to
the client. If there are significant latency discrepancies in either path, consider reviewing the excellent
article Traceroute (http://cluepon.net/ras/traceroute.pdf), by Richard A Steenbergen. The author goes
into considerable depth describing the intricacies of proper trace analysis.

Use the Linux mtr command

The Linux mtr command provides continually updated output which allows you to analyze network
performance over time.

Note: If mtr is not installed on Ubuntu Linux, get it by running the following command:

sudo apt-get install mtr

Running the mtr command from Ubuntu Linux

For a list of mtr command-line options, run mtr -h

ubuntu@ip-10-0-0-252:~$ mtr www.ubuntu.org

mtr also provides a --report option that summarizes the results of sending 10 ping packets to each hop:

ubuntu@ip-10-0-0-252:~$ mtr --report www.ubuntu.org

Note: Because the path between nodes on a TCP/IP network can change when the direction is reversed,
it is important to obtain trace route results in both directions.