UPENET

ICT Security

Anatomy of An Intrusion

Miguel Sánchez-López

2006, Miguel Sánchez-López, under "Creative Commons Attribution-NonCommercial-NoDerivs 2.5" License,

<http://creativecommons.org/licenses/by-nc-sa/2.5/>

This paper was first published, in Spanish, by Novática (issue no. 178, Nov.-Dec. 2005, pp. 69–73). Novática, <http://www.ati.es/
UPENET, is a bimonthly journal published, in Spanish, by the Spanish CEPIS society ATI (Asociación
novatica>, a founding member of UP
de Técnicos de Informática – Association of Computer Professionals). This paper was a finalist of the I Novática Award for the best article
published in 2005.

This article describes vividly what happened when the author detected his computer at a university network had been
broken into. From detection to tracking the intruders down there was a long and winding path that included reporting the
incident to the authorities and, later, the prosecution of those responsible.

Keywords: Computer Networks,
Intrusion, Secure Communications,
Security Policies, Vulnerabilities.
1 The Environment of The In-
cident
The Computer Engineering Depart-
ment (DISCA) of the Polytechnic Uni-
versity of Valencia, Spain, is one of the
largest of that university with around
one hundred faculty members.
Our campus network is one of the
most populated campus networks in
Spain, with more than 24,000 personal
computers, using mostly one flavour
or another of Microsoft Windows.
Fewer people use any of a number
GNU/Linux distributions (Debian/
Ubuntu, SuSE, Fedora, Mandrake are
the most popular). The Computer Cen-
tre runs a variety of Unix and Windows
servers (HP, Sun and others). We also
have some Apple fans running either
OS9 or OSX on their white comput-
ers.
Campus network infrastructure has
been evolving since the early days
when we had just a single Ethernet seg-
ment. From an almost flat network
there is a massive use of subnetting,
but some segments with several thou-
sand computers still remain. Each versity staff uses. Some departments
building connects to the backbone us- may have their own system admins too.
ing either Fast Ethernet or Gigabit My department, as a Computer
Ethernet, though for some time FDDI Engineering unit, does most of the ad-
rings and ATM links were also used. ministration of departmental servers
Our campus network is connected to and personal computers. Services like
three other nodes of the national re- print servers, email server, departmen-
search network infrastructure (Red tal database and web servers are man-
IRIS) by means of 2.5Gbps links. A aged by our own technicians. Most fac-
fourth node is connected at 622 Mbps. ulty members manage their own per-
RedIRIS network provides Internet sonal computer but help is available
access to universities and other re- should any of them request it. Each
search institutions. faculty member is also responsible for
Each computer set-up and admin- keeping the required software licenses
istration depends on location and own- for the software they use, save for
ership. Common services like univer- Microsoft products for which there is
sity web servers and databases are run a special campus-wide agreement. Lec-
by the Computer Centre (ASIC) and turers are free to use whatever software
so are most of the computers the uni- they like.
Author
Miguel Sánchez-López holds a PhD in Computer Science from the Universitat
Politècnica de Valencia (UPV), Spain, where he got his BSc and Master degrees too.
He joined the Faculty of the Computer Engineering department in 1988 and he has
been teaching different topics dealing with Computer Networks ever since. He was a
visiting researcher at the Wireless Network Lab at the School of Electrical Engineering
at Cornell University, USA, in 1999 as well as at the International Computer Science
Institute at Berkeley, USA, from 2000 to 2001. He consults for some companies and
has written six books. His research focuses around ad-hoc networking and sensor
networks for energy conscious applications. <misan@upvnet.upv.es>

66 UPGRADE Vol. VII, No. 4, August 2006 © CEPIS


2 The Circle of Trust
My computing environment at the
university has been changing over the
years; from the early days when I
started using DOS 3.0, to a clean sweep
of Windows 3.1, 3.11, 95, 98, and
eventually Windows 2000. Then, after
a research stay at Cornell University
where I started using Solaris, I came
back to using RedHat and then SuSE
Linux (as the latter was being installed
in several of our labs).
At the end of the last century my
department bought a new email server
equipped with some email server soft-
ware by SuSE to power our email re-
quirements, offering users the possibil-
ity of using encrypted Secure Socket
Layer connections to the server, so our
mail traffic and user credentials would
not be easy to eavesdrop. Meanwhile,
the university email servers that only
allow regular (plain text) connections
that were easy to eavesdrop, were con-
sidered at the time to be a security prob-
lem that could lead to unauthorised
access to our mail messages.
That server worked flawlessly for
several years, with minor hiccups
whenever a user’s disk quota was ex-
ceeded. In fact, it worked so well that
software updates were almost forgot-
ten and because nobody experienced
any problem we were not aware that
we were at risk.
I am not a security expert, but as a
Computer Networks lecturer I was
aware of some of the risks involved
when you use a LAN. I always use a
SSH (Secure Shell) application to con-
nect to the various computer systems I
use either at home or on campus. I use
this same application for file transfers
too. Over the years I have been using
the same password for a set of systems
I considered ‘safe’ So my office com-
puter and other accounts I had on sev-
eral (but not all) campus computers had
the same user credentials. Whenever I
thought a given system was not reli-
able I used a different password.
3 Regular Checks
As even the best security may fail
sometimes, it is always advisable to
perform a check on your system’s logs.
Hopefully nothing will show up, but if
you catch any unexpected pattern it
© CEPIS
UPENET
may be time to take some immediate
corrective action.
That was the case on April 30, 2004
when after returning from my Easter
holidays I discovered some connec-
tions on my ssh logs dated April 5 that
I had not made. I knew something fishy
was going on because of the IP address
the connections were coming from: it
was from (rootshell.be), <http://www.
rootshell.be>, and I had never heard of
that before. After checking on their
web site I learned that Rootshell.be was
a company offering free shells over the
Internet. Whoever was using that
server to access my office computer
was doing so to hide their real iden-
tity.
4 Panic Attack
OK, so somebody did it: my com-
puter had been wide open for some
time but logs showed that somehow
the intruder knew my password. The
easy part was to change the password
to prevent future intruder accesses, but
at the same time I started wondering
how intruders had obtained my pass-
word and who they were.
I needed to check if the intruder had
installed any type of rootkit on my sys-
tem and whether they had gained ac-
cess to any of my other accounts on
other systems. Also, my problem might
be symptomatic of a bigger problem,
so I had to inform my colleagues of
the situation so they could also check
their computers and servers.
When I checked my home compu-
ter and several of the accounts on cam-
pus servers I realized several illegal
connections had been made from
<http://www.rootshell.be> as well as
from another unknown IP address.
Unfortunately, this second IP ad-
dress was not leaking any details about
its owner, as it happened to belong to
the address pool of the Spanish ISP
"Telefónica Transmisión de Datos" So
it was a computer of one of their cli-
ents, but they had more than one hun-
dred thousand!
I did not have admin privileges on
some of the affected servers, so I
needed to ask for the logs from those
systems’ admins. When I got the in-
formation back I realized that the in-
truders had been very busy, jumping
UPGRADE Vol. VII, No. 4, August 2006 67
from one system to another, always
using my user credentials. To make
things worse they had been quite tidy
about removing any clues of what they
had done on each system.
5 More Victims on Board
So, as I was putting all the logs to-
gether and making a time diagram of
all the accesses (which was not easy
as one server had its clock out by an
unknown amount of time) I realized the
first access to any of my accounts came
from our departmental email server.
This was a surprising discovery as
it signalled the beginning of the intru-
sion on my personal computer. Once
again I had to ask system admin to pull
my user account data from the logs and
this revealed even worse news: there
were some users that admin was not
previously aware of. When you factor
in the fact that you need to have admin
privileges to create new user accounts
it was clear that intruders had obtained
root access to the university email
server.
The server was immediately dis-
connected from the network and sev-
eral people started to dig for more de-
tails. It turned out that intruders had
installed software to sniff the network
and they had been collecting data to
huge files hidden in the file system.
Some of the accesses of those previ-
ously unknown users were coming
from another local cable-ISP called
ONO.
It was now clear that intruders had
broken into the university mail server
and then, by sniffing on that system,
they had been able to obtain user pass-
words. So it was no longer just my
problem.
6 Let The Quest Begin
So now we know that the intruders
gained access to our department’s
email server first. And once they had
access they used a local exploit to es-
calate privileges. Unfortunately, the
poor maintenance of the system made
this process quite easy.
Intruders did try some exploits on
all the other systems they access to my
account but they failed; in several cases
the exploits code did not even match
the version of the running kernel.

As soon as I learned about the
break-in (none too quickly I might say,
as we had two weeks of Easter holi-
days) I contacted abuse@rootshell.be
telling them about the problem that was
indeed contrary to their own accept-
able use policy. Throughout the proc-
ess, help from their sysadmin was cru-
cial for discovering who was respon-
sible.
When I sent the first abuse report
to rootshell.be I informed them about
my IP address and the date and time of
the illegal ssh connections. On the ba-
sis of this information, Rootshell’s
sysadmin learned what user account
the intruders were using and the ac-
count was closed immediately. We
were also provided with some of the
information that the intruders provided
when opening the account "userEve1 ",
such as the Hotmail account mail_
Eve@hotmail.com and, more impor-
tantly, the IP addresses they established
the last connections from. Unfortu-
nately these addresses belong to open
machines that are ... on campus! These
computers can be accessed by any stu-
dent and although users have to pro-
vide a valid user account, the intrud-
ers were using a stolen account and not
their own.
What is now clear is that the intrud-
ers are on campus, so they are likely to
be students, maybe even one of my stu-
dents. As my department gives lectures
for several courses it is not easy to nar-
row down the list of possible candi-
dates.
7 Paperwork
From the very beginning I told the
Computer Centre people about the in-
cident in an attempt to get them in-
volved as they have more tools and
reach than I do Unfortunately, they
have had other cases in the past and
they know it is difficult and time con-
suming to catch these people.
I also reported the incident to my
department head. As soon as we
learned that the email server was be-
1
Information about the intruder is not pro-
vided to protect their identities. Unlike
them, I do think other people’s rights
should be respected.
68 UPGRADE Vol. VII, No. 4, August 2006
UPENET
ing spied on, all department members
were informed and passwords were
changed.
However, there was an election
going on at the university at that mo-
ment and this incident did not seem to
be a priority. Even though unauthor-
ized access to electronic email is a
crime in Spain our lawyers decided not
to report the incident to the police.
8 The MUST
As I mentioned before, the intrud-
ers used an IP address belonging to an
ADSL line of a Spanish ISP. The ad-
dress returned no hits on a Google
search but two hits were found on
Altavista, leading to two postings on
two forums. From that I got a not very
common name, and hey, I even have a
student with that same unusual name,
but after a number of telephone calls I
was talking to the owner of the name
(who was not my student) and I asked
him what type of network access he
had at home but, unfortunately, it is not
from that ISP. But wait ... there is an-
other voice in the background. The stu-
dent tells me he is with the network
admin of his school, so I ask him to
put me through and then, bingo! the IP
address belongs to that school. The
Mediterranean University of Science
and Technology (or MUST) which is
situated on the same campus as us and
quite close to the location of the other
open systems we tracked down. It
seems that the pieces of our puzzle are
starting to fit together.
I ask the person on the phone for a
meeting with him and his boss and off
I go to meet them. I bring along a list
of my logs with times and dates of the
offending accesses. I find them very
helpful but powerless as they cannot
give me a list of names because they
keep no record of lab attendance. Also,
they have around forty computers con-
nected to the same ADSL line and they
have no traffic record so they cannot
help me now (but we will be able to
remedy this in the future).
9 Looking for Eve
When opening the account at
rootshell.be the intruder (or intruders)
provided an email account that they
had to have access to, since the regis-
tration process requires users to con-
firm the email account by clicking on
a special link mailed to their email ac-
count. So even though I do not know
who the intruder is, it seems I can email
her (please note I am deliberately
avoiding the use of the term hacker).
Clues point to her being one of our
thirty thousand students. Looking for
mail_Eve@hotmail.com provides no
entries. But, once again, Altavista leads
us to a forum on www.hackhispano. com
where there is a user with the same alias
mail_Eve who has 34 postings about her
activities of the last two years.
I can believe it: that person has been
creating chaos on several computer
rooms first in high school and now at
the university. And, interestingly
enough, the last two questions seem to
be related to our case and within the
right timeframe: brute forcing ssh and
where the ssh logs are stored in GNU/
Linux systems. Maybe she is starting
to think she might be leaking some info
and she wants to learn how to cover
her tracks.
When checking this user personal
information on the forum we learn that
she has 'obscured' her email by remov-
ing the last character so it appears as
mail_Eve@hotmail.co (instead of
.com) and, voilà, there is an ICQ
number too.
Now we go to ICQ website to pull
that user’s personal info, where she
appears as "Eve Simp", 19 years old
(which fits our profile), living in a zip
code of a small town near Valencia. So
she is a freshman with a penchant for
breaking into computer systems and
causing chaos while staying under
cover.
A few telephone calls to people in
the area with the same last name re-
turns no further information. To get this
moving, something else needs to be
done. For security reasons we discon-
nect our corporate email server over
the weekend, so the intruders have
no way of accessing any of the ac-
counts they had before, not even the
Rootshell’s one.
So I send an email to Eve’s hotmail
account telling her the game is up,
warning her that the incident has been
reported, and asking her to meet us next
Monday at our office.
© CEPIS

Ten minutes later I get an answer
from somebody who has no objection
to me calling her Eve and asks me if
this is some kind of joke. I note down
the IP address the message was an-
swered from and I am able to trace it
back to an ADSL operator of another
area of Spain (La Rioja). It seems than
even now Eve is taking care not to re-
veal her real location.
I reply to her right away informing
her that I am not joking and telling her
to talk to Rootshell’s sysadmin if she
is in any doubt. Nobody shows up on
Monday, which is a pity because we
still do not know who that person is. I
send a message to the ADSL line ISP
asking them to keep the log of that IP
in case the police need to see it in a
few days or weeks time.
What we know now is that the
hotmail account is still alive and that
Eve is still controlling it. Two days later
some of the personal information (such
as the zip code) is removed from Eve’s
ICQ personal data. So it seems she is
worried about any information that can
help us track her down. So now we
know her real zip code.
10 Desperately Seeking Trudy
Over the weekend our server has
been disconnected. The fake accounts
that the intruders created were can-
celled and all user passwords were
changed But we are still closely moni-
toring the system, so before shutting
the system down for the weekend we
realize that a new access attempt has
been made from Rootshell’s server. It
seems they are up to their old tricks
once again.
I contact Rootshell’s sysadmin
again to tell him the news, but this time
I ask him not to cancel the account but
to monitor what they do. Of course they
are using another free account and this
time the name is "userTrudy" and the
email is mail_Trudy@terra.es. As
Trudy detects that she cannot login to
our server but the server is alive she
realizes that something has changed,
so she starts deleting some files on her
Rootshell account: files with names
like k3ys.txt, users.txt and several us-
ers .known_hosts files. It seems Trudy
is doing a clean-up job but this time
we are watching carefully.
© CEPIS
UPENET
On Sunday there is a connection
attempt (through Rootshell) to the
email server (which is switched off)
from an IP belonging to the cable ISP
ONO.
The same IP has been appearing
consistently in several of our logs. The
other accesses to Trudy’s Rootshell
account are made from the MUST net-
work. It seems that Trudy’s home IP
may well be this one. It is time to start
monitoring the MUST network.
11 Fishing in The MUST
While the head of the MUST fac-
ulty is willing to help in the case, they
have no monitoring equipment in
place, so around forty computers are
freely sharing a single ADSL line. So
we deploy a simple network monitor
that will record all the incoming and
outgoing traffic over the ADSL line,
including the Ethernet MAC addresses.
We use Ethereal software.
A few days later, our trap is set and
nothing has happened, but on Friday I
get a call from MUST telling me they
have caught Trudy red-handed and,
what’s more, she is not aware of it.
MUST sysadmin is analysing Trudy’s
computer and has discovered multiple
connections to Rootshell and to a cou-
ple of ONO’s IP addresses, one of them
matching the one we mentioned ear-
lier.
12 Unexpected Events
Knowing that the trail is fading fast
and given that the university was not
interested in filing a police report, I
decided to do so off my own bat, partly
because I did not know what those peo-
ple had been doing with my system and
I did not want to be prosecuted later
for a crime I did not do.
A special team of police travelled
from Madrid to Valencia to gather the
evidence and on the same trip they also
received an unrelated report of another
break-in at a computer at MUST.
One of the police team members
suggests that I search for mail_
Eve@hotmail.com on Google again.
It seems they have done their home-
work and yes, this time there is an
entry on Google for that email that
belongs to programming work a stu-
dent turned in at my university.
UPGRADE Vol. VII, No. 4, August 2006 69
Eve’s last name is not Simp but
Simpson and she is a he. His address,
phone number and zip code does ap-
pear in this assignment, and of course
the zip code matches the one we al-
ready knew.
I talk to the professor responsible
for that subject and he explains to me
that Eve is not one of his students but
the brother of one of his students. How-
ever, it seems that Eve was so proud of
his programming skills that he turned
his assignment in to be published in
December 2003. However, the pro-
gram was not posted on the subject’s
website till May 2004.
Eve is a first-year Telecom student,
and when I contacted the Telecom
school administrator he told me that
Eve had a long list of similar incidents.
The Computer Centre person in
charge of network security is informed
about my findings. The next day he
calls me back, surprised, as he has
caught the same person up to no good
at the University Library. Eve was
caught while using a stolen account
with administrator privileges. This time
Eve was caught red-handed by secu-
rity and was asked to produce a photo
ID so now he will have a hard time try-
ing to get away with it.
13 Following The Trail
While I was still going through the
our mail server logs I realized that one
teacher was connecting on March 22
from the same open systems lab that
the intruders were sometimes connect-
ing to Rootshell from. I contacted this
colleague right away as I needed him
to confirm or deny whether it was re-
ally him accessing from that location,
and it turns out he had never been at
that lab, so I have finally detected the
first step the intruders made to get into
our server.
It turns out that Eve was one of my
colleague’s students and at the time of
the intrusion he was using the class com-
puter which had Windows 98 installed
to access his office computer. Windows
98 stores a user password encrypted in a
.pwl file.
There are several programs that can
easily decrypt these password files. My
colleague was using the same password
for his email account and ssh daemon

was enabled on the email server, so that
is how they got in.
Once they successfully escalated
privileges on the server and installed
the sniffer software they could learn the
passwords of all the other professors.
They have an easy way to get the large
files containing capture data by using
a USB hard disk (or USB flash dongle)
as the campus network is quite fast for
moving files around in When the op-
eration was detected there was a
200MB capture file containing thou-
sands of private files.
I should remind the reader at this
point that this intrusion was possible
because users (including myself) were
using the same password for several
systems.
14 Regaining Trust
Most of my colleagues using
Microsoft Windows were not able to
check whether or not their computers
had been broken into as auditing func-
tions were not enabled. Other people
using GNU/Linux did not detect any
connection attempts from the email
server.
I had evidence of some failed con-
nection attempts to a second computer
I had in my office that had a database
of student grades on it (and a different
password). What the purpose of the
break-in is still a mystery to me, but it
is interesting that intruders were so
fond of my accounts on various com-
puters.
My second office computer web
server had been bombarded with a
number of exploits for a time period
matching the break-in. As this compu-
ter was only accessible from within the
campus network, attackers used more
than thirty different computers they had
compromised before. A couple of these
computers were from my department
too. Attackers failed completely in this
attack, but they kept on trying for a
couple of weeks.
People in my department were
warned about the break-in from the
beginning and they were all asked to
keep an eye on any bank account they
may have electronic access to. Up un-
til now it seems that no crime of this
nature has been detected.
As a result of these events, our de-
partment board decided to close down
70 UPGRADE Vol. VII, No. 4, August 2006
UPENET
our email server and to transfer all the
accounts to the main university-wide
email server (that now allows SSL con-
nections too).
Some of the changes we have made
I have already mention that our
departmental email server has been
replaced by moving the accounts to a
larger and (hopefully) better main-
tained university-wide email server. It
is ironic that the server we are now
doing away with was put in place be-
cause in the late nineties the univer-
sity server was not considered secure
at the time. It surprises me how a few
years later this same server was the
vehicle of an intrusion.
I had to talk to a great many people
during this research and it amazes me
that many of them are of the opinion
that "I’m not worried because I have
no valuable information on my com-
puter" Apparently some users are not
worried about other people spying on
them or using their computer as a ve-
hicle for conducting illicit actions such
as breaking into other computers or
storing illegal contents.
This incident has brought to light
the weaknesses of our systems and our
total lack of any security policy. This
is a shortcoming that, two years on, we
have yet to remedy. We all tend to use
insecure short cuts too often and
scarcely give a thought to the fact that
we might be targeted.
I have decided the time for using
passwords has passed and I have
changed to public key authentication
for all my accounts.
15 Conclusions
Campus networks seem to have
several interesting features many in-
truders crave:
� High-speed Internet access.
� Thousands of computers linked
together by a fast LAN.
� Academic information, such as
exams and students’ grades.
All these features, together with the
scant interest that the average user
tends to have in security issues affect-
ing his or her own personal computer,
make these networks an easy target for
intruders.
Whenever a break-in is detected the
worst is yet to come. A great many
man-hours are consumed in the proc-
ess of putting things right. Some lit-
erature suggests that detected intru-
sions may amount to around 4% of the
total number of intrusions. (This is a
tricky concept since in order to give a
percentage you need to know the total
number it is a percentage of, which in
this case is impossible).
If your organization has no secu-
rity policy then it is quite likely that
your computer systems are not prop-
erly protected and that any intruders
you may have will not be prosecuted
effectively.
A Legal Postscript
One month after the police report
was filed the university received a
court order to hand over the hard drives
of the compromised servers.
Several IPSs were required by the
judge to provide the connection logs
of several IP addresses belonging to
their clients.
And both Microsoft Hotmail and
Terra Networks were asked to provide
the details of the owners of the ac-
counts mail_Eve@hotmail.com and
mail_Trudy@terra.es.
As a result, the identities of the in-
truders were revealed and the judge
ordered the homes of these people to
be searched by the police. The intrud-
ers were arrested and all their comput-
ing equipment was confiscated. After
being questioned in court they were
released with charges.
A year and a half later there was a
trial. Trudy hired a famous (and expen-
sive) lawyer. The two intruders were
found guilty of the break-in but the
judge let them off on a technicality.
However, the prosecutor appealed and
they were later sentenced to a fine of
3,600 euros each.
Translation by the author
© CEPIS