Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Standard view
Full view
of .
Look up keyword
Like this
0 of .
Results for:
No results containing your search query
P. 1
Anatomy of Upstream Intelligence

Anatomy of Upstream Intelligence

Ratings: (0)|Views: 9|Likes:
Published by John Wakefield

More info:

Published by: John Wakefield on Apr 16, 2011
Copyright:Attribution Non-commercial


Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less





Vol 13 No 3 Summer 2010
Anatomy of UpstreamIntelligence
(Article 2 of 3)b
y Tyson Macaulay
his article reviews the anatomy of Upstream Intelligence (UI) andsecurity. It provides a description of themajor elements and activities within acarrier or service-provider network thatgenerate UI. UI is not something that isdiscovered intact. It is often seeded fromdisjointed threat intelligence fragmentsthat evolve and grow in clarity throughthe combination and correlation of quantitative indicators (a more detaileddiscussion of this process will beavailable in article 4 of this series).UI may be seeded from open sourceinformation, closed-source information,or developed “from scratch.” Thescratch approach requires more effortand resources and is usually aby-product of an investigation intoactive, but unrecognized attacks andzero-day exploits.This article begins with adiscussion on the usual seed sources of UI, as well as the application of thenetwork elements that husband andnurture the seed base into usable UI.
Open-source threat intelligenceinformation is freely available on theInternet through groups with openmemberships or simply posted to websites. Lists of suspected “bad”Internet protocol (IPs) addresses(such as spammers, distributed denial-of-service [DDOS] attackers, nefariousdomain name system [DNS] servers, or web-hosting sites) are published by various security vendors, as well asunafliated/not-for-prot sitesdedicated to security, such as the Spamand Open Relay Blocking System(SORBS) or SpamHaus. [1, 2] Opensource intelligence also includes thesignatures and proles of knownmalware, available from a source likethe US Computer Emergency ReadinessTeam (US-CERT). [3] The quality of opensource security information is as diverseas the available suppliers. In the end, alot of excellent information is availableon an open source basis, but one thing can also be generally counted on—thebest and most up-to-date security andthreat information reaches open-sources last. In a world where threatscan change minute to minute, andsecurity posture changes at the samerate, open source information ranging inage from hours to days or weeks only begins to address the enterprise needsfor cyber threat intelligence.
Closed-source information is notpublicly available and is associated withinformation security operations,intelligence gathering, “softer” business,and professional relationships,particularly among carriers and serviceproviders, of which there areapproximately 1600 worldwide. [4]These carriers and service-providersshare intelligence about compromiseddevices and networks on a practical andsymbiotic basis at the engineering level,even while they may be harshcompetitors at the management level.Customer complaints are anotherform of closed-source information;persons or businesses attempting tocope with degraded network service willusually contact carrier or serviceprovider because they gure (wrongly)that the degradation they areexperiencing is related to a network problem. Such support calls frequently reveal severely compromised machines,much to the surprise of their owners
In a world where threats can change minute tominute, and security posture changes at the samerate, open source information ranging in age fromhours to days or weeks only begins to address theenterprise needs for cyber threat intelligence.
Article reprinted from IA Newsletter Volume 3, No. 3, Summer 2010 - a publication of Defense Technical Information Center, U.S. Department of Defense.http://iac.dtic.mil
Vol 13 No 3 Summer 2010
 who are frequently running some formof reputable anti-virus or intrusiondetection software.Information fusion among openand closed sources occurs now, withoutthe cost of complex information sharing methodologies or large teams in fortied24/7 operations centers. These aremostly unfunded systems using 
ad hoc 
or improvised tools, frequently withincarrier operations centers—the last lineof defense against cyber attacks. These
ad hoc 
tools and processes are effectivefor their stakeholders by providing aprimordial form of UI, by applying openand closed-source threat intelligence within network elements, such asswitches and routers. The discussion tofollow will seek to build up from theseinitial approaches for UI creation.
Cooking from Scratch
Rather than harvesting a bulk list of suspect IPs, domains and autonomoussystem numbers (ASNs) from open andclosed-sources, seed intelligence can be“cooked from scratch” through forensicprocesses where a degraded device isdiagnosed and traced to externalsources. Scratch sources often start witha single device exposing an externalmalicious entity, which underobservation at the enterprise oroptimally, the carrier-level, exposes itsrelationships with other malicious orcompromised entities. The typicalapproach would begin with theidentication of the device suspectedof compromise. The network communication patterns and protocolsof these devices are closely observed forrelatively simple criteria, such asoutbound destination, port andprotocols, and especially the timing andtrafc characteristics. Some of the mostpopularized UI investigations havestarted from scratch sources, such asthe recent GhostNet research. [5]
Network Elements
Open-source, closed-source, andscratch seed information needs to beaggregated, correlated, and combined with observations from various network elements to form UI and tools like “heatmaps” of compromised internal andexternal devices, as revealed by whatthey are doing on the Internet atlarge—not through signature basedle inspection. At a minimum, four majorinformation sources can be combined with seed information within carrierand service provider networks. Thisprocess generates much richerinformation about the activities,intentions, and operating modes of thecompromised devices and threat agents.These information sources are: trafcows, DNSs, messaging infrastructureand peer-to-peer (P2P) infrastructure.
Trafc Flows
Most, if not all, large carriers and serviceprovider networks will employ systemsfor monitoring the ow of trafc throughthe network junction points, bothinternally and at borders with otherproviders. A typical means of doing thisis with a proprietary, but widely supported protocol from Cisco calledNetFlow. [6] NetFlow allows providers tomaintain a picture of trafc ows andvolumes—basic tools for managing network quality and assurance. Thisinformation is also useful forunderstanding the threats posed by entities using the network for illicit andmalicious purposes. Basic informationsupported by NetFlow includes sourceIP address, destination IP address,source port, destination port, IPprotocol, ingress interface to thenetwork, and some information aboutthe type or quality of service associated with the trafc. NetFlow does notcapture packets or payloads, and is not acontent/media interception technology. Analysis on large carrier trafc ow statistics (
NetFlow) is like a satelliteview of road conditions—taking in anentire region or country at once with theability to zoom down to very granularactivities. Trafc ows can show ambiguous devices talking to suspiciousdestinations, and devices being scannedand probed from suspicious locations.However, trafc ow alone can beinconclusive because the Internet is
Vol 13 No 3 Summer 2010
made up of many independent carriersand service providers that do not sharetrafc ow data (for competitive andproprietary reasons) - thereforerendering observations incomplete.Figure 1 illustrates where trafcow data for UI might be derived from acarrier or large service provider network.If intelligence about a malicious orcompromised device or network can beseeded, centralized trafc ow analyticscan reveal the devices communicating  with the seeded IPs, domains, and ASNsand ag them as suspicious.
Trafc Flow Caveats
There are challenges to gathering trafcow information. For instance, logically,it is an expensive process because itburdens the routers. Trafc ow statistics gathered for typical operationalpurposes may only sample packets atrates of anywhere from 1:100 to 1:10,000.This provides sufcient information fornetwork management, but can alsoresult in lost or incomplete intelligence.Capturing trafc ow statistics on a 1:1basis (receiving information for roughly every packet) is not practical for mostoperations oriented infrastructure.Requiring a specialized security infrastructure. Similarly, many indicators from trafc ow analysis willbe inconclusive without examining theentire packet or data stream, a capability substantially beyond trafc ow analysisinfrastructure.
Domain Name Service
Domain name service (DNS) is one of theInternet’s most critical workhorses. It is apart of all IP infrastructure and essentialplumbing. DNS translates humanreadable addresses (Ex.
 into a machine readable and routableaddress (Ex. If DNS fails we all know about it very quickly becausemost or all IP-based communication willslow down or come to a stop. DNS is also akey infrastructure to threat agents whorely upon like everyone else, andfrequently seek to compromise it. DNSservice compromise can result in a wholesale fraud of dependent users. [7]DNS infrastructure in carrier andservice provider networks is large, andsupports millions of users and queries ata scale beyond most enterprises.Through this scale, DNS can providevaluable forms of UI, for instance: whichdevices have been compromised by malware, who is attempting to controlthe compromised devices, who islaunching attacks against specicassets, and where are they maliciously redirecting users (typically acompromised server). [8] Typically, the worst forms of malware encode a DNSname as the “call-home” command-and-control (C&C) address once a devicehas been compromised. Using a DNSname rather than an IP address providesthe botmaster (controller of themalware) with the ability to change C&Cservers to avoid detection and forredundancy. Awareness of DNS namesbeing used for C&C operations allowsDNS operators to set alerts whenever theC&C domain name is queried, and thencommence response operations. DNSrecords may reveal useful information,such as the IP address of the victim, themachine’s operating system, the timethe malware was installed, the variant of malware active, and of course the C&Caddress itself. Alternately, DNS lookupstatistics can reveal incongruousmatches between IP addresses anddomain names, or where a legitimate website has its users redirected tomalicious servers masquerading as alegitimate site (an attack form known as‘pharming’) in an effort to steal identity information and/or infect devices.Figure 2 illustrates DNSinfrastructure designs that providesubstantial UI through the queries madeby both consumers and businesses. Thisdiagram shows enterprises routing theirDNS queries through a carrier or serviceprovider, where logs can be aggregatedfor common benet—however, this isnot a mandatory design.
DNS Caveats
Gathering DNS intelligence is greatly facilitated by large, centralized DNSservices with large user bases. Whileconsumer based ISPs often have thisinfrastructure design in place, many enterprises do not. Instead, they have DNSservices scattered throughout network domains without centralized logs.Similarly, internal users might be pointing their computers to external DNS services,
Attack/Command and Control TrafficBenign Traffic
Malicious HostingNetworkPeer WPeer XPeer YPeer ZCompromisedServerTargeted Device/User Connect toMalicious NetworkCompromisedDevicePeer W
AS 666Enterprise WAN
Traffic Flow Analytics
Open/ClosedSource Seed IntelAlertsCore RouterBorder RoutersConsumer Access
Figure 1
Trafc Flow Intelligence

You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->