Evolution of the Internet and Its Measures
Bin Yang1,Yuliang Lu1,Kailong Zhu1,Ye Zhang1,Jingwei Liu1,2
1.Electronic Engineering Institute,Dept.NetworkHefei,China
2. Electronic System Engineering Company of China, Beijing 100079, China
Abstract—The Internet has being growing rapidly with the
development of technique. Understanding the evolution of
Internet is essential to the Internet security. there has been some
studies on the evolution of Internet, which only analyze the
change of the Internet during a quiet short period. In this paper,
we have roughly analyzed the evolution of the Internet for last 20
years and take days as the granularity of time. Some important
measures , such as average-distance and average-node-degree,
have been analyzed. Our findings show that the growth of
Internet is nearly binomial and the evolution of Internet can be
divided into 3 different phases. Moreover, these findings can be
used to model the evolution of Internet which would help us to
better protect network security.
Keywords—Internet; measurement; evolution; AS level
I. INTRODUCTION (HEADING 1)
With the development of technique, Internet has being
growing rapidly, having more than tenth in size in the past,
from 3000 Autonomous Systems in 1998 to 60000
Autonomous Systems in 2016.Since the fast changing of
Internet, it is necessary to have a clear understanding on the
evolution of Internet. On the other hand, as the progress of
attack technique, Internet is facing increasing challenges than
before. Hence, it is helpful to protect Internet security if we
could understand the characteristics and the evolution of
Internet.
During the past, there are many studies focus on the
analysis on the evolution of Internet[1-6].However, these
studies mainly analyze the evolution of Internet during a short
period. Since the RouteViews which collect the BGP data
have collecting data for over 20 years, we would like to
analyze the evolution of Internet according to these data to get
a better understood of Internet. And it is important for us to
know the evolution of Internet if we want to create the
evolution model for Internet.
In this paper, we attempt to measure and understand the
evolution of Internet ecosystem during the last twenty
years(1997-2016).We mainly focus on some important
characteristic of network, such as node degree, average AS
path. The growth of Internet is nearly binomial. Furthermore,
three phases of Internet evolution is concluded. Firstly, new
added nodes tend to link to the center of Internet. At the
second phase, new added nodes would like to connect to the
nodes located in the edge of Internet. Finally, Internet has
become stable for a long time.
The rest of this paper is organized as follows. In section
2,we would introduce the related work on the evolution of
Internet. The section 3 gives the source of data and the method
of processing. In section 4,the evolution of Internet is
discussed. The conclusion of this paper would be presented in
section 5.
II. RELATED WORKS
During past several years, many researchers has analyzed
the evolution of Internet. In history, the most cited literature is
literature [7].The authors firstly focus on the node degree of
AS, and conclude that the distribution of AS obey a power-law
distribution. Later, Zhou et al[8] further analyses the
distribution of degree, experimental results show that 0.1% of
nodes would take up over 80% of links, which means that the
node of Internet would like to connect to the nodes with large
degree. At the same time,Tier1 network is extracted from
Internet. CAIDA[9] emphasis on the difference of three
different data source, including whois, skitter and RIPE, and
analyses their characteristics. The results show that the data
from RIPE is more complete than others, and the
characteristics of whois is quite different with that of others. In
2008,Gill et al.[10] mainly focus on the behavior of
construction of text ISP results show that with the aim of
ensuring its safety and stable, text ISP tend to connect to
nodes with small degree. This phenomenon means that
Internet may be flat in the future. Reference [1] analyses the
size of Internet data collected from 1998 to 2007,and conclude
that the number of AS exhibits an exponential increase before
2001,and growing linearly later. However, literature [3]
shows that the number of AS is growing as exponentially.
III. DATA SOURCE
A study of the evolution of Internet needs a frequent
snapshots of the AS level Internet topology. Given that such
information is not available, we have to collect the data from
other data source. The datasets we use and the way of filtering
dirty data is introduced in this section.
We collect BGP data from BGP table dump obtained from
Route Views[11] for each day since November 1997.Note that
not all AS path is useful, since BGP data may be false
sometimes. The situation of false data can be divided into
three categories as follows:
1.Loops of Path. Loops of path means that one route has
been received by a AS twice, which would cause the cycling
of Internet traffic.e.g.AS path with 7652 8769 7652.
2.Containing of Bogus Internet resource. The resource of
Internet here mentioned mainly is the unallocated AS number
and the unallocated prefix.e.g.AS path with 458 785 65534 is
not useful in our analysis since the AS 65534,a private AS,
which is not available in the Internet.
3.Reduplicate AS path. There are many reduplicate AS
path, since the reduplication of AS in the path can restrict the
traffic translation, which would relief the traffic pass through
itself.e.g.AS path is 234 5631 667 667 899.
In order to get a nearly clean dataset for the analysis of the
Internet, we clear these dirty data from the collected BGP data.
Almost 5700 records are collected from November 8,1997 to Fig. 2. Correlation between AS number and interAS links
July 23,2016 for each day.
B. Evolution of node degree
IV. EXPREIMENTAL ANALYSIS In this section, we mainly analyze the evolution of node
degree in the Internet.Fig.3 shows the changing of the average
A. Grows and Trends of ASlevel Internet degree. As the figure shows, the changing of the average
The trends of AS level Internet mainly refer to the growing degree can be divided into three stages: firstly, the value of
of AS nodes and their connection between each other.Fig.1 average degree increase from November 1997 to 2003.The
shows that the number of ASes and the number of inter AS second phase is that the slowly decreasing of average degree
links in each snapshot. From Fig.1,a first observation is that, since 2003 until 2009.And at last, the node average degree
the increasing of the Internet is very fast during the past 20 increase to the largest value,5.It can be observed that the
years. Secondly, it appears that the growing trends of the AS growth of the firstly stage and the thirdly stage is similar.
number is the same as that of inter-AS links. And this situation
From the overall point of view, average degree of the
continues all the time. Then, the growing trends is regressed,
Internet is creasing all the way. It means that instead of
and the optimal fitting curves is switched to binomial for both
connecting to only one AS, more and more AS tend to connect
the number of nodes and edges.
to more ASs so that suddenly shutdown of some ASs may not
influence its normal work.

Fig. 3. Evolution of AS-level average degree

Moreover, we seek to analysis the evolution of node

centrality. In order to make a clear comparison in the node
centrality, we just choose a snapshot for each year. The
evolution of the distribution of node centrality can be shown
in Fig.4.
1998 1999 2000
0 2001 2002 2003
10 2004 2005 2006
Fig. 1. Growing trends of the AS numbers and interAS links 2007 2008 2009
-1 2010 2011 2012
10 2013 2014 2015
Furthermore, to find out the relationship between the edges
and AS number, we investigate the correlation between the AS 10
-2
P(D>x)

number and the edges. The results is shown in Fig.2.It is

clearly to know that the growth of the AS number and edges is 10
-3

switched to be linearly.
-4
10

-5
10
-4 -3 -2 -1
10 10 10 10
Node Centrality x

Fig. 4. Evolution of node centrality

 It is clearly to see that although the node centrality is
changing during the time passes, the distribution of node
centrality is almost stable in the passing years. Moreover, they
all obey the power-law distribution.
C. Evolution of cluster coefficient
In this section, we seek to understand the evolution of
cluster coefficient during the past 20 years. From Fig.5 a
interesting situation can be observed that the value of the
average cluster coefficient rises to the highest number 0.33
firstly and fall to 0.2 later .It means that the aggregation of the
AS level Internet is chasing for the compact, which each AS is
seek for the situation that ‘My friend's friend is a friend of
mine’,in several years. The reason causes the circumstance is
that the increasing number of stub ASs which only connect to
only one AS namely provider AS. The second reason is that
the rapidly increasing of AS’s number which would make it
difficult to reach the status named
‘My friend's friend is a friend of mine’.
D. Evolution of average distance
Average distance of the Internet gives the information
about how long one AS should take to reach another AS at
average. It mainly measure the efficiency of the Internet traffic
transtation.Fig.7 gives the evolution of the Internet average
distance.
It can be observed that the value of average distance is
float up and down, and the value of average distance is almost
between three and four. It is clearly to get the information that
the average distance has appeared to rising slowly since
2000.It means that it would take a longer distance to
translation traffic from one AS to another AS at average.

Fig. 7. Evolution of average distance

E. Evolution of assortativity coefficient

Assortativity coefficient is one of the most important
measure for a network. It is mainly used to estimate whether
new nodes prefer to link to node with large degree or no.
Fig. 5. Evolution of average cluster coefficient during the past 20 years Obviously, assortativity coefficient helps us know how
Internet is growing. The evolution of assortativity coefficient
To better understand the evolution of node cluster can be seen in Fig.8.
coefficient in the past years,Fig.6 gives the evolution of the
distribution of node cluster coefficient.
0.50
1998 1999 2000
0.45
2001 2002 2003
0.40 2004 2005 2006
2007 2008 2009
0.35 2010 2011 2012
2013 2014 2015
0.30
P(D>lc)

0.25

0.20 Fig. 8. Evolution of assortativity coefficient

0.15

0.10
We can see that the value of assortativity coefficient of the
Internet is negative all the time. It means that the nodes with
0.05
small degree prefer to link to the nodes with large degree than
0.00
0.0 0.2 0.4 0.6 0.8 1.0
the nodes with low degree. However, as shown in the figure,
Local Clustering lc
the value of assortativity coefficient is increasing slowly
through the whole time. This suggests that there are more low
Fig. 6. Evolution of node local cluster coefficient degree nodes prefer to connect the small degree nodes than
before, but the circumstance that Internet is disassortative
It is obviously that the distribution of node local cluster never changes all the time.
coefficient is stable as well as the distribution of node
centrality degree. But the difference is that the node local F. Evolution of standard network structure entropy
cluster is not strictly obey the power law distribution since that Standard network structure entropy(SNSE) is used to study
there is a suddenly decline in the distribution when the value the heterogeneity of the network. The evolution of SNSE is
of local clustering is 0.5. shown in Fig.7.It is obviously that the value of SNSE is
waving nearby a constant value,0.63,during the whole time.

Fig. 9. Evolution of SNSE
Furthermore, the increasing rate of the SNSE is learned as
Fig.8 shows. It is obviously to see that the increasing of SNSE
is very small, which means that the structure of the Internet is
stable.
Fig. 10. Evolution of increasing rate of SNSE
G. Summary of the evolution of Internet
Analysis on evolution of each characteristic is independent
above, this section want to give a summary of the evolution of
Internet. It is clearly to see that the distribution of node degree
and local cluster coefficient is mainly obey the power-law
distribution, and this phenomenon never changes. Also, the
evolution of increasing rate of SNSE shows that the structure
of Internet is always stable. The growth of the AS numbers
and inter-AS links is nearly binomial when taking days as the
granularity of time. It may give us a new way to predict the
number of AS and links. Also, it can help us create evolution
model of Internet which takes days as the granularity of time.
According to the analysis above, we think that Internet can
be divided into three phases. The first phase is before
2003.During the phase, the average degree of AS is increasing
as well as the average cluster coefficient. However, at the
same time, the average distance decreases. It means that the
new node may be added to the nearly center of Internet. The
second phase can be concluded from 2003 to 2009.During the
period, the trends of measures is different with the first phase.
Firstly, the value of average degree decrease as well as the
average cluster coefficient. The value of average distance
between each AS has increased since 2003 as well as the
assortativity coefficient of Internet. The reason may be the
increasing number of the stub network at the edge of the
Internet. At the third phase, the change of each measure has
become steady, which means that the Internet is becoming
more stable than before.
V. CONLUTION
During the past, there has been many studies focus on the
analysis of evolution of the Internet. However, the studies
mainly analyze the evolution of Internet during a short period.
In this paper, we measure the evolution of the AS-level
topology over the last 20 years in terms of the change of
measure. Our findings highlight some important trends: The
growth of the Internet is nearly binomial when taking days as
the granularity of time. Based on the evolution of some
important measure, the evolution of Internet can be divided
into three different phases, which each phase has its special
characteristics. With this findings, we could then simulate the
evolution of Internet, and model Internet evolution which
could be used to protect the Internet safety better.
REFERENCES
[1] Dhamdhere A, Dovrolis C. Ten years in the evolution of the Internet
ecosystem[C]//Proceedings of the 8th ACM SIGCOMM conference on
Internet measurement. ACM, 2008: 183196.
[2] Oliveira R V, Zhang B, Zhang L. Observing the evolution of Internet AS
topology[C]//ACM SIGCOMM Computer Communication Review.
ACM, 2007, 37(4): 313324.
[3] Zhang G Q, Zhang G Q, Yang Q F, et al. Evolution of the Internet and
its cores[J]. New Journal of Physics, 2008, 10(12): 123027.
[4] Dhamdhere A, Dovrolis C. Twelve years in the evolution of the Internet
ecosystem[J]. IEEE/ACM Transactions on Networking (ToN), 2011,
19(5): 14201433.
[5] Soffer S N, Vazquez A. Network clustering coefficient without
degreecorrelation biases[J]. Physical Review E, 2005, 71(5): 057101.
[6] Chang H, Willinger W. Difficulties measuring the Internet's ASlevel
ecosystem[C]//2006 40th Annual Conference on Information Sciences
and Systems. IEEE, 2006: 14791483.
[7] Faloutsos M, Faloutsos P, Faloutsos C. On powerlaw relationships of the
Internet topology[C]//ACM SIGCOMM computer communication
review. ACM, 1999, 29(4): 251262.
[8] Zhou S, Mondragón R J. The richclub phenomenon in the Internet
topology[J]. IEEE Communications Letters, 2004, 8(3): 180182.
[9] Mahadevan P, Krioukov D, Fomenkov M, et al. The Internet ASlevel
topology: three data sources and one definitive metric[J]. ACM
SIGCOMM Computer Communication Review, 2006, 36(1): 1726.
[10] Gill P, Arlitt M, Li Z, et al. The flattening Internet topology: Natural
evolution, unsightly barnacles or contrived collapse?[C]//International
Conference on Passive and Active Network Measurement. Springer
Berlin Heidelberg, 2008: 110.
[11] Route Views Project Page[OL].http://www.routeviews.org.2005.