Professional Documents
Culture Documents
Textbook Big Data and Computational Intelligence in Networking 1St Edition Yulei Wu Ebook All Chapter PDF
Textbook Big Data and Computational Intelligence in Networking 1St Edition Yulei Wu Ebook All Chapter PDF
https://textbookfull.com/product/computational-intelligence-
applications-in-business-intelligence-and-big-data-analytics-1st-
edition-vijayan-sugumaran/
https://textbookfull.com/product/data-science-and-big-data-an-
environment-of-computational-intelligence-1st-edition-witold-
pedrycz/
https://textbookfull.com/product/5g-enabled-internet-of-
things-1st-edition-yulei-wu/
https://textbookfull.com/product/machine-intelligence-and-big-
data-in-industry-1st-edition-dominik-ryzko/
Computational Intelligence in Data Mining Himansu
Sekhar Behera
https://textbookfull.com/product/computational-intelligence-in-
data-mining-himansu-sekhar-behera/
https://textbookfull.com/product/artificial-intelligence-for-big-
data-complete-guide-to-automating-big-data-solutions-using-
artificial-intelligence-techniques-anand-deshpande/
https://textbookfull.com/product/ai-and-big-datas-potential-for-
disruptive-innovation-advances-in-computational-intelligence-and-
robotics-1st-edition-moses-strydom/
https://textbookfull.com/product/artificial-intelligence-for-
fashion-industry-in-the-big-data-era-sebastien-thomassey/
https://textbookfull.com/product/microbial-data-intelligence-and-
computational-techniques-for-sustainable-computing-1st-edition-
aditya-khamparia/
Big Data and
Computational Intelligence
in Networking
This book contains information obtained from authentic and highly regarded sources. Reasonable
efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and
publishers have attempted to trace the copyright holders of all material reproduced in this publication
and apologize to copyright holders if permission to publish in this form has not been obtained. If any
copyright material has not been acknowledged please write and let us know so we may rectify in any
future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information
storage or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.
copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC),
222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that
provides licenses and registration for a variety of users. For organizations that have been granted a
photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and
are used only for identification and explanation without intent to infringe.
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515
Recent years have witnessed a deluge of network data propelled by the emerging
online social media, user-generated video contents, and global-scale communi-
cations, bringing people into the era of big data. Such network big data holds
much critical and valuable information including customer experiences, user
behaviors, service levels, and other contents, which could significantly improve
the efficiency, effectiveness, and intelligence on the optimization of the current
Internet, facilitate the smart network operation and management, and help service
providers and content providers reduce capital expenditure (CapEx) and opera-
tional expenditure (OpEx) while maintaining a relatively high-level quality of
service (QoS) and quality of experience (QoE).
Typical examples of network intelligence received from network big data
include rapid QoE impairment detection and mitigation, optimization of network
asset utilization, proactive maintenance, rapid outage restoration, and graceful
disaster recovery. These aims can be achieved from high-level computational
intelligence based on emerging analytical techniques such as big data pro-
cessing, Web analytics, and network analytics employing software tools from
advanced analytics disciplines such as machine learning, data mining, and pre-
dictive analytics. The computational intelligence for big data analysis is playing
an ever-increasingly important role in supporting the evolution of the current
Internet toward the next-generation intelligent Internet.
However, the unstructured, heterogeneous, sheer volume and complex nature
of network big data pose great challenges on the computational intelligence
of these emerging analytical techniques due to high computational overhead
and communication cost, non-real-time response, sparse matrix-vector multi-
plications, and high convergence time. It is therefore of critical importance
to understand network big data and design novel solutions of computational
intelligence, scaling up for big data analytics of large-scale networks to auto-
matically discover the hidden and valuable information available for smart
network operations, management, and optimization. This has been established as
ix
of network big data, efficient collection and management of network big data,
distributed and scalable online analytics for network big data, and emerging
applications of network big data for computational intelligence.
Targeted audiences: This book targets both academia and industry readers. Grad-
uate students can select promising research topics from this book that are suitable
for their thesis or dissertation research. Researchers will have a deep under-
standing of the challenging issues and opportunities of network big data and
can thus easily find an unsolved research problem to pursue. Industry engineers
from IT companies, service providers, content providers, network operators, and
equipment manufacturers can get to know the engineering design issues and cor-
responding solutions after reading some practical schemes described in some
chapters.
We have required all chapter authors to provide as much technical detail as
possible. Each chapter also includes references for readers’ further studies and
investigations. If you have any comments or questions on certain chapters, please
contact the chapter authors for more information.
Thank you for reading this book. We wish that this book will help you with
the scientific research and practical problems of network big data.
xiii
Zhe Jiang Yu Lu
Department of Computer Science Electrical and Computer Engineering
University of Alabama University of Alabama
Tuscaloosa, Alabama Tuscaloosa, Alabama
Vasileios Karyotis
Xiaoqiang Ma
School of Electrical and Computer
School of Electronic Information and
Engineering
Communications
National Technical University of
Huazhong University of Science and
Athens (NTUA)
Technology
Athens, Greece
Wuhan, China
Samee U. Khan
North Dakota State University Immanuel Manohar
Fargo, North Dakota Electrical and Computer Engineering
University of Alabama
Joonseok Lee Tuscaloosa, Alabama
Google Research
Video Understanding Team, Machine
Norashidah Md. Din
Perception
Universiti Tenaga Nasional
Mountain View, California
Selangor, Malaysia
Seunghak Lee
Human Longevity, Inc., Geyong Min
San Diego, California University of Exeter
Exeter, United Kingdom
Weidong Li
Department of Geography & Center
Ion Necoara
for Environmental Sciences and
Automatic Control and Systems
Engineering
Engineering Department
University of Connecticut
University Politehnica Bucharest
Storrs, Connecticut
Bucharest, Romania
Nan Lin
Department of Mathematics Guo Ren
Washington University in St. Louis China University of Geosciences
St. Louis, Missouri Wuhan, China
Jiangchuan Liu
Shashi Shekhar
School of Computing Science
Department of Computer Science
Simon Fraser University
University of Minnesota
Burnaby, British Columbia, Canada
Minneapolis, Minnesota
Z. Y. Liu
Department of Mechanical Mukesh Singhal
Engineering Cloud Lab
University of Alabama University of California, Merced
Tuscaloosa, Alabama Merced, California
Aidong Zhang
Department of Computer Science and
Engineering
State University of New York
Buffalo, New York
Albert Zomaya
The University of Sydney
Sydney, Australia
Fei Hu
University of Alabama
Tuscaloosa, Alabama
CONTENTS
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Comprehensive Understanding of Networked Big Data . . . . . . . . . . . . . . 5
1.2.1 Challenges for networked big data . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.1.1 Distributed and decentralized data collection
and storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.1.2 Distributed and parallel data processing . . . . . . 7
1.1 Introduction
With the rapid development of computing capabilities and storage techniques,
there is no doubt that we are facing a series of opportunities and challenges
brought about by the big data era. Simply put, big data not only injects wis-
dom into decision-making and performance improvement but also poses a great
threat to traditional data processing techniques. For example, NeteaseMusic is
a popular musical software among Chinese teenagers and it is well known
for a particular function, Daily Recommendation, which aims to provide users
with the songs in which they may be interested. It tracks what songs indi-
vidual users listen to most frequently and what pages they look at; it usually
develops some relevant tags for each song, such as “sad,” “pleasure,” “Country
music,” and “Quiet.” Besides these tags, it is able to utilize some initial infor-
mation like similar singers and albums. This software develops algorithms to
better understand the customers by taking advantage of the digital data men-
tioned above. Customers can respond to each song with “like” or “dislike” by
highlighting, or not, a heart tag beneath the song. This process is similar to a
training process in machine learning. According to these valuable data, indicat-
ing users’ preference NeteaseMusic can recommend songs to users with very
high accuracy.
The success of NeteaseMusic can showcase the benefits that big data brings—
smarter decisions and better predictions. By utilizing it appropriately, we can
explore some fresh areas so far dominated by intuition and gut. Big data spreads
to every aspect in people’s daily lives, changing long-standing ideas about the
value of experience, the nature of expertise, and the practice of management [1].
Networking has become an indispensable part of the modern world, pro-
viding convenient access to the Internet, remote communication, information
exchange, etc. As mentioned above, big data also has a great influence on net-
working. The amount of data has exploded in recent years, with a transformation
from traditional simple-structured physical networks to complex-structured vir-
tual networks. Considering network management, a novel method is to collect
unstructured system logs (which can reach the magnitude of terabytes (TB) per
day) and analyze these valuable data to evaluate system performance or tackle
faults. So far, we have explored what opportunities networked big data brings,
but the challenges are also significant. In the next chapter, a variety of challenges
will be discussed.
Big data
sources Big data
(sender) destination
(receiver)
sources, such as sensors and data centers. For example, visualizing data collected
within the IoT are geo-related and sparsely distributed. Internet geographic infor-
mation system (GIS)-based solutions are required to cope with this challenge.
Apache Hadoop [7] is well known for its distributed storage and processing of
big data. Hadoop Distributed File System (HDFS) [8] is the core part of the
storage system in Hadoop, which is a distributed file system that stores data on
commodity machines, providing very high aggregate bandwidth across the clus-
ters. HDFS was inspired by Google file system (GFS) [9], which is the most
widely adopted mechanism for distributed file systems.
and followers. Mining the correlations between users’ behavior and preference
can reveal some underlying but valuable information for these social network
operators. In this dynamic world, there is no doubt that these relationships
evolve all the time, temporally and spatially. Another example is troubleshoot-
ing in networks, where topology graphs can be regarded as spatial information
while key performance indicators (KPI) or traffic records represented in time
series can be regarded as temporal information. Kimura et al. [13] once claimed
that log messages should be characterized as spatial-temporal data, since event
logs are usually generated from distributed hosts, routers, or switches spanning
across several geographical locations (spatial information) and include times-
tamps indicating generating time (temporal information). Based on deep insights
of spatial-temporal features of logs, they proposed two novel techniques, sta-
tistical template extraction (STE) and log tensor factorization (LTF), to capture
particular patterns of log messages which provide useful knowledge into root
cause analysis.
1.2.3.3 Tensor
Although many studies have been performed on networked big data processing,
very few have addressed the problems of representing the various formats of
data with a simple model and extracting the core data sets that are smaller but
still contain valuable information. Kuang et al. [26] targeted these issues and pro-
posed a unified tensor model to represent data with various formats. With the help
of tensor extension operators, the authors represented the various types of data
as subtensors and merged them to a unified tensor. They further devised an incre-
mental high-order singular value decomposition method (IHOSVD) to extract
the core tensor, which is able to update the orthogonal bases and compute the
new core tensor by recursively applying the incremental matrix decomposition
algorithm. In comparison with the traditional approaches of data dimensionality
reduction—e.g., principal component analysis (PCA) [27], incremental singu-
lar value decomposition (SVD) [28], and dynamic tensor analysis (DTA) [29],
which are mainly used for low dimension reduction—IHOSVD is particularly
suitable for handling high-dimensional data and extracting the core data sets from
streaming networked big data with a relatively low time consumption.
5. the greater house] i.e. the holy place. It was forty cubits long (1
Kings vi. 17), whereas the shrine was twenty (1 Kings vi. 16, 20).
13. twenty cubits] Extending across the width of the whole house.
on their feet] Not “couchant” nor “rampant” but standing, as the
winged bulls of Assyria stand.
17. Jachin ... Boaz] Margin translates the two words; Jachin “He
shall establish,” Boaz perhaps “In it is strength.” LXX. gives
Κατόρθωσις “setting up”) and Ἰσχύς (“strength”). The meaning of Boaz
is uncertain. It may be only a pious correction of an original Baal.
(For the avoidance of the word Baal, see the notes on xvii. 3, 1
Chronicles viii. 33; and for further comments on “Jachin” and “Boaz”
see Encyclopedia Biblia II. 2. 304, and Barnes in Journal of
Theological Studies, v. 447 ff.)
Chapter IV.
1.
The Altar of Brass.
The great altar was probably a flat oblong expanse, the highest of
a series of terraces, of which the base measurement is given by the
Chronicler—compare the description of Ezekiel’s altar (Ezekiel xliii.
13 ff.).
ten cubits from brim to brim ... and a line of thirty cubits
compassed it] The mathematical inaccuracy in the measurements
here given—10 in diameter, 30 in circumference—has often been
pointed out. But the literal Hebrew is “ten with the cubit ... and thirty
with the cubit,” and F. C. Burkitt in a communication to the
Cambridge Review for May 13, 1914 offers an interesting vindication
of the phrase. He writes “... What the verse says about the
circumference of the ‘sea’ is that they stretched a string round it, and
when they laid the string out flat they had to go thirty times with the
cubit, i.e. a man had to put his elbow down thirty times before he got
to the end.” [The distance from the point of the elbow to the tip of the
longest finger is 1 cubit.] “No doubt the last time he put his elbow
down the string came short: in other words, the ‘sea’ was nine-and-
a-bittock across and twenty-nine-and-a-bittock round. As a matter of
fact, if a circle be 9 ft. 6 in. across, it is just over 29 ft. 10 in. round.
Such a circle I think would be described in Hebrew as ‘ten with the
foot-rule’ across and ‘thirty with the foot-rule’ in circumference.”
for ten cubits] Obviously an error, for the “knops” encircle the sea,
and its circumference was thirty not ten cubits. The rendering of the
margin “ten in a cubit” gives good sense, but is not a fair translation.
Whatever the mistake in the Hebrew may be, it appears in 1 Kings
vii. 24 also.
The oxen were in two rows, cast when it was cast] Correct the
reading as before and render, The knops were in two rows, cast
when it was cast. It is mentioned as a triumph of the founder’s art
that the laver was cast complete, with its ornaments, from the first.
8.
The Tables and Basons.
9, 10.
The Two Courts.
V. 2‒VIII. 10.
The Dedication of the Temple.
Chapter V.
2‒10 (= 1 Kings viii. 1‒9).
The Ark brought into the Sanctuary.
the city of David] See 1 Chronicles xi. 5, note on the strong hold
of Zion.
³And all the men of Israel assembled
themselves unto the king at the feast, which
was in the seventh month.
3. the feast] i.e. the Feast of Tabernacles, i.e. after the fruit
harvest.
the priests the Levites] here the older phrase, which does not
indicate a distinction between Priests and Levites, has been allowed
to stand, perhaps “because certain utensils might well have been
borne by the priests” (so Curtis), or possibly through slight
carelessness on the Chronicler’s part. The parallel in Kings has “the
priests and the Levites.”
9. from the ark] Read (with LXX. and 1 Kings viii. 8) from the
holy place. One standing in the Holy Place and looking towards the
Holy of Holies could see the heads of the staves.
and there it is, unto this day] These words are taken over with the
loss of one letter (which here makes the difference between singular
and plural) from 1 Kings viii. 8, but they are out of place in
Chronicles, for when the Chronicler wrote the Ark had long ago
disappeared. The vessels which were brought back from the
Babylonian captivity are specified in Ezra i. 9, 10, but the Ark of the
covenant is not reckoned among them.
at Horeb] Deuteronomy v. 2.