Big Data and Science: Myths and Reality

Muhammad Umar
Department of Computer Science & Information Technology University
of Sargodha
University Road Sargodha, Punjab, Pakistan
Email:

Abstract

As Big Data unbending draws attention
from every part of a society, it has also
suffered from many characterizations
that are incorrect. This article will
explores common myths about Big Data
and exposes the truths which are
underlying. Big Data impacts nearly
every aspect of our modern society,
including business, government, health
care and research in almost every
discipline: life sciences, engineering,
natural sciences, art and humanities. As
it has drawn much attention and become
economically important, there are many
who have preferred angles on the
interpretation of Big Date. At the same
time, as many have been exposed to the
term with little prior knowledge of
computing or technology they are easily
sawayed by the EXPERTS. There is a
widely use of the term Big Data in ways
that are inappropriate but self serving. In
many cases these erroneous
interpretations have then been taken up
and amplified by others, including even
technically sophisticated people. In this
article I will discuss some of the more
common myths.
Introduction
Big Data is a huge size Data. It is a term
used to describe a collection of data in
huge volume and then growing with
time. In short it is such a data which is
so large and complex that none of the
old or traditional data management tools
are able to store it or process it very
efficiently. For example the New York
Exchange generates about one Terabyte
of new trade data per day.
Or The statistic shows that 500+
terabytes of new data get ingested into
the databases of social media
site facebook, every day. This data is
mainly generated in terms of photo and
video uploads, message exchanges and
putting like, comments etc.
Types of Big Data
1. Structured
2. Unstructured
3. Semi-Structured
Structured Data
Any data that can be stored accessed and
processed in the form of fixed format is
termed as a structured data.
Unstructured Data
Data with unknown structure is
classified as unstructured data.
Semi-Structured Data
It can contain both forms of data, such
as structured and unstructured data at the
same time.
Myths:
1 Big Data Myth: Size is all that
matters
The word “Big” means size. It is a
method that takes the dimensions of the
masses easily passed. We've all heard
the statements of how a high amount of
phone books are needed to store data
that is easily stored on a single disk
drive. So it's no surprise that for most
sleepers, Big Data is about size. One
would think that people working with
technology would know better.
Unfortunately, size also helps in simple
estimation. It is straightforward to
calculate the number of bytes in another
data store, and it is relatively easy to plot
the sequence of such measurements in a
chart showing high growth. In fact, such
charts have become so common that
even the most nominated people get the
idea. Where does this lead, among other
things, seriously if people have a few
hundred gigabytes of data and are not
sure they have a big data problem. This
is sad, because we put down so many
people we should be able to help. In
addition to the above points, I believe a
better idea would have been in our
understanding of Big Data if it weren't
for the economics of the IT industry.
Today we have a great ecosystem of Big
Data systems. These systems, in
particular, are innovative: collectively,
they form a new paradigm for
measurement. There are many problems
that require this measure and are useful
for new construction. These facts have
led to the creation of a new industrial
segment and have many benefits, all of
which are good. But the great progress
made in this space has also absorbed O
oxygen in the air with everything else,
as it were. The industry wants to talk
about volume, for economic reasons.
And money is talking.Several years ago,
the Gartner team saw this unwavering
attention focused on size, and suggested
the now popular "3Vs" of Big Data [5].
IBM then urged the addition of the 4th
V [6], and this was adopted by the
majority. So, of course, most people in
technology will tell you that Big Data
raises the issues of Volume, Velocity,
Variety, and Veracity (or at least the
first three of these). However they will
immediately go on to discuss how many
Petabytes have some problem. I
discussed above, why the Volume (or
size) gets the wrong attention. Let me
turn now to why I think Variety and
Veracity don't get the attention they
deserve. The main reason for this
disregard is that there is no
well-accepted measure of any. When
there is no measurement, it is difficult to
track progress. If I own companies and
create a new system that can handle
larger volume than competitors, I can
suggest this with some benchmark
resistance measures. If I am a student
and develop an algorithm that scales
better than a competitor, I know exactly
how to compare my algorithm against a
competitor and persuade skeptical
reviewers. In contrast, consider the
variety. If I have a product that makes
management easy to use, what technical
claim would make it sound like
advertising hype? When I write a paper
about a data model that is better for
handling diversity than the current state
of the art, I have to think hard about how
I will compare it to the competition and
find the beauty of my idea. Progress is
hard on things you can't measure, in the
field of education and in education. The
variants can be very difficult for 4Vs to
talk about, but they are the ones people
can be motivated to talk about. Veracity
encounters many problems such as
Variety. Under the simplest models, at
least we can start measuring other things,
establishing certain probabilities and
distributions, and so on. But everyone
recognizes that these steps are based on
a rational model: for example those that
claim independence when we know it is description is round because it actually
not true. So, steps are taken with a grain does not define what goes into the
of salt, and Veracity is much easier to toolbox. Big Data, or similar statement.
deal with than variety. To conclude, This description works for yourself
Volume and Velocity are really because it greets the tool set and system
challenging. building style as a solution to the Big
Data problem. This description is
incorrect because almost everything in
2 Big Data Myth: Central challenge the Big Data toolbox is focused on
with Big Data is that of devising new Volume (often in conjunction with
computing algorithms and Velocity), with very little attention given
architectures to Variety and Veracity challenges. I
believe that the cloud, and what is today
considered the “Big Data Ecosystem,”
If we think smartly about Big Data in
has its place where the right technology
terms of 4Vs, we immediately have the
is built, but it is not a complete solution
question of what the limits are to call
in itself or a necessary piece of the
something “Big”. Diversity and of
whole solution. My Big Data threshold
course we know that this is definitely
is higher (aside from the other 4 axes)
not an unanswerable question, because
than we know you have to handle
we have no means at first. So let's just
context. The scientist (or manager) is
look at Volume and Velocity. The limit,
responsible for the Big Data system
of some people, is within the limits of
where he has so much data that he can
what we can manage. Obviously, this is
process it using a spreadsheet system he
a moving target. But it has the benefit of
knows. The solution to this is probably
being an inspiration. The sad (in my
as simple as moving to a database. But
opinion) setback is that it limits the size
even simple simple variables can have
of the market to 1: there is only one
many hidden problems: the current
largest shipment in the world at any one
spreadsheet layout may not be ready for
time (bond closure). Increasing the size
a related table (for example, a new
of this submission is a worthwhile
column can be added every month), it
challenge, but not one that could build a
may depend on other elements of a
whole industry around and develop a
particular workflow, and so on.
comprehensive educational field. The
Identifying and eliminating such barriers
limit, in some definitions, is then
is a legitimate task for Big Big. See, for
modified, based on the main construct at
example, a report by the National
some point, say 2010. The data set is
Academies reports “Frontiers in Massive
therefore appropriate, depending on
Data Analysis” [4]. It is also notable that
Volume, as Big Data if it is larger than
we can buy larger systems, more
hosted using the "standard" structures
machines, faster CPUs, and larger disks.
used at the beginning of the Big Data era.
But human ability is unmatched! In
With the ever-increasing popularity of
addition, sizes that challenge humans
Map-Drop style integration, and most of
tend to be too small for computers. For
the programs and tools in the "Big Data
example, consider a graph with just 40
ecosystem, we have a clear, albeit
locations and 200 edges. Try editing it
circular and self-explanatory definition:
on the screen with your favorite graphic
Big Data Problem is one that is best
design and look for patterns. Or such a
addressed using Big Data toolbox
small graph may be at the limit of what
features. there is general agreement as to
we can handle with technology today.
which tools in the Big toolbox are: most
Big data poses great challenges for
toolmakers distinguish themselves .This
human interaction. Many of the most
interesting problems in the Big Data
area deal with simplifying this human
interaction.
3. Big Data Myth 3: Analytic’s is the
central problem with Big Data
It is completely understandable that
many people posing for an image equate
the Big Data System as part of the magic
of software that takes Big Data as input
and produces in-depth information such
as output. Unfortunately, this idea of
remembering resonates with many
companies, even some professionals,
very well. This way, the person who is
building the Big Data system (in the
sense described above) can create the
illusion of solving the whole problem
from soup to nuts or focusing on just a
piece of it. The same goes for the person
who created the novel algorithm for the
analysis. But Big Data definitely doesn't
learn about machines in map reduction.
A group of leading researchers across
America wrote a white paper to correct
this misconception, they [1]. Short
version, making the same key points,
appeared in CACM, July 2014 [2].
Figure 1 is reproduced in this white
paper. The main point to make is that
there are many steps to go through the
Big Data analysis pipeline, and the
important decisions needed for each step,
as well as the many challenges to face in
each one. The first decision is what data
to record or retrieve, and how to make
the best incomplete data. After that
decisions have to be made to represent
the data correctly for analysis, possibly
after extraction, cleaning, and
integration with other data sources. Even
in the analysis phase, which has received
a lot of attention, there is a
misconception of the content of multiple
clusters that appear when several user
programs are running at the same time.
The final interpretive step is perhaps the
most important, since it cannot be
transferred - someone is responsible for
making decisions based on the results of
data analysis and the individual must
understand and trust the results obtained
first. Finding this certainty will require
understanding and interpretation, it may
require observation, it may require
sensitivity analysis of various types. All
of this has to be organized and done
successfully in analyzing Big Data to
produce any real value.
Phases of Big data life cycle
Big Data analysis is in the pipeline.
Major steps in Big Data analysis shown
in the top half of the figure. Be aware of
possible ways of responding at every
stage. The bottom half of the figure
shows the Big Data features that make
these steps challenging.
4 Big Data Myth: Data Reuse is low
hanging fruit
We have to collect data for some
purpose. You should be able to use it for
a different purpose, thus eliminating the
huge cost of collecting data a second
time. In fact, reuse may not be safe in
most cases, if a second analysis is done
later, in which case there will be no
opportunity to go back to collect
historical data again. While this is a
stressful opportunity, exploiting it
requires a number of challenges. First,
the original data set must be available
during the desired reuse. It's easy to
mark data tables (or to use existing
labels in a data set, such as attributes and
table names) to find the data sets that are
in the subject of your favorite topic.
However, in the vast universe of data
sets, there may be hundreds of data sets
that are somehow related to the topic of
interest by only a few who have the
information in the relationship of
interest measured under the conditions
of interest. We are now starting to think
about how we use the data sets to enable
them. Second, the data set must be
understood and interpreted in order to
function. Obviously, this requires
sufficient meta-data. Unfortunately, the
word “enough” in the previous sentence
is often overlooked. If we know the
manufacturer and the date, and the
schema declaration, that is enough meta
data in many cases. It is likely to be
important under what conditions the
data are obtained, using what resources,
after which sample preparation. There is
a useful function with meta data rates in
many communities. Following these
values ​ ​ will undoubtedly motivate
us even more. However, we also need to
address the issue of incentives, at least
in the scientific community: why a
scientist will spend time recording meta
data more carefully. Why not just do the
minimum amount required by a
publishing agency or financial agency?
In addition, there is still enough
variability even under any of the few
studies studied that most of these meta
data levels do not require information
that may be relevant to a specific
situation, even if it is not fully functional.
Efforts to establish a data collection
culture are essential to address these
issues. Third, the data sets received are
usually not as efficient as you used them
to be. Sometimes this is just a question
of mapping. But too much frequency is
usually resolved. One issue I am talking
about right now has to do with
administrative data, which is often
reported to have been collected by
management administrators. When such
information is reused, it needs to be
compared (or combined) with the data
established according to a specific
management area. If the two hierarchies
are different, such a parallel cannot
happen immediately. For example, it is
not straightforward to compare the data
reported by the school district with the
data reported by county. Our approach
to this problem is to develop new
translation methods. Data reuse is
important to fix and it promises good.
But there are also a number of
challenging questions, which have only
now been overlooked.
5. Big Data Myth 5: Data Science is
the same as Big Data
The ability to collect and analyze large
amounts of data is revolutionizing the
way scientific research is done [3]. The
Sloan Digital Sky Survey [9]
transformed Astronomy from a field
where astronomical photography was a
major part of the astronomical work to
another with a focus on finding
interesting objects and contexts based on
data. in Biological Science, there is now
a well-established tradition of
incorporating scientific data into public
archives, as well as building databases
used by other scientists. The size and
number of test data sets in many
applications are increasing dramatically.
Consider, for example, the advent of
Next Generation Sequiling NGS [7].
The growth rate of the release of current
NGS methods is faster than the increase
in SPU benchmark (SPECint)
performance, indicating an increase in
computational power due to Moore's law.
Both volume and velocity of data
require new methods of data control and
analysis. For example, the raw image
data sets in the NGS are so large that it
is useless today to visualize and
maintain them. Instead, these images are
analyzed in transit to produce sequence
data. Many people use the terms
“Scientific Data” and “Big Data”
interchangeably, using these terms in all
the examples listed above. This is
completely wrong: the first difference
between the two terms is their idea: "Big
Data" starts with data signals (and works
there), while "Data Science" starts with
data usage (and works down there).
However, their formal definitions differ
from more than mere observation. The
National Consortium for Data Science,
an industry and educational partnership
founded in unc, Chapel hill in 2013,
describes data science as "a systematic
study of digital data using visual science
techniques, concept development,
systematic analysis, hypothesis testing
and validation. The main purpose of data
science [8] is to use data to describe,
describe, and predict ecological and
social phenomena by creating
information about the properties of large
and dynamic data sets. latency, and cost.
When we compare this definition of
Data Science with the Gartner definition
of Big Data we saw earlier, we see
immediately that it's possible to do Data
Science without doing Big Data, with
viceversa. Of course, nothing stops Data
Science from including Big Data, and it
often does. However, paying our
attention to the connection of these two
objects is of limited importance.
Another point to note is that Data
Science activities often involve data
analysis by a domain expert with limited
expertise. If a domain expert can
succeed, the data should work.
Undoubtedly, data systems are very
difficult to use. There is even an urban
myth about some merchants in order to
keep them hard to use because they
make a lot of money from consulting
and support fees. In addition to the
programs themselves, there are
analytical tasks - often, we use users to
formulate unsupported ideas about the
data at hand, e.g. in relation to
independence or impartiality or the way
in which a data set is represented. If we
do not help people to use their data
intelligently, they will be burned and
challenged by all the good our
technology can bring you. Database and
data analytics usability research are
important.
6. Big Data Myth 6: Big Data is all
hype
Periodic data analysis. Reduce
information too. So what has changed.
Why now is the time to be happy with
Big Data. Whether or not this is simply
cooked by soulless journalists with a lot
of attention to Big Data, this is a
question worth asking. But we see that
data collection is cheap today, thanks to
ubiquitous access, automated business
processes, the web and sensor networks,
in a way that has never been seen before.
Data storage is also cheaper, thanks to
media pricing. As a result, almost every
category of change changes from "poor
data" to "rich data." So it's no surprise
that everywhere around us we have
people asking about the potential of Big
Data. At the same time, we have a
growing public understanding of the
effects of Big Data. We're just starting to
get serious today in our measurement of
data privacy. Our appreciation for the
principles of data analysis is also at its
infancy. Errors and excesses in this case
may result in a backlog that can shut
many things down. But to prevent such
errors, it's safe to say that Big Data may
be hyped, but there is just enough
something to be overlooked.
References
[1] http://data2discovery.org/dev/wp-conten
Challenges and opportunities with Big t/uploads/2012/09/NCDS-Consortium-R
Data oadmap_July.pdf (2012)
a community white paper available at Google Scholar
http://cra.org/ccc/docs/init/bigdatawhite [9]
paper.pdf SDSS-III: Massive Spectroscopic
Google Scholar Surveys of the Distant Universe, the
[2] Milky Way Galaxy, and Extra-Solar
H.V. Jagadish, Johannes Gehrke, Alexan Planetary Systems
dros Labrinidis, Yannis Papakonstantino available at
u, Jignesh http://www.sdss3.org/collaboration/desc
M. Patel, Raghu Ramakrishnan, Cyrus S ription.pdf (Jan. 2008).
hahabiBig data and its technical
challenges
Commun. ACM, 57 (7) (July 2014),
pp. 86-94, 10.1145/2611567
CrossRefView Record in ScopusGoogle
Scholar
[3]
Advancing Discovery in
Science, Engineering Computing
Community Consortium (Spring 2011)
Google Scholar
[4]
Frontiers in Massive Data
Analysis, National Academies
Press (2013)
Google Scholar
[5]
Pattern-Based Strategy: getting value
from Big Data
Gartner Group press release, available at
http://www.gartner.com/it/page.jsp?id=1
731916 (July 2011)
Google Scholar
[6]
The 4 V's of Big Data
http://www.ibmbigdatahub.com/tag/587
Google Scholar
[7]
Scott D. KahnOn the future of
genomic data
Science, 11 (February 2011),
pp. 728-729
CrossRefView Record in ScopusGoogle
Scholar
[8]
Establishing a National Consortium
for Data Science
available at