Real-Time Big Data Analytical

Architecture for Remoting Sensing

Application

Author: M. M. U. Rathore, A. Paul, A. Ahmad, B.-W. Chen, B. Hunag and W. Ji

Presenter: Junyi Shen
 Big Data
High speed continuous stream of data
or
high volume offline data
Why?

• Various fields of remote sensory satellite image data are needed

• Consequences of transformation of remotely sensed data to the scientific

understanding are critical

• Sensors produce an overwhelming amount of raw data

• Data collected from remote area are not in a format ready for analysis
Architecture
1.jpg DADU:
Compilation, storage of results,
and generation of decision

DPU:
Filtration, load balancing, and
parallel processing

RSDU:
Data acquisition from satellite
and delivery to Base Station
Remote Sensing Big Data Acquisition Unit (RSDU)

Ground station
• Remove distortions from data

• Assume satellite can

correct error data
• Raw data -> image format

Integrate data
• Decrease storage cost
• Improve analysis accuracy
Data Processing Unit (DPU)

• Identify useful information,

• Each has its algorithm
block and discard the rest
implementation
• Divide filtered data into parts
• Perform tasks independently
and assign to various
and in parallel
servers
Data Analysis and Decision Unit (DADU)

• Display and broadcast the

decision

• Discover hidden things

and make decisions

• Aggregate partial results • Any server can access

received in DPU and
organize into proper form
Flowchart
 Case Study
Detect Land, Sea or Ice Area
for remote sensing Big Data images
Data Set

• Take satellite-sensed Big

Data samples from European
Satellite Agency (ESA) to
analyze land, sea and ice
separately

• Analyze the ENVISAT

mission data sets
(continuously providing
global measurements for the
earth including sea, land and
ice since 2002)
 Analysis
Ø Mean value for land areas
are quite lower compared
to sea area
• Land normally has
greenery (except dessert)
• Color of other objects is
nearer to black

Ø SD values of land are

higher than sea area
• Sea has one color
• Few particles on the
surface of sea
• Land has more different
colors on its surface
 Reason:
Large lake between
land surface

• PVD for land area

• PVD for Sea area

Algorithm Design

Algorithm IV

Algorithm III

Algorithm II

Algorithm I
Parameters and Variables Introduction
Algorithm I && II
Algorithm III & IV
Results and Evaluation

• TP : percentage of land blocks

correctly identified
• FP : percentage of sea blocks
incorrectly identified as land blocks
Hadoop MapReduce vs. Simple Java
Reference
[1] M. M. U. Rathore, A. Paul, A. Ahmad, B.-W. Chen, B. Hunag and W. Ji, "Real-Time Big Data Analytical
Architecture for Remote Sensing Application", IEEE Journal of Selected Topics in Applied Earth
Observations and Remote Sensing, vol. 8, no. 10, pp. 4610-4621, 2015.

[2] S. Kalluri, Z. Zhang, J. JaJa, S. Liang, and J. Townshend, “Characterizing land surface anisotropy from
AVHRR data at a global scale using high performance computing,” Int. J. Remote Sens., vol. 22, pp. 2171–
2191, 2001.

[3] European Space Agency. (Oct. 14, 2014). [2312] [Online]. Available: https://earth.esa.int/

[4] EnviSat, A. S. A. R. Product Handbook, European Space Agency, Issue 2.2, Feb., 2007.
Discussion - Strength & Weakness

• A genetic architecture for any type of • RSDU assumes satellite can correct error
remote sensing real-time Big Data analysis data. Does it work for all the cases?
• Efficiently process and analyze real-time • How to classify real-time Big Data and

and offline data offline Big Data as it is not clearly described

• Detail explanation of architecture in this paper?

implementation in real case • A simple case study can reflect the

effectiveness of this architecture?
Discussion - Future Work

o Add erroneous data handler unit in RSDU part

o Perform complex analysis on earth observatory data for real time decision making

o Compatible for Big Data analysis for more applications

Discussion – Related Papers
q Md. Ali Hossain, Xiuping Jia and Jón Atli Benediktsson, "One-Class Oriented Feature Selection and
Classification of Heterogeneous Remote Sensing Images", Selected Topics in Applied Earth Observations
and Remote Sensing IEEE Journal of, vol. 9, pp. 1606-1612, 2016, ISSN 1939-1404.
§ Propose a one-class oriented approach for effective feature selection and classification of remote sensing
images.

q Alexandru-Cosmin Grivei and Mihai Datcu, "HyperMINE — An earth observation spatio-temporal data
mining system", Telecommunications Forum Telfor (TELFOR) 2015 23rd, pp. 906-909, 2015.
§ Present An Earth Observation Spatio-Temporal Data Mining System (HyperMINE), which integrates fast
and complex query methods in order to generate SITS (Satellite Image Time Series) regarded as a data
hypercube.
§ The system is built on a modular multilayer architecture that allows effective processing of various sources
of data.
Discussion - Questions
Ø Is this proposed architecture essential for remote sensing Big Data? Can we use or modify the
already existing architecture used for common big data such as video/audio and email to deal with
the remote sensing data?

Ø Can we improve the efficiency of DPU by classifying and storing different information fetched
from RSDU? Getting data from RSDU and discarding useless information every time cost high.

Ø From simple case study in this paper, we can see analysis processing time is not low enough. So is
it positive to deal with complex analysis of Big Data in real-time?

Ø Hadoop implementation and simple Java implementation have difference performances when
product size is increasing. Is it possible to make a combination to improve the whole efficiency?