SCIT Issues1 Big Data Ali Farhan

Master in Computer Information Systems
SEMINARS ON
CONTEMPORARY IT ISSUES
‫علي علي ناجي عبدهللا فرحان‬:‫أسم الطالب‬

Issues 1
Cloud computing and
data storage
THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

FINANCIAL SCIENCES (AAMBFS)
HTTPS://AAMBFSYE.ORG/EN/
What is Big Data?
Big data primarily refers to data sets that are too large or complex to be
dealt with by traditional data-processing application software.
Data with many entries (rows) offer greater statistical power, while data
with higher complexity (more attributes or columns) may lead to a
higher false discovery rate.[2]

FINANCIAL SCIENCES (AAMBFS) 3
Importance of Big Data
1. Uncovering Hidden Insights and Patterns:
Traditional data analysis methods often fall short when dealing with
massive datasets.
2. Improved Efficiency and Productivity:
By analyzing vast amounts of data related to operations, customer
behavior, and resource utilization, organizations can identify areas for
improvement and streamline processes.
3. Personalized Experiences and Targeted Marketing:
Businesses can leverage big data to gain deeper insights into individual
customer preferences and behaviors.

Importance of Big Data
4. Enhanced Risk Management and Fraud Detection:
Analyzing large datasets of financial transactions, security logs, and
other relevant data can help identify anomalies and suspicious patterns
that
5. Scientific Discovery and Research:
Big data is revolutionizing scientific research by enabling researchers to
analyze massive datasets from various sources like genomics,
astronomy, and climate science.
might indicate fraud, security breaches, or other potential risks.

How do Big Data work?

1. Data Collection:
Data comes from various sources like sensors, social media, financial
transactions, web logs, and scientific experiments.
Structured, semi-structured, and unstructured data formats need to be
captured and stored efficiently.
2. Data Storage:
Traditional databases struggle with big data volume and variety.
Distributed File Systems (DFS) and NoSQL databases like Cassandra and
MongoDB handle large datasets efficiently.

3. Data Processing:
Traditional data processing tools are slow and inefficient for big data.
Hadoop, a distributed processing framework, breaks down tasks into smaller chunks and
processes them across multiple machines simultaneously.
Spark, another framework, offers faster processing for real-time analytics.
4. Data Analysis:
Tools like Apache Pig and Hive enable analyzing large datasets using structured query
language (SQL).
Machine learning algorithms and predictive analytics tools extract meaningful patterns
and insights from the data.
5. Visualization:
Data visualization tools like Tableau and Power BI present complex data insights in clear
and understandable formats.

Types of Big Data
1. Structured data:
This is the most common type of data, and it is organized in a fixed and
predefined format. It can be easily queried and analyzed using traditional
database techniques.
Examples: Customer records, financial transactions, sensor data, log files.
2. Unstructured data:
This data does not have a fixed format and is difficult to process using
traditional methods. It includes text, images, videos, audio, and social
media data.
Examples: Email, social media posts, documents, images, videos, sensor
data.

Types of Big Data
3. Semi-structured data:
This data has some structure, but it is not as rigid as structured data. It
often includes tags or markers that provide some context, but it does
not fit neatly into rows and columns.
Examples: JSON, XML, log files, web pages.
Metadata:
This data provides information about other data. It can be used to
describe the origin, format, and content of other data sets.
Examples: Tags, timestamps, geolocation data, sensor readings.

Types of Big Data
5. Real-time data:
This data is generated continuously and needs to be processed and
analyzed in real time. It includes data from sensors, social media, and
financial markets.
Examples: Stock prices, sensor readings, social media activity, traffic
data.

Examples of uses of Cloud computing
Healthcare:
Predicting patient readmissions: Hospitals analyze patient data to
identify high-risk individuals for readmission after discharge, allowing
for preventive measures and personalized care plans.
Developing personalized medicine: By analyzing genetic and health
data from large populations, researchers can develop targeted
treatments and therapies for specific patient groups.

Retail:
Demand forecasting and inventory management: Retailers
use customer purchase data to predict future demand and
optimize inventory levels, minimizing stockouts and
overstocking.
Personalized recommendations: Analyzing customer
preferences and purchase history allows retailers to offer
personalized product recommendations, improving
customer satisfaction and sales.

Finance:
Fraud detection: Financial institutions analyze transaction data to
identify suspicious activity and prevent fraudulent transactions in real-
time.
Credit risk assessment: By analyzing financial data and credit
history, lenders can assess creditworthiness more accurately and offer
personalized loan terms to borrowers.

Transportation:
Traffic prediction and route optimization: Analyzing traffic
data from connected vehicles allows for real-time traffic
prediction and dynamic route optimization, reducing
congestion and improving travel efficiency.
Predictive maintenance for vehicles: Airlines and
transportation companies use sensor data from vehicles to
predict potential maintenance issues before they
occur, preventing breakdowns and ensuring safety.

Media and Entertainment:
Content personalization: Streaming services analyze user viewing
habits to recommend personalized content, increasing engagement and
user satisfaction.
Targeted advertising: Analyzing social media data and browsing
behavior allows for targeted advertising, delivering ads relevant to
individual users' interests.

Techniques & Tools used in Big Data
Analytics
Distributed File Systems (DFS): HDFS (Hadoop Distributed File
System), GFS (Google File System), Cassandra - Distribute data across
multiple nodes for scalability and fault tolerance.
NoSQL Databases: MongoDB, Cassandra, Couchbase - Offer flexible and
scalable storage for unstructured or semi-structured data.
Data Warehouses: Amazon Redshift, Snowflake, Google BigQuery -
Provide centralized repositories for large-scale data analysis and
reporting.

Analytics
Data Integration:
ETL (Extract, Transform, Load) Tools: Informatica PowerCenter, Talend
Open Studio, IBM DataStage - Extract data from various
sources, transform it into a consistent format, and load it into target
systems.
Data Lakes: Store raw data in its native format for future exploration
and analysis, allowing flexible schema and scalability.

Analytics
Data Processing and Analysis:
MapReduce: A programming framework for processing large datasets
on distributed systems.
Spark: A unified engine for large-scale data processing, offering faster
performance and broader capabilities than MapReduce.
Stream Processing: Apache Flink, Apache Kafka - Analyze and process
data streams in real-time for faster insights and decision-making.
Machine Learning and Artificial Intelligence (AI): Leverage algorithms
to uncover patterns, make predictions, and automate tasks using large
datasets.

Analytics
Data Visualization:
Tableau: Create interactive dashboards and visualizations for data
exploration and communication.
Power BI: Analyze and visualize data from various sources with
interactive reports and dashboards.
Python libraries (matplotlib, seaborn): Create custom data
visualizations for deeper analysis and exploration.

Benefits of Big Data
1. Improved Decision Making:
Data-driven insights: Analyze vast amounts of data to identify
patterns, trends, and correlations, enabling better-informed and
evidence-based decisions.
Risk assessment and mitigation: Predict potential issues and
proactively address them, reducing risks and improving outcomes.
Targeted marketing and customer engagement: Understand customer
behavior and preferences, personalize offerings, and increase customer
satisfaction.

2. Operational Efficiency and Optimization:
Resource optimization: Analyze resource utilization patterns to
optimize allocation and reduce costs.
Predictive maintenance: Identify potential equipment failures before
they occur, preventing downtime and reducing maintenance costs.
Streamlined processes: Analyze process data to identify bottlenecks
and inefficiencies, leading to smoother operations and faster
turnaround times.

3. Innovation and Product Development:
Identify new market opportunities: Analyze customer data and market
trends to discover unmet needs and potential new products or services.
Develop personalized products and services: Tailor offerings to
individual customer needs and preferences, leading to higher customer
satisfaction and loyalty.
Faster innovation cycles: Analyze data from various sources to
accelerate development and testing of new products and features.

4. Scientific Research and Discovery:
Analyze large datasets: Analyze complex datasets from various fields
like genomics, astronomy, and climate science to make groundbreaking
discoveries.
Develop new medical treatments: Analyze patient data to understand
diseases and develop personalized treatment plans.
Improve climate change models: Analyze environmental data to better
understand climate patterns and predict future trends.

Challenges of Cloud computing
Big data, while offering immense potential for insights and innovation,
presents several challenges that need to be addressed:
1. Volume: The sheer volume of data can be overwhelming, requiring
specialized storage, processing, and analytical tools.
2. Variety: Data comes in diverse formats (structured, semi-structured,
unstructured) requiring flexible handling and integration techniques.
3. Velocity: Data generation happens very quickly, demanding real-time
or near-real-time processing for valuable insights.
4. Veracity: Ensuring data accuracy, consistency, and completeness is
crucial for reliable analysis and decision-making.

Challenges of Cloud computing
5. Security: Protecting sensitive information within vast datasets requires
robust security measures and access control.
6. Privacy: Balancing data analysis with individual privacy concerns is an
ongoing challenge, requiring adherence to data regulations and ethical
considerations.
7. Cost: Storing, processing, and analyzing big data can be expensive, requiring
cost-effective solutions and optimized resource utilization.
8. Skills Gap: Skilled professionals with expertise in big data management,
analysis, and visualization are in high demand.
9. Integration: Integrating big data with existing IT systems and data
infrastructure can be complex and time-consuming.
10. Governance: Establishing clear governance policies and frameworks for big
data management is essential for responsible and ethical use.

References
•Big data – Wikipedia
•https://gemini.google.com/
•McKinsey Global Institute: https://
www.mckinsey.com/business-functions/mckinsey-digital/our-insights/bi
g-data-the-next-frontier-for-innovation
•IBM Big Data & Analytics Hub:
https://www.ibm.com/analytics/big-data-analytics
•"Big Data: A Revolution That Will Transform How We Live, Work, and
Think" by Viktor Mayer-Schönberger and Kenneth Cukier
•Big Data Challenges - ResearchGate:
https://www.researchgate.net/publication/332543555_Big_Data_Impac
ts_and_Chal


SCIT Issues1 Big Data Ali Farhan

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

SCIT Issues1 Big Data Ali Farhan

Uploaded by

Copyright:

Available Formats

Master in Computer Information Systems

‫علي علي ناجي عبدهللا فرحان‬:‫أسم الطالب‬

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

THE ARAB ACADEMY FOR MANAGEMENT, BANKING AND

You might also like