BIG DATA, HADOOP & SPARK

DOW CHEMICAL COMPANY: A CASE STUDY OF BIG DATA IN

MANUFACTURING

Group 2:
KANDRA MEGHANA- F21032
JOSEPH JOHN  MATTAM -
F21088
INESH ABISHA  S - F21144
RENNITA  S - F21166
ASSIGNMENT QUESTIONS

1. What is big data?

Big Data is characterized as extraordinarily massive data collections, with potentially billions
of rows and parameters. It is a massive collection of data that is also expanding exponentially
over time. No typical data management systems can effectively store or process this data
because of its magnitude and complexity. Big data is a type of data that is extremely large.

a) What is big data in the context of manufacturing?

The big data in manufacturing market was worth USD 3.22 billion in 2018 and is
expected to grow to USD 9.11 billion by 2026, with a CAGR of 14.0%. With the rapidly
changing business environment and advancements in technology, manufacturers are
investing in automation of manufacturing operations. In addition, manufacturers are
focusing on increasing the productivity and lowering the costs by deploying advanced
technologies such as big data, artificial intelligence, and the internet of things. Big data
technology is supporting the manufacturing industry to improve the production process and
help to extract data from various devices to improve the production, quality, automated
operational workflow and reduce maintenance costs.

Big data analytics software helps to analyze risk patterns, maintain workflows,
improve quality of the processes, production, and others. Big data solutions analyze,
collect, and monitor a large volume of unstructured and structured data generated from a
variety of sources such as production unit, product quality, factory floor etc. Big data
technology helps to uncover newer trends and patterns and provide actionable insights to
businesses.

b) What are the opportunities that big data provides in manufacturing?

Big data provides variety of opportunities when it comes to manufacturing. Big Data
is being considered by manufacturing companies as a targeted and organized method of
identifying and resolving process problems. According to a 2016 study, despite pressure to
minimize production costs, almost 67% of industrial executives planned to invest in big
data analytics. The argument is that technology has demonstrated to be a crucial element in
finding solutions to problems that result in lost revenue and downtime. There were huge
data in the manufacturing sector but at that point we had no clue on how to use these data.
But now we have advanced analytical tools that make statistical assessments. A major
opportunity prevails now in using data for improving operations rather than for tracking,
which has been its use. The case mentions that over 200 variables exist in chemical
manufacturing and that yields can vary for no reason. Big data can be deployed to
minimize those uncertainties. A practical example is that by using advanced analytics
associated with big data, a vaccine manufacturer was able to pin down yield variations to
nine specific parameters. On tracking those parameters, the manufacturer was able to
increase the vaccine's yield by 50 per cent, worth between US$5 million and $10 million in
yearly savings for a single vaccine alone.

2. Is the chemical industry amenable to the deployment of enterprise manufacturing

intelligence (EMI)?

Terabytes of structured data were captured by data historians from sources like the sensors,
monitors, and meters that come with every piece of equipment on a factory floor. R&D focused
on historical sites, while process engineers in a factory tended to be more concerned with
regional issues. Years might have to go by while they proceeded to measure a parameter
without pausing to consider if the measurements were significant or helpful. The adoption of
 EMI, which will involve collecting and analysing structured and unstructured data from many
sources along the industrial value chain, is acceptable to the chemical sector. It will focus on
identifying value in already-existing data rather than adding additional measures to produce
more data. Instead, than fixing or foreseeing problems in real time, the majority of the study
was on recognising problem statements that had already occurred. To track tags and collect
data in order to measure the product quality, The Dow employed its Research and Development
facility and a number of engineers.

3. What is EMI? How does EMI help a company such as Dow?

EMI is Enterprise Manufacturing Intelligence (EMI). It is a software solution that brings

together data from various sources in a manufacturer’s operations. From that the software
produces reports, analysis and other such useful tools. Ultimately it increases the manufacturing
performance.

In this case study, the EMI helps Dow to analyze, visualize, contextualize, aggregate, and
propagate data. Through which the real knowledge is gained, without which it would be
redundant or unusable otherwise. Big Data is used in addressing approaches of analyzing,
systematically taking data or having datasets that are large or complex enough to be handled by
traditional data-processing software. Here the characteristics applied by the company are –

A. Velocity of big data

B. Volume of big data
C. Veracity of big data
D. Variety of big data

• Velocity is about the speed or rate of the data that is generated. When the data has high
velocity with high speed, it needs distinct or distributed processing techniques. Dow
possibly attempted to create production capacities within the US depending on the
availability of a low-cost fuel to keep their manufacturing running.

• Volume of big data is required to analyze the large amounts of data in the company. The
company data process in typical data warehousing and business intelligence installation was
measured in petabytes. Such large data is equivalent of text requiring a storage capacity of
20 million filing cabinets of the kind using in a typical office.

• The veracity of data referred to the inherent discrepancies in the data collected, bringing the
reliability and trustworthiness of the data itself into question. Here, if there were to be low
veracity, there would be a large meaningless data known as Noise.

• The variety of sources from which big data could be mined included both structured data,
such as database tables, and unstructured data originating from diverse sources such as
mobile phones, social networks, sensors, video archives, radiofrequency identification
(RFID) chips, and global positioning systems.

4. What are the skills required of a big data professional?

Typical skills required for a big data professional is analytical and soft skills. While technical
knowledge could be an add on, there is always scope for learning and re-learning in the forever
evolving technology. Understanding on Hadoop, Hive and MapReduce is required. How these
technologies are implemented in real world applications. Having knowledge in conceptual and
predictive modeling, statistical analysis and hypothesis testing.

Soft skills are a must, where the ability to advise another colleague or sub-ordinate should be
 there. To have presentation skills, using query databases and using statistical analysis could be
adding to the statistical aspects in a big data sector. Having exposure to visualization tools like
tableau or Qlik would be an add-on.

Attention to detail is main aspect, showing how effort is being put. Where the Dow company
could have considered that there is potential revenue opportunity besides manufacturing. They
could give consulting to other chemical manufacturing companies as they have expertise with
experience in the area.

5. Why should Dow access data at the points of origin, in real time?

For valuable information, having data from real time at the points of origin would be
beneficial. The plant engineers at the company are expected to fix the challenge of large data
amounts when there is information starvation. The company has 5,000 products in 36 counties
and 188 sites, which would mean a large data. A batter application is required for its data. Here
accessing the data was challenging because of creation of parallel data spaces and redundancy.
Even with upgrades of data systems, the solution would be unrealistic. Dow’s design is an ideal
approach comprising an agnostic or neutral method. The design has capacity to investigate and
access data on real time data at the points of origin.

6. How should Dow gain user acceptance and scale up usage of data company-wide?

A culture change is necessary for Dow to accept and use big data analytics. A major obstacle
to implementing change is finding the right technology fit and individuals for the procedure.
People's perceptions of data and how they make decisions may change as a result of the use of
big data analytics. Change may often be unsettling. Employees may exhibit resistance or take a
defensive position in response to widespread or rapid change. By communicating the changes
consistently, conducting seminars and workshops to raise understanding of the advantages of
implementing Big Data, and bringing the changes little by bit, Dow may overcome the
difficulty. Additionally, providing additional technical training may be crucial in overcoming
resistance to change.

7. What are the metrics by which Dow could measure the return on investment of EMI?

Dow can use various metrics to measure the return on investment of EMI. The various factors
also include expansion, profitability and client happiness. The following metrics help us in
measuring the return on investment (ROI) of a company using big data.

 Accuracy
 Efficiency
 Scalability

8. What are the alternative paths you would suggest for implementing EMI at Dow?

Several possibilities existed for the application of EMI. For example, EMI would enable Dow
engineers to ascertain which of the multiple build-to-order configurations for a new chemical
compound would have the least impact on manufacturing costs. Fine-grained data from EMI
could also provide insights on developing substitutes for and variants of existing products.
Real-time data could enable plant operators to be proactive rather than reactive, potentially
eliminating their use of non-valuable data in situations where data collection had become a
habit.

Another potential benefit of EMI was maintaining consistency in practices when dealing with
issues. Plant supervisors at Dow factories around the world often interpreted similar data
differently owing to different experiences, educational backgrounds, and even personal beliefs,
 which often bordered on biases. With the adoption of well-developed algorithms to process
data and a common dashboard to display the results, EMI could ensure that plant supervisors
would reach identical conclusions on witnessing identical process behaviors.

EMI was also necessary from the point of view of Dow’s customers, who were large and
small businesses in a variety of industries. For example, a customer making personal care
products expected its raw material vendors to provide historical evidence about the sources of
its chemical ingredients, and EMI facilitated compliance with regulations around product
safety, which was crucial in the personal care products industry. It also enabled its customers in
the pharmaceuticals industry to comply with pharmacopeia rules on plant monitoring as part of
drug regulatory requirements.

Multi-plant acceleration of services can be used, to achieve collaboration-based knowledge.

The engineers ought to integrate core EMI functionalities. It could be role-based dashboards,
real time aggregation of data, direct data source linkage, alarm and notification. As the
company has high manufacturing capabilities, there could be different times of data collection
and volume. The Multi-plant alternative is kind of a realistic approach that could be
implemented in the company’s data analytics requirements.

9. What are your recommendations to Colegrove for implementing EMI?

EMI assists companies like Dow by providing data that may be used to choose products and
production strategies. This makes it possible for firms to select more sensible products and
production processes. By collecting information at the sources, businesses may get the most
reliable information possible and make better decisions about their goods. This helpful resource
may help manufacturers remain up to date on their goods and the tendencies that are important
to them.

This may be used, for example, to improve the accuracy of production data. By collecting and
analysing data from many sources, businesses may better understand the production system and
the information they are using to make decisions. someplace inside the company Using big data
in manufacturing may also be used to improve production data in order to comprehend client
wants and identify unauthorised activities.